diff --git a/documentation/webdocs/markdowndocs/Connector.md b/documentation/webdocs/markdowndocs/Connector.md index a0433d1f09d7c5f2ec1205f89d2efe638703dc7d..563d306128d5fe27959b67d845aab4ae66988a80 100644 --- a/documentation/webdocs/markdowndocs/Connector.md +++ b/documentation/webdocs/markdowndocs/Connector.md @@ -2,6 +2,8 @@ TDengine provides many connectors for development, including C/C++, JAVA, Python, RESTful, Go, Node.JS, etc. +NOTE: All APIs which require a SQL string as parameter, including but not limit to `taos_query`, `taos_query_a`, `taos_subscribe` in the C/C++ Connector and their counterparts in other connectors, can ONLY process one SQL statement at a time. If more than one SQL statements are provided, their behaviors are undefined. + ## C/C++ API C/C++ APIs are similar to the MySQL APIs. Applications should include TDengine head file _taos.h_ to use C/C++ APIs by adding the following line in code: diff --git a/documentation/webdocs/markdowndocs/Super Table-ch.md b/documentation/webdocs/markdowndocs/Super Table-ch.md index 38e6f8c17f900ad9597366f5131b3f1eebeb8863..626f695c27a128e48353d4c1f674c273107b1fbc 100644 --- a/documentation/webdocs/markdowndocs/Super Table-ch.md +++ b/documentation/webdocs/markdowndocs/Super Table-ch.md @@ -53,10 +53,11 @@ STable从属于库，一个STable只属于一个库，但一个库可以有一 说明： - 1. TAGS列总长度不能超过512 bytes； + 1. TAGS列总长度不能超过64k bytes； 2. TAGS列的数据类型不能是timestamp； 3. TAGS列名不能与其他列名相同; 4. TAGS列名不能为预留关键字. + 5. TAGS总数的上限是128. - 显示已创建的超级表 @@ -114,7 +115,7 @@ INSERT INTO USING TAGS (, ...) VALUES ( ADD TAG ``` - 为STable增加一个新的标签，并指定新标签的类型。标签总数不能超过6个。 + 为STable增加一个新的标签，并指定新标签的类型。标签总数不能超过128个。 - 删除标签 @@ -202,7 +203,7 @@ INSERT INTO therm4 VALUES ('2018-01-01 00:00:00.000', 23); ###3:按标签聚合查询 -查询位于北京(beijing)和天津(tianjing)两个地区的温度传感器采样值的数量count(*)、平均温度avg(degree)、最高温度max(degree)、最低温度min(degree)，并将结果按所处地域(location)和传感器类型(type)进行聚合。 +查询位于北京(beijing)和天津(tianjin)两个地区的温度传感器采样值的数量count(*)、平均温度avg(degree)、最高温度max(degree)、最低温度min(degree)，并将结果按所处地域(location)和传感器类型(type)进行聚合。 ```mysql SELECT COUNT(*), AVG(degree), MAX(degree), MIN(degree) diff --git a/documentation/webdocs/markdowndocs/Super Table.md b/documentation/webdocs/markdowndocs/Super Table.md index efc95c5f79216b2a887f26a565b4e5e123768c6b..6c80c2097c32aba7efdd5bcfad4c129a11cdb577 100644 --- a/documentation/webdocs/markdowndocs/Super Table.md +++ b/documentation/webdocs/markdowndocs/Super Table.md @@ -22,11 +22,11 @@ New keyword "tags" is introduced, where tag_name is the tag name, and tag_type i Note： -1. The bytes of all tags together shall be less than 512 +1. The bytes of all tags together shall be less than 64k 2. Tag's data type can not be time stamp 3. Tag name shall be different from the field name 4. Tag name shall not be the same as system keywords -5. Maximum number of tags is 6 +5. Maximum number of tags is 128 For example: @@ -168,7 +168,7 @@ You can add, delete and change the tags for a STable, and you can change the tag ALTER TABLE ADD TAG ``` -It adds a new tag to the STable with a data type. The maximum number of tags is 6. +It adds a new tag to the STable with a data type. The maximum number of tags is 128. ### Drop a Tag diff --git a/documentation/webdocs/markdowndocs/TAOS SQL-ch.md b/documentation/webdocs/markdowndocs/TAOS SQL-ch.md index cd184cbc71b36ee8cac3738c4f28772547c9d8c0..81fc3ffcc428d20e92506f25521026fb70a643ef 100644 --- a/documentation/webdocs/markdowndocs/TAOS SQL-ch.md +++ b/documentation/webdocs/markdowndocs/TAOS SQL-ch.md @@ -63,7 +63,7 @@ TDengine缺省的时间戳是毫秒精度，但通过修改配置参数enableMic | 3 | BIGINT | 8 | 长整型，范围 [-2^63+1, 2^63-1], -2^63用于NULL | | 4 | FLOAT | 4 | 浮点型，有效位数6-7，范围 [-3.4E38, 3.4E38] | | 5 | DOUBLE | 8 | 双精度浮点型，有效位数15-16，范围 [-1.7E308, 1.7E308] | -| 6 | BINARY | 自定义 | 用于记录字符串，最长不能超过504 bytes。binary仅支持字符串输入，字符串两端使用单引号引用，否则英文全部自动转化为小写。使用时须指定大小，如binary(20)定义了最长为20个字符的字符串，每个字符占1byte的存储空间。如果用户字符串超出20字节，将被自动截断。对于字符串内的单引号，可以用转义字符反斜线加单引号来表示， 即 **\’**。 | +| 6 | BINARY | 自定义 | 用于记录字符串，理论上，最长可以有65526字节，但由于每行数据最多64K字节，实际上限一般小于理论值。 binary仅支持字符串输入，字符串两端使用单引号引用，否则英文全部自动转化为小写。使用时须指定大小，如binary(20)定义了最长为20个字符的字符串，每个字符占1byte的存储空间。如果用户字符串超出20字节，将被自动截断。对于字符串内的单引号，可以用转义字符反斜线加单引号来表示， 即 **\’**。 | | 7 | SMALLINT | 2 | 短整型， 范围 [-32767, 32767], -32768用于NULL | | 8 | TINYINT | 1 | 单字节整型，范围 [-127, 127], -128用于NULL | | 9 | BOOL | 1 | 布尔型，{true, false} | @@ -106,7 +106,7 @@ TDengine缺省的时间戳是毫秒精度，但通过修改配置参数enableMic ```mysql CREATE TABLE [IF NOT EXISTS] tb_name (timestamp_field_name TIMESTAMP, field1_name data_type1 [, field2_name data_type2 ...]) ``` - 说明：1）表的第一个字段必须是TIMESTAMP，并且系统自动将其设为主键；2）表的每行长度不能超过4096字节；3）使用数据类型binary或nchar，需指定其最长的字节数，如binary(20)，表示20字节。 + 说明：1）表的第一个字段必须是TIMESTAMP，并且系统自动将其设为主键；2）表的每行长度不能超过64K字节；3）使用数据类型binary或nchar，需指定其最长的字节数，如binary(20)，表示20字节。 - **删除数据表** @@ -402,7 +402,7 @@ count(tbname) | SELECT * FROM tb1 WHERE ts >= NOW - 1h ``` -- 查询表tb1从2018-06-01 08:00:00.000 到2018-06-02 08:00:00.000时间范围，并且clo3的字符串是'nny'结尾的记录，结果按照时间戳降序 +- 查询表tb1从2018-06-01 08:00:00.000 到2018-06-02 08:00:00.000时间范围，并且col3的字符串是'nny'结尾的记录，结果按照时间戳降序 ```mysql SELECT * FROM tb1 WHERE ts > '2018-06-01 08:00:00.000' AND ts <= '2018-06-02 08:00:00.000' AND col3 LIKE '%nny' ORDER BY ts DESC diff --git a/documentation/webdocs/markdowndocs/TAOS SQL.md b/documentation/webdocs/markdowndocs/TAOS SQL.md index 99aa73b4354c60fb3c50ff8b3d2454eb6691c340..2b9960b860f5452b1b233355984a411b2c4a90a4 100644 --- a/documentation/webdocs/markdowndocs/TAOS SQL.md +++ b/documentation/webdocs/markdowndocs/TAOS SQL.md @@ -39,7 +39,7 @@ The full list of data types is listed below. For string types of data, we will | 6 | DOUBLE | 8 | A standard nullable double float type with 15-16 significant digits and a range of [-1.7E308, 1.7E308]​ | | 7 | BOOL | 1 | A nullable boolean type, [**`true`**, **`false`**] | | 8 | TIMESTAMP | 8 | A nullable timestamp type with the same usage as the primary column timestamp | -| 9 | BINARY(*M*) | *M* | A nullable string type whose length is *M*, any exceeded chars will be automatically truncated. This type of string only supports ASCii encoded chars. | +| 9 | BINARY(*M*) | *M* | A nullable string type whose length is *M*, any exceeded chars will be automatically truncated, the maximum length of *M* is 65526, but as maximum row size is 64K bytes, the actual upper limit will generally less than 65526. This type of string only supports ASCii encoded chars. | | 10 | NCHAR(*M*) | 4 * *M* | A nullable string type whose length is *M*, any exceeded chars will be truncated. The **`NCHAR`** type supports Unicode encoded chars. | All the keywords in a SQL statement are case-insensitive, but strings values are case-sensitive and must be quoted by a pair of `'` or `"`. To quote a `'` or a `"` , you can use the escape character `\`. @@ -86,7 +86,7 @@ All the keywords in a SQL statement are case-insensitive, but strings values are 1) The first column must be a `timestamp`, and the system will set it as the primary key. - 2) The record size is limited to 4096 bytes + 2) The record size is limited to 64k bytes 3) For `binary` or `nchar` data types, the length must be specified. For example, binary(20) means a binary data type with 20 bytes. diff --git a/documentation/webdocs/markdowndocs/connector-ch.md b/documentation/webdocs/markdowndocs/connector-ch.md index b5d8fb5afb12ede82f2cdcd9ea29e20e8a82d6b8..3cd8da1bd50d6e7b52ca9eaa5b276c8cb430202c 100644 --- a/documentation/webdocs/markdowndocs/connector-ch.md +++ b/documentation/webdocs/markdowndocs/connector-ch.md @@ -2,6 +2,8 @@ TDengine提供了丰富的应用程序开发接口，其中包括C/C++、JAVA、Python、RESTful、Go等，便于用户快速开发应用。 +注意：所有执行 SQL 语句的 API，例如 C/C++ Connector 中的 `tao_query`、`taos_query_a`、`taos_subscribe` 等，以及其它语言中与它们对应的API，每次都只能执行一条 SQL 语句，如果实际参数中包含了多条语句，它们的行为是未定义的。 + ## C/C++ Connector C/C++的API类似于MySQL的C API。应用程序使用时，需要包含TDengine头文件 _taos.h_（安装后，位于 _/usr/local/taos/include_）： diff --git a/importSampleData/app/main.go b/importSampleData/app/main.go index aef413320710012fec79e56677e16864a881ca8f..d714fc339c177f8a22f46215f5ef9ce3f2ade825 100644 --- a/importSampleData/app/main.go +++ b/importSampleData/app/main.go @@ -18,7 +18,7 @@ import ( "sync" "time" - _ "github.com/taosdata/TDengine/src/connector/go/src/taosSql" + _ "github.com/taosdata/TDengine/src/connector/go/taosSql" ) const ( @@ -634,6 +634,7 @@ func insertData(threadIndex, start, end int, wg *sync.WaitGroup, successRows [] if appendRows == batch { // executebatch insertSql := buffers.String() + connection.Exec("use " + db) affectedRows := executeBatchInsert(insertSql, connection) successRows[threadIndex] += affectedRows @@ -658,6 +659,7 @@ func insertData(threadIndex, start, end int, wg *sync.WaitGroup, successRows [] if appendRows > 0 { // executebatch insertSql := buffers.String() + connection.Exec("use " + db) affectedRows := executeBatchInsert(insertSql, connection) successRows[threadIndex] += affectedRows diff --git a/importSampleData/bin/taosimport b/importSampleData/bin/taosimport index b042549341bced364e0fd77909b115d1b5b6dc04..235fde9f06ce00cde32057c0a4a91060dd771b41 100755 Binary files a/importSampleData/bin/taosimport and b/importSampleData/bin/taosimport differ diff --git a/packaging/cfg/taos.cfg b/packaging/cfg/taos.cfg index 38e960eb0679453c3d7cd624ceed5616b0b5deb7..0d2ebf9eda8c5fd97ea0c62c76582a079f57d90b 100644 --- a/packaging/cfg/taos.cfg +++ b/packaging/cfg/taos.cfg @@ -155,7 +155,7 @@ # maxVnodeConnections 10000 # mnode take into account while balance, for cluster version only -# mgmtEqualVnodeNum 4 +# mnodeEqualVnodeNum 4 # number of seconds allowed for a dnode to be offline, for cluster version only # offlineThreshold 864000 diff --git a/src/client/inc/tsclient.h b/src/client/inc/tsclient.h index 08536a505d0a260074f14379fffa79024a6905f1..c04c31dfb7659e9c2e494650a96980f5edc9235d 100644 --- a/src/client/inc/tsclient.h +++ b/src/client/inc/tsclient.h @@ -284,8 +284,8 @@ typedef struct { typedef struct STscObj { void * signature; void * pTimer; - char mgmtIp[TSDB_USER_LEN]; - uint16_t mgmtPort; + char mnodeIp[TSDB_USER_LEN]; + uint16_t mnodePort; char user[TSDB_USER_LEN]; char pass[TSDB_KEY_LEN]; char acctId[TSDB_DB_NAME_LEN]; diff --git a/src/client/src/TSDBJNIConnector.c b/src/client/src/TSDBJNIConnector.c index 802d383152734e584a0694b61cb226c1bc1328a3..6ab1b73d1e8fec5c910d17f36110e95ecc5f12d8 100644 --- a/src/client/src/TSDBJNIConnector.c +++ b/src/client/src/TSDBJNIConnector.c @@ -23,7 +23,7 @@ #include "ttime.h" #define jniError(...) { if (jniDebugFlag & DEBUG_ERROR) { taosPrintLog("ERROR JNI ", jniDebugFlag, __VA_ARGS__); }} -#define jniWarn(...) { if (jniDebugFlag & DEBUG_WARN) { taosPrintLog("WARN JNI ", jniDebugFlag, __VA_ARGS__); }} +#define jniWarn(...) { if (jniDebugFlag & DEBUG_WARN) { taosPrintLog("WARN JNI ", jniDebugFlag, __VA_ARGS__); }} #define jniTrace(...) { if (jniDebugFlag & DEBUG_TRACE) { taosPrintLog("JNI ", jniDebugFlag, __VA_ARGS__); }} #define jniPrint(...) { taosPrintLog("JNI ", tscEmbedded ? 255 : uDebugFlag, __VA_ARGS__); } diff --git a/src/client/src/tscParseInsert.c b/src/client/src/tscParseInsert.c index 9202203fac0654eac249e52cd00f60cf8734b963..79872e22c8bf88e13e45714bbeeceec9b7cf6914 100644 --- a/src/client/src/tscParseInsert.c +++ b/src/client/src/tscParseInsert.c @@ -1361,7 +1361,7 @@ int tsParseSql(SSqlObj *pSql, bool initialParse) { /* * the pRes->code may be modified or released by another thread in tscTableMetaCallBack function, * so do NOT use pRes->code to determine if the getTableMeta/getMetricMeta function - * invokes new threads to get data from mgmt node or simply retrieves data from cache. + * invokes new threads to get data from mnode or simply retrieves data from cache. * * do NOT assign return code to pRes->code for the same reason since it may be released by another thread * pRes->code = ret; diff --git a/src/client/src/tscServer.c b/src/client/src/tscServer.c index 7d0ac09e66c14160f4dd715f0c44cec40674549c..9123318d3bfa181bb4a21db12d6e8095c620fc4f 100644 --- a/src/client/src/tscServer.c +++ b/src/client/src/tscServer.c @@ -60,10 +60,10 @@ static void tscSetDnodeIpList(SSqlObj* pSql, SCMVgroupInfo* pVgroupInfo) { void tscPrintMgmtIp() { if (tscMgmtIpSet.numOfIps <= 0) { - tscError("invalid mgmt IP list:%d", tscMgmtIpSet.numOfIps); + tscError("invalid mnode IP list:%d", tscMgmtIpSet.numOfIps); } else { for (int i = 0; i < tscMgmtIpSet.numOfIps; ++i) { - tscTrace("mgmt index:%d %s:%d", i, tscMgmtIpSet.fqdn[i], tscMgmtIpSet.port[i]); + tscTrace("mnode index:%d %s:%d", i, tscMgmtIpSet.fqdn[i], tscMgmtIpSet.port[i]); } } } @@ -78,7 +78,7 @@ void tscSetMgmtIpList(SRpcIpSet *pIpList) { void tscUpdateIpSet(void *ahandle, SRpcIpSet *pIpSet) { tscMgmtIpSet = *pIpSet; - tscTrace("mgmt IP list is changed for ufp is called, numOfIps:%d inUse:%d", tscMgmtIpSet.numOfIps, tscMgmtIpSet.inUse); + tscTrace("mnode IP list is changed for ufp is called, numOfIps:%d inUse:%d", tscMgmtIpSet.numOfIps, tscMgmtIpSet.inUse); for (int32_t i = 0; i < tscMgmtIpSet.numOfIps; ++i) { tscTrace("index:%d fqdn:%s port:%d", i, tscMgmtIpSet.fqdn[i], tscMgmtIpSet.port[i]); } @@ -217,10 +217,12 @@ void tscProcessMsgFromServer(SRpcMsg *rpcMsg, SRpcIpSet *pIpSet) { STscObj *pObj = pSql->pTscObj; // tscTrace("%p msg:%s is received from server", pSql, taosMsg[rpcMsg->msgType]); - if (pSql->freed || pObj->signature != pObj) { + if (pObj->signature != pObj) { tscTrace("%p sql is already released or DB connection is closed, freed:%d pObj:%p signature:%p", pSql, pSql->freed, pObj, pObj->signature); - tscFreeSqlObj(pSql); + if (pSql != pObj->pSql) { + tscFreeSqlObj(pSql); + } rpcFreeCont(rpcMsg->pCont); return; } @@ -235,10 +237,8 @@ void tscProcessMsgFromServer(SRpcMsg *rpcMsg, SRpcIpSet *pIpSet) { rpcMsg->code = TSDB_CODE_NETWORK_UNAVAIL; } else { STableMetaInfo *pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, pCmd->clauseIndex, 0); - if (rpcMsg->code == TSDB_CODE_NOT_ACTIVE_TABLE || rpcMsg->code == TSDB_CODE_INVALID_TABLE_ID || - rpcMsg->code == TSDB_CODE_INVALID_VNODE_ID || rpcMsg->code == TSDB_CODE_NOT_ACTIVE_VNODE || - rpcMsg->code == TSDB_CODE_NETWORK_UNAVAIL || rpcMsg->code == TSDB_CODE_NOT_ACTIVE_TABLE || - rpcMsg->code == TSDB_CODE_TABLE_ID_MISMATCH) { + if (rpcMsg->code == TSDB_CODE_INVALID_TABLE_ID || rpcMsg->code == TSDB_CODE_INVALID_VGROUP_ID || + rpcMsg->code == TSDB_CODE_NETWORK_UNAVAIL) { /* * not_active_table: 1. the virtual node may fail to create table, since the procedure of create table is asynchronized, * the virtual node may have not create table till now, so try again by using the new metermeta. @@ -1867,8 +1867,8 @@ int tscProcessTableMetaRsp(SSqlObj *pSql) { return TSDB_CODE_CLI_OUT_OF_MEMORY; } - free(pTableMeta); tscTrace("%p recv table meta: %"PRId64 ", tid:%d, name:%s", pSql, pTableMeta->uid, pTableMeta->sid, pTableMetaInfo->name); + free(pTableMeta); return TSDB_CODE_SUCCESS; } diff --git a/src/client/src/tscSql.c b/src/client/src/tscSql.c index 14605a571deefbf5543de7ebc57287f6a2eedaac..1ab14a4eae1758652f896436bbc40efc1133f10c 100644 --- a/src/client/src/tscSql.c +++ b/src/client/src/tscSql.c @@ -88,7 +88,7 @@ STscObj *taosConnectImpl(const char *ip, const char *user, const char *pass, con strncpy(pObj->user, user, TSDB_USER_LEN); taosEncryptPass((uint8_t *)pass, strlen(pass), pObj->pass); - pObj->mgmtPort = port ? port : tsDnodeShellPort; + pObj->mnodePort = port ? port : tsDnodeShellPort; if (db) { int32_t len = strlen(db); @@ -576,25 +576,24 @@ void taos_free_result_imp(TAOS_RES *res, int keepCmd) { * for each subquery. Because the failure of execution tsProcessSql may trigger the callback function * be executed, and the retry efforts may result in double free the resources, e.g.,SRetrieveSupport */ - if ((pCmd->command == TSDB_SQL_SELECT || pCmd->command == TSDB_SQL_SHOW || pCmd->command == TSDB_SQL_RETRIEVE || - pCmd->command == TSDB_SQL_FETCH) && - (pRes->code != TSDB_CODE_QUERY_CANCELLED && ((pRes->numOfRows > 0 && pCmd->command < TSDB_SQL_LOCAL && pRes->completed == false) || - (pRes->code == TSDB_CODE_SUCCESS && pRes->numOfRows == 0 && pCmd->command == TSDB_SQL_SELECT && - pSql->pStream == NULL && pTableMetaInfo->pTableMeta != NULL)))) { + if ((pCmd->command == TSDB_SQL_SELECT || + pCmd->command == TSDB_SQL_SHOW || + pCmd->command == TSDB_SQL_RETRIEVE || + pCmd->command == TSDB_SQL_FETCH) && pRes->code == TSDB_CODE_SUCCESS && + ((pCmd->command < TSDB_SQL_LOCAL && pRes->completed == false) || + (pCmd->command == TSDB_SQL_SELECT && pSql->pStream == NULL && pTableMetaInfo->pTableMeta != NULL))) { pCmd->command = (pCmd->command > TSDB_SQL_MGMT) ? TSDB_SQL_RETRIEVE : TSDB_SQL_FETCH; - tscTrace("%p code:%d, numOfRows:%d, command:%d", pSql, pRes->code, pRes->numOfRows, pCmd->command); + tscTrace("%p send msg to free qhandle in vnode, code:%d, numOfRows:%d, command:%s", pSql, pRes->code, pRes->numOfRows, + sqlCmd[pCmd->command]); pSql->freed = 1; tscProcessSql(pSql); - - /* - * If release connection msg is sent to vnode, the corresponding SqlObj for async query can not be freed instantly, - * since its free operation is delegated to callback function, which is tscProcessMsgFromServer. - */ - STscObj* pObj = pSql->pTscObj; - if (pObj->pSql == pSql) { - pObj->pSql = NULL; + + // waits for response and then goes on + STscObj* pTscObj = pSql->pTscObj; + if (pTscObj->pSql == pSql) { + sem_wait(&pSql->rspSem); } } else { // if no free resource msg is sent to vnode, we free this object immediately. STscObj* pTscObj = pSql->pTscObj; diff --git a/src/client/src/tscSubquery.c b/src/client/src/tscSubquery.c index 3a205924bf934ddb3ddf85604f349cbd9431c85c..c4fe0b202bc7a05163bd25f6f2cb17482b450781 100644 --- a/src/client/src/tscSubquery.c +++ b/src/client/src/tscSubquery.c @@ -1763,7 +1763,12 @@ int32_t tscHandleMultivnodeInsert(SSqlObj *pSql) { tscError("%p failed to malloc buffer for subObj, orderOfSub:%d, reason:%s", pSql, i, strerror(errno)); break; } - + + /* + * assign the callback function to fetchFp to make sure that the error process function can restore + * the callback function (multiVnodeInsertMerge) correctly. + */ + pNew->fetchFp = pNew->fp; pSql->pSubs[i] = pNew; tscTrace("%p sub:%p create subObj success. orderOfSub:%d", pSql, pNew, i); } @@ -1888,11 +1893,14 @@ static void transferNcharData(SSqlObj *pSql, int32_t columnIndex, TAOS_FIELD *pF /* string terminated char for binary data*/ memset(pRes->buffer[columnIndex], 0, pField->bytes + TSDB_NCHAR_SIZE); - if (taosUcs4ToMbs(pRes->tsrow[columnIndex], pField->bytes - VARSTR_HEADER_SIZE, pRes->buffer[columnIndex])) { + int32_t length = taosUcs4ToMbs(pRes->tsrow[columnIndex], pRes->length[columnIndex], pRes->buffer[columnIndex]); + if ( length >= 0 ) { pRes->tsrow[columnIndex] = pRes->buffer[columnIndex]; + pRes->length[columnIndex] = length; } else { tscError("%p charset:%s to %s. val:%ls convert failed.", pSql, DEFAULT_UNICODE_ENCODEC, tsCharset, pRes->tsrow[columnIndex]); pRes->tsrow[columnIndex] = NULL; + pRes->length[columnIndex] = 0; } } } diff --git a/src/client/src/tscSystem.c b/src/client/src/tscSystem.c index 5d56fef1e9c9e3ea1a8f4ba6215fcd816b931011..75249e44eefd86f1e8ff1cc30e230ed01c5fcac4 100644 --- a/src/client/src/tscSystem.c +++ b/src/client/src/tscSystem.c @@ -57,9 +57,9 @@ int32_t tscInitRpc(const char *user, const char *secret, void** pDnodeConn) { memset(&rpcInit, 0, sizeof(rpcInit)); rpcInit.localPort = 0; rpcInit.label = "TSC"; - rpcInit.numOfThreads = tscNumOfThreads; + rpcInit.numOfThreads = 1; // every DB connection has only one thread rpcInit.cfp = tscProcessMsgFromServer; - rpcInit.sessions = tsMaxVnodeConnections; + rpcInit.sessions = tsMaxConnections; rpcInit.connType = TAOS_CONN_CLIENT; rpcInit.user = (char*)user; rpcInit.idleTime = 2000; @@ -116,12 +116,12 @@ void taos_init_imp() { } if (tscSetMgmtIpListFromCfg(tsFirst, tsSecond) < 0) { - tscError("failed to init mgmt IP list"); + tscError("failed to init mnode IP list"); return; } tscInitMsgsFp(); - int queueSize = tsMaxVnodeConnections + tsMaxMeterConnections + tsMaxMgmtConnections + tsMaxMgmtConnections; + int queueSize = tsMaxConnections*2; if (tscEmbedded == 0) { tscNumOfThreads = tsNumOfCores * tsNumOfThreadsPerCore / 2.0; @@ -137,7 +137,7 @@ void taos_init_imp() { return; } - tscTmr = taosTmrInit(tsMaxMgmtConnections * 2, 200, 60000, "TSC"); + tscTmr = taosTmrInit(tsMaxConnections * 2, 200, 60000, "TSC"); if(0 == tscEmbedded){ taosTmrReset(tscCheckDiskUsage, 10, NULL, tscTmr, &tscCheckDiskUsageTmr); } diff --git a/src/client/src/tscUtil.c b/src/client/src/tscUtil.c index ec9908ae969bbcb8d65c8c59603899bcfe9bdcf8..1925546222520e8714cf8cc46fc11ef6788e4eb6 100644 --- a/src/client/src/tscUtil.c +++ b/src/client/src/tscUtil.c @@ -423,6 +423,8 @@ void tscFreeSqlObj(SSqlObj* pSql) { tfree(pCmd->payload); pCmd->allocSize = 0; + + tfree(pSql->sqlstr); free(pSql); } @@ -1848,8 +1850,7 @@ SSqlObj* createSubqueryObj(SSqlObj* pSql, int16_t tableIndex, void (*fp)(), void STableMetaInfo* pFinalInfo = NULL; if (pPrevSql == NULL) { - STableMeta* pTableMeta = taosCacheAcquireByName(tscCacheHandle, name); - // todo handle error + STableMeta* pTableMeta = taosCacheAcquireByData(tscCacheHandle, pTableMetaInfo->pTableMeta); // get by name may failed due to the cache cleanup assert(pTableMeta != NULL); pFinalInfo = tscAddTableMetaInfo(pNewQueryInfo, name, pTableMeta, pTableMetaInfo->vgroupList, pTableMetaInfo->tagColList); } else { // transfer the ownership of pTableMeta to the newly create sql object. diff --git a/src/common/inc/tdataformat.h b/src/common/inc/tdataformat.h index 8990b69c5aee57dfcfad696e06dc982574aac6b4..528e9b2825befe74beb8381f6291e2b9dda66b1e 100644 --- a/src/common/inc/tdataformat.h +++ b/src/common/inc/tdataformat.h @@ -217,6 +217,59 @@ void tdPopDataColsPoints(SDataCols *pCols, int pointsToPop); //!!!! int tdMergeDataCols(SDataCols *target, SDataCols *src, int rowsToMerge); void tdMergeTwoDataCols(SDataCols *target, SDataCols *src1, int *iter1, SDataCols *src2, int *iter2, int tRows); + +// ----------------- Tag row structure + +/* A tag row, the format is like below: ++----------+----------------------------------------------------------------+ +| STagRow | STagCol | STagCol | STagCol | STagCol | ...| STagCol | STagCol | ++----------+----------------------------------------------------------------+ + +pData ++----------+----------------------------------------------------------------+ +| value 1 | value 2 | value 3 | value 4 | ....|value n | ++----------+----------------------------------------------------------------+ + + */ + + +#define TD_TAG_ROW_HEAD_SIZE sizeof(int16_t) + +#define tagRowNum(r) (*(int16_t *)(r)) +#define tagRowArray(r) POINTER_SHIFT(r, TD_TAG_ROW_HEAD_SIZE) +//#define dataRowKey(r) (*(TSKEY *)(dataRowTuple(r))) +//#define dataRowSetLen(r, l) (dataRowLen(r) = (l)) +//#define dataRowCpy(dst, r) memcpy((dst), (r), dataRowLen(r)) +//#define dataRowMaxBytesFromSchema(s) (schemaTLen(s) + TD_DATA_ROW_HEAD_SIZE) + +typedef struct { + int16_t colId; // column ID + int16_t colType; + uint16_t offset; //to store value for numeric col or offset for binary/Nchar +} STagCol; + +typedef struct { + int32_t len; + void * pData; // Space to store the tag value + uint16_t dataLen; + int16_t ncols; // Total columns allocated + STagCol tagCols[]; +} STagRow; + + +#define tagColSize(r) (sizeof(STagCol) + r.colLen) + +int tdSetTagCol(SDataRow row, void *value, int16_t len, int8_t type, int16_t colId); //insert tag value and update all the information +int tdDeleteTagCol(SDataRow row, int16_t colId); // delete tag value and update all the information +void * tdQueryTagByID(SDataRow row, int16_t colId, int16_t *type); //if find tag, 0, else return -1; +int tdAppendTagColVal(SDataRow row, void *value, int8_t type, int32_t bytes, int16_t colId); +SDataRow tdTagRowDup(SDataRow row); +void tdFreeTagRow(SDataRow row); +SDataRow tdTagRowDecode(SDataRow row); +int tdTagRowCpy(SDataRow dst, SDataRow src); +void * tdNewTagRowFromSchema(STSchema *pSchema, int16_t numofTags); +STSchema *tdGetSchemaFromData(SDataRow *row); + #ifdef __cplusplus } #endif diff --git a/src/common/inc/tglobal.h b/src/common/inc/tglobal.h index 319772b60652d6e567e9aa70d06cbffb8167b218..0af8430ab176ef266f77230e6500fb7a46ad3165 100644 --- a/src/common/inc/tglobal.h +++ b/src/common/inc/tglobal.h @@ -87,20 +87,18 @@ extern int16_t tsWAL; extern int32_t tsReplications; extern int16_t tsAffectedRowsMod; -extern int32_t tsNumOfMPeers; +extern int32_t tsNumOfMnodes; extern int32_t tsMaxShellConns; extern int32_t tsMaxTables; extern char tsDefaultDB[]; extern char tsDefaultUser[]; extern char tsDefaultPass[]; -extern int32_t tsMaxMeterConnections; -extern int32_t tsMaxVnodeConnections; -extern int32_t tsMaxMgmtConnections; +extern int32_t tsMaxConnections; extern int32_t tsBalanceInterval; extern int32_t tsOfflineThreshold; -extern int32_t tsMgmtEqualVnodeNum; +extern int32_t tsMnodeEqualVnodeNum; extern int32_t tsEnableHttpModule; extern int32_t tsEnableMqttModule; diff --git a/src/common/inc/tulog.h b/src/common/inc/tulog.h index a47e894e311245ca2c0006639e72bb98204e9984..07120d7cbe55e5d0dbeef89cc88acc065921b1de 100644 --- a/src/common/inc/tulog.h +++ b/src/common/inc/tulog.h @@ -26,7 +26,7 @@ extern int32_t uDebugFlag; extern int32_t tscEmbedded; #define uError(...) { if (uDebugFlag & DEBUG_ERROR) { taosPrintLog("ERROR UTL ", uDebugFlag, __VA_ARGS__); }} -#define uWarn(...) { if (uDebugFlag & DEBUG_WARN) { taosPrintLog("WARN UTL ", uDebugFlag, __VA_ARGS__); }} +#define uWarn(...) { if (uDebugFlag & DEBUG_WARN) { taosPrintLog("WARN UTL ", uDebugFlag, __VA_ARGS__); }} #define uTrace(...) { if (uDebugFlag & DEBUG_TRACE) { taosPrintLog("UTL ", uDebugFlag, __VA_ARGS__); }} #define uDump(x, y) { if (uDebugFlag & DEBUG_DUMP) { taosDumpData(x, y); }} #define uPrint(...) { taosPrintLog("UTL ", tscEmbedded ? 255 : uDebugFlag, __VA_ARGS__); } diff --git a/src/common/src/tdataformat.c b/src/common/src/tdataformat.c index 81ea801c944011443b4d0f341427dadfd85f8153..922c8bdea0a98e624497838006f4dcf2b3bea087 100644 --- a/src/common/src/tdataformat.c +++ b/src/common/src/tdataformat.c @@ -14,6 +14,7 @@ */ #include "tdataformat.h" #include "wchar.h" +#include "talgo.h" /** * Create a SSchema object with nCols columns @@ -151,6 +152,151 @@ SDataRow tdNewDataRowFromSchema(STSchema *pSchema) { return row; } +int tdSetTagCol(SDataRow row, void *value, int16_t len, int8_t type, int16_t colId){ //insert/update tag value and update all the information + ASSERT(((STagRow *)row)->pData != NULL); + //STagCol * stCol = tdQueryTagColByID() + + return 0; +}; + +int tdDeleteTagCol(SDataRow row, int16_t colId){ // delete tag value and update all the information + //todo + return 0; +}; + +static int compTagId(const void *key1, const void *key2) { + if (((STagCol *)key1)->colId > ((STagCol *)key2)->colId) { + return 1; + } else if (((STagCol *)key1)->colId == ((STagCol *)key2)->colId) { + return 0; + } else { + return -1; + } +} + +/** + * Find tag structure by colId, if find, return tag structure, else return NULL; + */ +STagCol * tdQueryTagColByID(SDataRow row, int16_t colId, int flags) { //if find tag, 0, else return -1; + ASSERT(((STagRow *)row)->pData != NULL); + STagCol *pBase = ((STagRow *)row)->tagCols; + int16_t nCols = ((STagRow *)row)->ncols; + STagCol key = {colId,0,0}; + STagCol * stCol = taosbsearch(&key, pBase, nCols, sizeof(STagCol), compTagId, flags); + return stCol; +}; + +/** +* Find tag value by colId, if find, return tag value, else return NULL; +*/ +void * tdQueryTagByID(SDataRow row, int16_t colId, int16_t *type) { + ASSERT(((STagRow *)row)->pData != NULL); + STagCol *pBase = ((STagRow *)row)->tagCols; + int16_t nCols = ((STagRow *)row)->ncols; + STagCol key = {colId,0,0}; + STagCol * stCol = taosbsearch(&key, pBase, nCols, sizeof(STagCol), compTagId, TD_EQ); + if (NULL == stCol) { + return NULL; + } + + void * pData = ((STagRow *)row)->pData; + *type = stCol->colType; + + return pData + stCol->offset; +}; + +int tdAppendTagColVal(SDataRow row, void *value, int8_t type, int32_t bytes, int16_t colId){ + ASSERT(value != NULL); + //ASSERT(bytes-2 == varDataTLen(value)); + ASSERT(row != NULL); + STagRow *pTagrow = row; + pTagrow->tagCols[pTagrow->ncols].colId = colId; + pTagrow->tagCols[pTagrow->ncols].colType = type; + pTagrow->tagCols[pTagrow->ncols].offset = pTagrow->dataLen; + + switch (type) { + case TSDB_DATA_TYPE_BINARY: + case TSDB_DATA_TYPE_NCHAR: + memcpy((char *)pTagrow->pData + pTagrow->dataLen, value, varDataTLen(value)); + pTagrow->dataLen += varDataTLen(value); + break; + default: + memcpy((char *)pTagrow->pData + pTagrow->dataLen, value, TYPE_BYTES[type]); + pTagrow->dataLen += TYPE_BYTES[type]; + break; + } + + pTagrow->ncols++; + + return 0; +}; + +void * tdNewTagRowFromSchema(STSchema *pSchema, int16_t numofTags) { + int32_t size = sizeof(STagRow) + numofTags * sizeof(STagCol); + + STagRow *row = malloc(size); + if (row == NULL) return NULL; + + int32_t datasize = pSchema->tlen; + row->pData = malloc(datasize); + if (NULL == row->pData) { + free(row); + return NULL; + } + + row->len = size; + row->dataLen = 0; + row->ncols = 0; + return row; +} +/** + * free tag row + */ + +void tdFreeTagRow(SDataRow row) { + if (row) { + free(((STagRow *)row)->pData); + free(row); + } +} + +SDataRow tdTagRowDup(SDataRow row) { + STagRow *trow = malloc(dataRowLen(row)); + if (trow == NULL) return NULL; + + dataRowCpy(trow, row); + trow->pData = malloc(trow->dataLen); + if (NULL == trow->pData) { + free(trow); + return NULL; + } + memcpy(trow->pData, ((STagRow *)row)->pData, trow->dataLen); + return trow; +} + +SDataRow tdTagRowDecode(SDataRow row) { + STagRow *trow = malloc(dataRowLen(row)); + if (trow == NULL) return NULL; + + dataRowCpy(trow, row); + trow->pData = malloc(trow->dataLen); + if (NULL == trow->pData) { + free(trow); + return NULL; + } + char * pData = (char *)row + dataRowLen(row); + memcpy(trow->pData, pData, trow->dataLen); + return trow; +} + +int tdTagRowCpy(SDataRow dst, SDataRow src) { + if (src == NULL) return -1; + + dataRowCpy(dst, src); + void * pData = dst + dataRowLen(src); + memcpy(pData, ((STagRow *)src)->pData, ((STagRow *)src)->dataLen); + return 0; +} /** * Free the SDataRow object */ diff --git a/src/common/src/tglobal.c b/src/common/src/tglobal.c index 763b3f5c22f6e055d0e0f8da5e55efa189a985f4..2b4e59e92addcf91ed0bc75f41b42b80e9f31f31 100644 --- a/src/common/src/tglobal.c +++ b/src/common/src/tglobal.c @@ -105,19 +105,17 @@ int32_t tsReplications = TSDB_DEFAULT_REPLICA_NUM; * 1: affected rows include those duplicate records */ int16_t tsAffectedRowsMod = 0; -int32_t tsNumOfMPeers = 3; -int32_t tsMaxShellConns = 2000; +int32_t tsNumOfMnodes = 3; +int32_t tsMaxShellConns = 5000; char tsDefaultDB[TSDB_DB_NAME_LEN] = {0}; char tsDefaultUser[64] = "root"; char tsDefaultPass[64] = "taosdata"; -int32_t tsMaxMeterConnections = 10000; -int32_t tsMaxMgmtConnections = 2000; -int32_t tsMaxVnodeConnections = 10000; +int32_t tsMaxConnections = 50; int32_t tsBalanceInterval = 300; // seconds int32_t tsOfflineThreshold = 86400*100; // seconds 10days -int32_t tsMgmtEqualVnodeNum = 4; +int32_t tsMnodeEqualVnodeNum = 4; int32_t tsEnableHttpModule = 1; int32_t tsEnableMqttModule = 0; // not finished yet, not started it by default @@ -407,8 +405,8 @@ static void doInitGlobalConfig() { cfg.unitType = TAOS_CFG_UTYPE_NONE; taosInitConfigOption(cfg); - cfg.option = "numOfMPeers"; - cfg.ptr = &tsNumOfMPeers; + cfg.option = "numOfMnodes"; + cfg.ptr = &tsNumOfMnodes; cfg.valType = TAOS_CFG_VTYPE_INT32; cfg.cfgType = TSDB_CFG_CTYPE_B_CONFIG | TSDB_CFG_CTYPE_B_SHOW; cfg.minValue = 1; @@ -427,7 +425,7 @@ static void doInitGlobalConfig() { cfg.unitType = TAOS_CFG_UTYPE_NONE; taosInitConfigOption(cfg); - // 0-any; 1-mgmt; 2-dnode + // 0-any; 1-mnode; 2-vnode cfg.option = "alternativeRole"; cfg.ptr = &tsAlternativeRole; cfg.valType = TAOS_CFG_VTYPE_INT32; @@ -607,7 +605,7 @@ static void doInitGlobalConfig() { cfg.minValue = TSDB_MIN_CACHE_BLOCK_SIZE; cfg.maxValue = TSDB_MAX_CACHE_BLOCK_SIZE; cfg.ptrLength = 0; - cfg.unitType = TAOS_CFG_UTYPE_BYTE; + cfg.unitType = TAOS_CFG_UTYPE_Mb; taosInitConfigOption(cfg); cfg.option = "blocks"; @@ -617,7 +615,7 @@ static void doInitGlobalConfig() { cfg.minValue = TSDB_MIN_TOTAL_BLOCKS; cfg.maxValue = TSDB_MAX_TOTAL_BLOCKS; cfg.ptrLength = 0; - cfg.unitType = TAOS_CFG_UTYPE_BYTE; + cfg.unitType = TAOS_CFG_UTYPE_NONE; taosInitConfigOption(cfg); cfg.option = "days"; @@ -680,7 +678,7 @@ static void doInitGlobalConfig() { cfg.unitType = TAOS_CFG_UTYPE_NONE; taosInitConfigOption(cfg); - cfg.option = "wallevel"; + cfg.option = "walLevel"; cfg.ptr = &tsWAL; cfg.valType = TAOS_CFG_VTYPE_INT16; cfg.cfgType = TSDB_CFG_CTYPE_B_CONFIG | TSDB_CFG_CTYPE_B_SHOW; @@ -814,32 +812,12 @@ static void doInitGlobalConfig() { cfg.unitType = TAOS_CFG_UTYPE_NONE; taosInitConfigOption(cfg); - cfg.option = "maxMeterConnections"; - cfg.ptr = &tsMaxMeterConnections; - cfg.valType = TAOS_CFG_VTYPE_INT32; - cfg.cfgType = TSDB_CFG_CTYPE_B_CONFIG | TSDB_CFG_CTYPE_B_SHOW; - cfg.minValue = 10; - cfg.maxValue = 50000000; - cfg.ptrLength = 0; - cfg.unitType = TAOS_CFG_UTYPE_NONE; - taosInitConfigOption(cfg); - - cfg.option = "maxMgmtConnections"; - cfg.ptr = &tsMaxMgmtConnections; + cfg.option = "maxConnections"; + cfg.ptr = &tsMaxConnections; cfg.valType = TAOS_CFG_VTYPE_INT32; cfg.cfgType = TSDB_CFG_CTYPE_B_CONFIG | TSDB_CFG_CTYPE_B_SHOW; - cfg.minValue = 10; - cfg.maxValue = 50000000; - cfg.ptrLength = 0; - cfg.unitType = TAOS_CFG_UTYPE_NONE; - taosInitConfigOption(cfg); - - cfg.option = "maxVnodeConnections"; - cfg.ptr = &tsMaxVnodeConnections; - cfg.valType = TAOS_CFG_VTYPE_INT32; - cfg.cfgType = TSDB_CFG_CTYPE_B_CONFIG | TSDB_CFG_CTYPE_B_SHOW; - cfg.minValue = 10; - cfg.maxValue = 50000000; + cfg.minValue = 1; + cfg.maxValue = 100; cfg.ptrLength = 0; cfg.unitType = TAOS_CFG_UTYPE_NONE; taosInitConfigOption(cfg); @@ -875,8 +853,8 @@ static void doInitGlobalConfig() { taosInitConfigOption(cfg); // module configs - cfg.option = "mgmtEqualVnodeNum"; - cfg.ptr = &tsMgmtEqualVnodeNum; + cfg.option = "mnodeEqualVnodeNum"; + cfg.ptr = &tsMnodeEqualVnodeNum; cfg.valType = TAOS_CFG_VTYPE_INT32; cfg.cfgType = TSDB_CFG_CTYPE_B_CONFIG | TSDB_CFG_CTYPE_B_SHOW; cfg.minValue = 0; diff --git a/src/cq/src/cqMain.c b/src/cq/src/cqMain.c index 6e81db7db735b49a4a94b9b5a5af76469dc84fb9..9406a2fdce90713d784758d479e953aa18aacdb9 100644 --- a/src/cq/src/cqMain.c +++ b/src/cq/src/cqMain.c @@ -15,17 +15,19 @@ #define _DEFAULT_SOURCE +#include +#include #include #include -#include -#include + +#include "taos.h" #include "taosdef.h" #include "taosmsg.h" +#include "tcq.h" +#include "tdataformat.h" #include "tglobal.h" #include "tlog.h" #include "twal.h" -#include "tcq.h" -#include "taos.h" #define cError(...) { if (cqDebugFlag & DEBUG_ERROR) { taosPrintLog("ERROR CQ ", cqDebugFlag, __VA_ARGS__); }} #define cWarn(...) { if (cqDebugFlag & DEBUG_WARN) { taosPrintLog("WARN CQ ", cqDebugFlag, __VA_ARGS__); }} @@ -46,15 +48,14 @@ typedef struct { } SCqContext; typedef struct SCqObj { - int tid; // table ID - int rowSize; // bytes of a row - char *sqlStr; // SQL string - int columns; // number of columns - SSchema *pSchema; // pointer to schema array - void *pStream; - struct SCqObj *prev; - struct SCqObj *next; - SCqContext *pContext; + int tid; // table ID + int rowSize; // bytes of a row + char * sqlStr; // SQL string + STSchema * pSchema; // pointer to schema array + void * pStream; + struct SCqObj *prev; + struct SCqObj *next; + SCqContext * pContext; } SCqObj; int cqDebugFlag = 135; @@ -152,7 +153,7 @@ void cqStop(void *handle) { pthread_mutex_unlock(&pContext->mutex); } -void *cqCreate(void *handle, int tid, char *sqlStr, SSchema *pSchema, int columns) { +void *cqCreate(void *handle, int tid, char *sqlStr, STSchema *pSchema) { SCqContext *pContext = handle; SCqObj *pObj = calloc(sizeof(SCqObj), 1); @@ -162,11 +163,7 @@ void *cqCreate(void *handle, int tid, char *sqlStr, SSchema *pSchema, int column pObj->sqlStr = malloc(strlen(sqlStr)+1); strcpy(pObj->sqlStr, sqlStr); - pObj->columns = columns; - - int size = sizeof(SSchema) * columns; - pObj->pSchema = malloc(size); - memcpy(pObj->pSchema, pSchema, size); + pObj->pSchema = tdDupSchema(pSchema); cTrace("vgId:%d, id:%d CQ:%s is created", pContext->vgId, pObj->tid, pObj->sqlStr); diff --git a/src/cq/test/cqtest.c b/src/cq/test/cqtest.c index 7977bd85bc18fcf659fa3608b4ccf99923529d92..3aa649ee34ce84cdc79b9d42dffd8656137a618e 100644 --- a/src/cq/test/cqtest.c +++ b/src/cq/test/cqtest.c @@ -59,21 +59,16 @@ int main(int argc, char *argv[]) { exit(-1); } - SSchema schema[2]; - schema[0].type = TSDB_DATA_TYPE_TIMESTAMP; - strcpy(schema[0].name, "ts"); - schema[0].colId = 0; - schema[0].bytes = 8; - - schema[1].type = TSDB_DATA_TYPE_INT; - strcpy(schema[1].name, "avgspeed"); - schema[1].colId = 1; - schema[1].bytes = 4; + STSchema *pSchema = tdNewSchema(2); + tdSchemaAddCol(pSchema, TSDB_DATA_TYPE_TIMESTAMP, 0, 8); + tdSchemaAddCol(pSchema, TSDB_DATA_TYPE_INT, 1, 4); for (int sid =1; sid<10; ++sid) { - cqCreate(pCq, sid, "select avg(speed) from demo.t1 sliding(1s) interval(5s)", schema, 2); + cqCreate(pCq, sid, "select avg(speed) from demo.t1 sliding(1s) interval(5s)", pSchema); } + tdFreeSchema(pSchema); + while (1) { char c = getchar(); diff --git a/src/dnode/inc/dnodeInt.h b/src/dnode/inc/dnodeInt.h index 663914a95908efdc1f4a93db36f7a3ff00d33e1f..34bfe6e4f7d7390631169c1d4d4cbd6bc09cc73e 100644 --- a/src/dnode/inc/dnodeInt.h +++ b/src/dnode/inc/dnodeInt.h @@ -25,7 +25,7 @@ extern "C" { extern int32_t dDebugFlag; #define dError(...) { if (dDebugFlag & DEBUG_ERROR) { taosPrintLog("ERROR DND ", 255, __VA_ARGS__); }} -#define dWarn(...) { if (dDebugFlag & DEBUG_WARN) { taosPrintLog("WARN DND ", dDebugFlag, __VA_ARGS__); }} +#define dWarn(...) { if (dDebugFlag & DEBUG_WARN) { taosPrintLog("WARN DND ", dDebugFlag, __VA_ARGS__); }} #define dTrace(...) { if (dDebugFlag & DEBUG_TRACE) { taosPrintLog("DND ", dDebugFlag, __VA_ARGS__); }} #define dPrint(...) { taosPrintLog("DND ", 255, __VA_ARGS__); } diff --git a/src/mnode/inc/mgmtDClient.h b/src/dnode/inc/dnodeMPeer.h similarity index 72% rename from src/mnode/inc/mgmtDClient.h rename to src/dnode/inc/dnodeMPeer.h index 1bd222f4bce3b1512093cdc2efb6a9a703619bfd..9a48703110c389f10ac623ccaeaa85420a32817f 100644 --- a/src/mnode/inc/mgmtDClient.h +++ b/src/dnode/inc/dnodeMPeer.h @@ -13,17 +13,18 @@ * along with this program. If not, see . */ -#ifndef TDENGINE_MGMT_DCLIENT_H -#define TDENGINE_MGMT_DCLIENT_H +#ifndef TDENGINE_DNODE_MPEER_H +#define TDENGINE_DNODE_MPEER_H #ifdef __cplusplus extern "C" { #endif -int32_t mgmtInitDClient(); -void mgmtCleanupDClient(); -void mgmtAddDClientRspHandle(uint8_t msgType, void (*fp)(SRpcMsg *rpcMsg)); -void mgmtSendMsgToDnode(SRpcIpSet *ipSet, SRpcMsg *rpcMsg); +int32_t dnodeInitMnodePeer(); +void dnodeCleanupMnodePeer(); +int32_t dnodeAllocateMnodePqueue(); +void dnodeFreeMnodePqueue(); +void dnodeDispatchToMnodePeerQueue(SRpcMsg *pMsg); #ifdef __cplusplus } diff --git a/src/mnode/inc/mgmtUser.h b/src/dnode/inc/dnodeMRead.h similarity index 60% rename from src/mnode/inc/mgmtUser.h rename to src/dnode/inc/dnodeMRead.h index 2edd71f3e726fee724f168e0d1301261c33a530b..4e93838b7998850e1bec79f017fdfbd28f286a11 100644 --- a/src/mnode/inc/mgmtUser.h +++ b/src/dnode/inc/dnodeMRead.h @@ -13,23 +13,18 @@ * along with this program. If not, see . */ -#ifndef TDENGINE_MGMT_USER_H -#define TDENGINE_MGMT_USER_H +#ifndef TDENGINE_DNODE_MREAD_H +#define TDENGINE_DNODE_MREAD_H #ifdef __cplusplus extern "C" { #endif -#include "mgmtDef.h" -int32_t mgmtInitUsers(); -void mgmtCleanUpUsers(); -SUserObj *mgmtGetUser(char *name); -void * mgmtGetNextUser(void *pIter, SUserObj **pUser); -void mgmtIncUserRef(SUserObj *pUser); -void mgmtDecUserRef(SUserObj *pUser); -SUserObj *mgmtGetUserFromConn(void *pConn); -int32_t mgmtCreateUser(SAcctObj *pAcct, char *name, char *pass); -void mgmtDropAllUsers(SAcctObj *pAcct); +int32_t dnodeInitMnodeRead(); +void dnodeCleanupMnodeRead(); +int32_t dnodeAllocateMnodeRqueue(); +void dnodeFreeMnodeRqueue(); +void dnodeDispatchToMnodeReadQueue(SRpcMsg *rpcMsg); #ifdef __cplusplus } diff --git a/src/dnode/inc/dnodeMWrite.h b/src/dnode/inc/dnodeMWrite.h new file mode 100644 index 0000000000000000000000000000000000000000..498fea81c59329b4d30874d36d10182b6e3ae54f --- /dev/null +++ b/src/dnode/inc/dnodeMWrite.h @@ -0,0 +1,33 @@ +/* + * Copyright (c) 2019 TAOS Data, Inc. + * + * This program is free software: you can use, redistribute, and/or modify + * it under the terms of the GNU Affero General Public License, version 3 + * or later ("AGPL"), as published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ + +#ifndef TDENGINE_DNODE_MWRITE_H +#define TDENGINE_DNODE_MWRITE_H + +#ifdef __cplusplus +extern "C" { +#endif + +int32_t dnodeInitMnodeWrite(); +void dnodeCleanupMnodeWrite(); +int32_t dnodeAllocateMnodeWqueue(); +void dnodeFreeMnodeWqueue(); +void dnodeDispatchToMnodeWriteQueue(SRpcMsg *pMsg); + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/src/dnode/inc/dnodeMain.h b/src/dnode/inc/dnodeMain.h index df7698ffc3246fbb419c554123b98ed74dbef9a7..c1480407bde0bbcf7aada20f90084b96fc380a73 100644 --- a/src/dnode/inc/dnodeMain.h +++ b/src/dnode/inc/dnodeMain.h @@ -21,7 +21,7 @@ extern "C" { #endif int32_t dnodeInitSystem(); -void dnodeCleanUpSystem(); +void dnodeCleanUpSystem(); #ifdef __cplusplus } diff --git a/src/dnode/inc/dnodeMgmt.h b/src/dnode/inc/dnodeMgmt.h index 6f2af423bc2d92e3d323a7a1dff9fcabe1f2dd2c..826f4ff1c1deaf4742f0b247de23ffa2c163ada9 100644 --- a/src/dnode/inc/dnodeMgmt.h +++ b/src/dnode/inc/dnodeMgmt.h @@ -22,7 +22,7 @@ extern "C" { int32_t dnodeInitMgmt(); void dnodeCleanupMgmt(); -void dnodeDispatchToDnodeMgmt(SRpcMsg *rpcMsg); +void dnodeDispatchToMgmtQueue(SRpcMsg *rpcMsg); void* dnodeGetVnode(int32_t vgId); int32_t dnodeGetVnodeStatus(void *pVnode); @@ -32,6 +32,10 @@ void* dnodeGetVnodeWal(void *pVnode); void* dnodeGetVnodeTsdb(void *pVnode); void dnodeReleaseVnode(void *pVnode); +void dnodeSendRedirectMsg(SRpcMsg *rpcMsg, bool forShell); +void dnodeGetMnodeIpSetForPeer(void *ipSet); +void dnodeGetMnodeIpSetForShell(void *ipSet); + #ifdef __cplusplus } #endif diff --git a/src/dnode/inc/dnodePeer.h b/src/dnode/inc/dnodePeer.h index 2ce8d80c0f7e2924d93f326418ec3819696a806a..0dcf48f2322be91de792148165b2a127fa983196 100644 --- a/src/dnode/inc/dnodePeer.h +++ b/src/dnode/inc/dnodePeer.h @@ -13,8 +13,8 @@ * along with this program. If not, see . */ -#ifndef TDENGINE_DNODE_DNODE_H -#define TDENGINE_DNODE_DNODE_H +#ifndef TDENGINE_DNODE_PEER_H +#define TDENGINE_DNODE_PEER_H #ifdef __cplusplus extern "C" { diff --git a/src/dnode/inc/dnodeVRead.h b/src/dnode/inc/dnodeVRead.h index 9e0c7b312000b9af2b84cb2a0c493b7501873090..a1035200475259bd91757f934d35a0dd5a69b1fe 100644 --- a/src/dnode/inc/dnodeVRead.h +++ b/src/dnode/inc/dnodeVRead.h @@ -13,15 +13,15 @@ * along with this program. If not, see . */ -#ifndef TDENGINE_DNODE_READ_H -#define TDENGINE_DNODE_READ_H +#ifndef TDENGINE_DNODE_VREAD_H +#define TDENGINE_DNODE_VREAD_H #ifdef __cplusplus extern "C" { #endif -int32_t dnodeInitRead(); -void dnodeCleanupRead(); +int32_t dnodeInitVnodeRead(); +void dnodeCleanupVnodeRead(); void dnodeDispatchToVnodeReadQueue(SRpcMsg *pMsg); #ifdef __cplusplus diff --git a/src/dnode/inc/dnodeVWrite.h b/src/dnode/inc/dnodeVWrite.h index 461e51983fb6e8fa84a6a3da83a8d72f5d7a930c..7da701a8e270239ee2f48e090138f048cc81f880 100644 --- a/src/dnode/inc/dnodeVWrite.h +++ b/src/dnode/inc/dnodeVWrite.h @@ -13,17 +13,16 @@ * along with this program. If not, see . */ -#ifndef TDENGINE_DNODE_WRITE_H -#define TDENGINE_DNODE_WRITE_H +#ifndef TDENGINE_DNODE_VWRITE_H +#define TDENGINE_DNODE_VWRITE_H #ifdef __cplusplus extern "C" { #endif -int32_t dnodeInitWrite(); -void dnodeCleanupWrite(); +int32_t dnodeInitVnodeWrite(); +void dnodeCleanupVnodeWrite(); void dnodeDispatchToVnodeWriteQueue(SRpcMsg *pMsg); -void dnodeSendWriteResponse(void *pVnode, void *param, int32_t code); #ifdef __cplusplus } diff --git a/src/dnode/src/dnodeMPeer.c b/src/dnode/src/dnodeMPeer.c new file mode 100644 index 0000000000000000000000000000000000000000..e3ba5fcf017cb51bd6eef69d45350ddaf03b490c --- /dev/null +++ b/src/dnode/src/dnodeMPeer.c @@ -0,0 +1,158 @@ +/* + * Copyright (c) 2019 TAOS Data, Inc. + * + * This program is free software: you can use, redistribute, and/or modify + * it under the terms of the GNU Affero General Public License, version 3 + * or later ("AGPL"), as published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ + +#define _DEFAULT_SOURCE +#include "os.h" +#include "taoserror.h" +#include "taosmsg.h" +#include "tutil.h" +#include "tqueue.h" +#include "twal.h" +#include "tglobal.h" +#include "mnode.h" +#include "dnode.h" +#include "dnodeInt.h" +#include "dnodeMgmt.h" +#include "dnodeMWrite.h" + +typedef struct { + pthread_t thread; + int32_t workerId; +} SMPeerWorker; + +typedef struct { + int32_t num; + SMPeerWorker *peerWorker; +} SMPeerWorkerPool; + +static SMPeerWorkerPool tsMPeerPool; +static taos_qset tsMPeerQset; +static taos_queue tsMPeerQueue; + +static void *dnodeProcessMnodePeerQueue(void *param); + +int32_t dnodeInitMnodePeer() { + tsMPeerQset = taosOpenQset(); + + tsMPeerPool.num = 1; + tsMPeerPool.peerWorker = (SMPeerWorker *)calloc(sizeof(SMPeerWorker), tsMPeerPool.num); + + if (tsMPeerPool.peerWorker == NULL) return -1; + for (int32_t i = 0; i < tsMPeerPool.num; ++i) { + SMPeerWorker *pWorker = tsMPeerPool.peerWorker + i; + pWorker->workerId = i; + } + + dPrint("dnode mpeer is opened"); + return 0; +} + +void dnodeCleanupMnodePeer() { + for (int32_t i = 0; i < tsMPeerPool.num; ++i) { + SMPeerWorker *pWorker = tsMPeerPool.peerWorker + i; + if (pWorker->thread) { + taosQsetThreadResume(tsMPeerQset); + } + } + + for (int32_t i = 0; i < tsMPeerPool.num; ++i) { + SMPeerWorker *pWorker = tsMPeerPool.peerWorker + i; + if (pWorker->thread) { + pthread_join(pWorker->thread, NULL); + } + } + + dPrint("dnode mpeer is closed"); +} + +int32_t dnodeAllocateMnodePqueue() { + tsMPeerQueue = taosOpenQueue(); + if (tsMPeerQueue == NULL) return TSDB_CODE_SERV_OUT_OF_MEMORY; + + taosAddIntoQset(tsMPeerQset, tsMPeerQueue, NULL); + + for (int32_t i = 0; i < tsMPeerPool.num; ++i) { + SMPeerWorker *pWorker = tsMPeerPool.peerWorker + i; + pWorker->workerId = i; + + pthread_attr_t thAttr; + pthread_attr_init(&thAttr); + pthread_attr_setdetachstate(&thAttr, PTHREAD_CREATE_JOINABLE); + + if (pthread_create(&pWorker->thread, &thAttr, dnodeProcessMnodePeerQueue, pWorker) != 0) { + dError("failed to create thread to process mpeer queue, reason:%s", strerror(errno)); + } + + pthread_attr_destroy(&thAttr); + dTrace("dnode mpeer worker:%d is launched, total:%d", pWorker->workerId, tsMPeerPool.num); + } + + dTrace("dnode mpeer queue:%p is allocated", tsMPeerQueue); + return TSDB_CODE_SUCCESS; +} + +void dnodeFreeMnodePqueue() { + taosCloseQueue(tsMPeerQueue); + tsMPeerQueue = NULL; +} + +void dnodeDispatchToMnodePeerQueue(SRpcMsg *pMsg) { + if (!mnodeIsRunning() || tsMPeerQueue == NULL) { + dnodeSendRedirectMsg(pMsg, false); + return; + } + + SMnodeMsg *pPeer = (SMnodeMsg *)taosAllocateQitem(sizeof(SMnodeMsg)); + mnodeCreateMsg(pPeer, pMsg); + taosWriteQitem(tsMPeerQueue, TAOS_QTYPE_RPC, pPeer); +} + +static void dnodeFreeMnodePeerMsg(SMnodeMsg *pPeer) { + mnodeCleanupMsg(pPeer); + taosFreeQitem(pPeer); +} + +static void dnodeSendRpcMnodePeerRsp(SMnodeMsg *pPeer, int32_t code) { + if (code == TSDB_CODE_ACTION_IN_PROGRESS) return; + + SRpcMsg rpcRsp = { + .handle = pPeer->rpcMsg.handle, + .pCont = pPeer->rpcRsp.rsp, + .contLen = pPeer->rpcRsp.len, + .code = code, + }; + + rpcSendResponse(&rpcRsp); + dnodeFreeMnodePeerMsg(pPeer); +} + +static void *dnodeProcessMnodePeerQueue(void *param) { + SMnodeMsg *pPeerMsg; + int32_t type; + void * unUsed; + + while (1) { + if (taosReadQitemFromQset(tsMPeerQset, &type, (void **)&pPeerMsg, &unUsed) == 0) { + dTrace("dnodeProcessMnodePeerQueue: got no message from qset, exiting..."); + break; + } + + dTrace("msg:%s will be processed in mpeer queue", taosMsg[pPeerMsg->rpcMsg.msgType]); + int32_t code = mnodeProcessPeerReq(pPeerMsg); + dnodeSendRpcMnodePeerRsp(pPeerMsg, code); + } + + return NULL; +} diff --git a/src/dnode/src/dnodeMRead.c b/src/dnode/src/dnodeMRead.c new file mode 100644 index 0000000000000000000000000000000000000000..f22346b61c3726858e319b1a40f7405efd94c7a2 --- /dev/null +++ b/src/dnode/src/dnodeMRead.c @@ -0,0 +1,168 @@ +/* + * Copyright (c) 2019 TAOS Data, Inc. + * + * This program is free software: you can use, redistribute, and/or modify + * it under the terms of the GNU Affero General Public License, version 3 + * or later ("AGPL"), as published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ + +#define _DEFAULT_SOURCE +#include "os.h" +#include "taoserror.h" +#include "taosmsg.h" +#include "tutil.h" +#include "tqueue.h" +#include "twal.h" +#include "tglobal.h" +#include "mnode.h" +#include "dnode.h" +#include "dnodeInt.h" +#include "dnodeMgmt.h" +#include "dnodeMRead.h" + +typedef struct { + pthread_t thread; + int32_t workerId; +} SMReadWorker; + +typedef struct { + int32_t num; + SMReadWorker *readWorker; +} SMReadWorkerPool; + +static SMReadWorkerPool tsMReadPool; +static taos_qset tsMReadQset; +static taos_queue tsMReadQueue; + +static void *dnodeProcessMnodeReadQueue(void *param); + +int32_t dnodeInitMnodeRead() { + tsMReadQset = taosOpenQset(); + + tsMReadPool.num = tsNumOfCores * tsNumOfThreadsPerCore / 2; + tsMReadPool.num = MAX(2, tsMReadPool.num); + tsMReadPool.num = MIN(4, tsMReadPool.num); + tsMReadPool.readWorker = (SMReadWorker *)calloc(sizeof(SMReadWorker), tsMReadPool.num); + + if (tsMReadPool.readWorker == NULL) return -1; + for (int32_t i = 0; i < tsMReadPool.num; ++i) { + SMReadWorker *pWorker = tsMReadPool.readWorker + i; + pWorker->workerId = i; + } + + dPrint("dnode mread is opened"); + return 0; +} + +void dnodeCleanupMnodeRead() { + for (int32_t i = 0; i < tsMReadPool.num; ++i) { + SMReadWorker *pWorker = tsMReadPool.readWorker + i; + if (pWorker->thread) { + taosQsetThreadResume(tsMReadQset); + } + } + + for (int32_t i = 0; i < tsMReadPool.num; ++i) { + SMReadWorker *pWorker = tsMReadPool.readWorker + i; + if (pWorker->thread) { + pthread_join(pWorker->thread, NULL); + } + } + + taosCloseQset(tsMReadQset); + free(tsMReadPool.readWorker); + + dPrint("dnode mread is closed"); +} + +int32_t dnodeAllocateMnodeRqueue() { + tsMReadQueue = taosOpenQueue(); + if (tsMReadQueue == NULL) return TSDB_CODE_SERV_OUT_OF_MEMORY; + + taosAddIntoQset(tsMReadQset, tsMReadQueue, NULL); + + for (int32_t i = 0; i < tsMReadPool.num; ++i) { + SMReadWorker *pWorker = tsMReadPool.readWorker + i; + pWorker->workerId = i; + + pthread_attr_t thAttr; + pthread_attr_init(&thAttr); + pthread_attr_setdetachstate(&thAttr, PTHREAD_CREATE_JOINABLE); + + if (pthread_create(&pWorker->thread, &thAttr, dnodeProcessMnodeReadQueue, pWorker) != 0) { + dError("failed to create thread to process mread queue, reason:%s", strerror(errno)); + } + + pthread_attr_destroy(&thAttr); + dTrace("dnode mread worker:%d is launched, total:%d", pWorker->workerId, tsMReadPool.num); + } + + dTrace("dnode mread queue:%p is allocated", tsMReadQueue); + return TSDB_CODE_SUCCESS; +} + +void dnodeFreeMnodeRqueue() { + taosCloseQueue(tsMReadQueue); + tsMReadQueue = NULL; +} + +void dnodeDispatchToMnodeReadQueue(SRpcMsg *pMsg) { + if (!mnodeIsRunning() || tsMReadQueue == NULL) { + dnodeSendRedirectMsg(pMsg, true); + return; + } + + SMnodeMsg *pRead = (SMnodeMsg *)taosAllocateQitem(sizeof(SMnodeMsg)); + mnodeCreateMsg(pRead, pMsg); + taosWriteQitem(tsMReadQueue, TAOS_QTYPE_RPC, pRead); +} + +static void dnodeFreeMnodeReadMsg(SMnodeMsg *pRead) { + mnodeCleanupMsg(pRead); + taosFreeQitem(pRead); +} + +static void dnodeSendRpcMnodeReadRsp(SMnodeMsg *pRead, int32_t code) { + if (code == TSDB_CODE_ACTION_IN_PROGRESS) return; + if (code == TSDB_CODE_ACTION_NEED_REPROCESSED) { + // may be a auto create req, should put into write queue + dnodeReprocessMnodeWriteMsg(pRead); + return; + } + + SRpcMsg rpcRsp = { + .handle = pRead->rpcMsg.handle, + .pCont = pRead->rpcRsp.rsp, + .contLen = pRead->rpcRsp.len, + .code = code, + }; + + rpcSendResponse(&rpcRsp); + dnodeFreeMnodeReadMsg(pRead); +} + +static void *dnodeProcessMnodeReadQueue(void *param) { + SMnodeMsg *pReadMsg; + int32_t type; + void * unUsed; + + while (1) { + if (taosReadQitemFromQset(tsMReadQset, &type, (void **)&pReadMsg, &unUsed) == 0) { + dTrace("dnodeProcessMnodeReadQueue: got no message from qset, exiting..."); + break; + } + + dTrace("%p, msg:%s will be processed in mread queue", pReadMsg->rpcMsg.ahandle, taosMsg[pReadMsg->rpcMsg.msgType]); + int32_t code = mnodeProcessRead(pReadMsg); + dnodeSendRpcMnodeReadRsp(pReadMsg, code); + } + + return NULL; +} diff --git a/src/dnode/src/dnodeMWrite.c b/src/dnode/src/dnodeMWrite.c new file mode 100644 index 0000000000000000000000000000000000000000..95fa9f0bdd35f3829921a62db50913ca501fc872 --- /dev/null +++ b/src/dnode/src/dnodeMWrite.c @@ -0,0 +1,186 @@ +/* + * Copyright (c) 2019 TAOS Data, Inc. + * + * This program is free software: you can use, redistribute, and/or modify + * it under the terms of the GNU Affero General Public License, version 3 + * or later ("AGPL"), as published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ + +#define _DEFAULT_SOURCE +#include "os.h" +#include "taoserror.h" +#include "taosmsg.h" +#include "tutil.h" +#include "ttimer.h" +#include "tqueue.h" +#include "twal.h" +#include "tglobal.h" +#include "mnode.h" +#include "dnode.h" +#include "dnodeInt.h" +#include "dnodeMgmt.h" +#include "dnodeMWrite.h" + +typedef struct { + pthread_t thread; + int32_t workerId; +} SMWriteWorker; + +typedef struct { + int32_t num; + SMWriteWorker *writeWorker; +} SMWriteWorkerPool; + +static SMWriteWorkerPool tsMWritePool; +static taos_qset tsMWriteQset; +static taos_queue tsMWriteQueue; +extern void * tsDnodeTmr; + +static void *dnodeProcessMnodeWriteQueue(void *param); + +int32_t dnodeInitMnodeWrite() { + tsMWriteQset = taosOpenQset(); + + tsMWritePool.num = 1; + tsMWritePool.writeWorker = (SMWriteWorker *)calloc(sizeof(SMWriteWorker), tsMWritePool.num); + + if (tsMWritePool.writeWorker == NULL) return -1; + for (int32_t i = 0; i < tsMWritePool.num; ++i) { + SMWriteWorker *pWorker = tsMWritePool.writeWorker + i; + pWorker->workerId = i; + } + + dPrint("dnode mwrite is opened"); + return 0; +} + +void dnodeCleanupMnodeWrite() { + for (int32_t i = 0; i < tsMWritePool.num; ++i) { + SMWriteWorker *pWorker = tsMWritePool.writeWorker + i; + if (pWorker->thread) { + taosQsetThreadResume(tsMWriteQset); + } + } + + for (int32_t i = 0; i < tsMWritePool.num; ++i) { + SMWriteWorker *pWorker = tsMWritePool.writeWorker + i; + if (pWorker->thread) { + pthread_join(pWorker->thread, NULL); + } + } + + dPrint("dnode mwrite is closed"); +} + +int32_t dnodeAllocateMnodeWqueue() { + tsMWriteQueue = taosOpenQueue(); + if (tsMWriteQueue == NULL) return TSDB_CODE_SERV_OUT_OF_MEMORY; + + taosAddIntoQset(tsMWriteQset, tsMWriteQueue, NULL); + + for (int32_t i = 0; i < tsMWritePool.num; ++i) { + SMWriteWorker *pWorker = tsMWritePool.writeWorker + i; + pWorker->workerId = i; + + pthread_attr_t thAttr; + pthread_attr_init(&thAttr); + pthread_attr_setdetachstate(&thAttr, PTHREAD_CREATE_JOINABLE); + + if (pthread_create(&pWorker->thread, &thAttr, dnodeProcessMnodeWriteQueue, pWorker) != 0) { + dError("failed to create thread to process mwrite queue, reason:%s", strerror(errno)); + } + + pthread_attr_destroy(&thAttr); + dTrace("dnode mwrite worker:%d is launched, total:%d", pWorker->workerId, tsMWritePool.num); + } + + dTrace("dnode mwrite queue:%p is allocated", tsMWriteQueue); + return TSDB_CODE_SUCCESS; +} + +void dnodeFreeMnodeWqueue() { + taosCloseQueue(tsMWriteQueue); + tsMWriteQueue = NULL; +} + +void dnodeDispatchToMnodeWriteQueue(SRpcMsg *pMsg) { + if (!mnodeIsRunning() || tsMWriteQueue == NULL) { + dnodeSendRedirectMsg(pMsg, true); + return; + } + + SMnodeMsg *pWrite = (SMnodeMsg *)taosAllocateQitem(sizeof(SMnodeMsg)); + mnodeCreateMsg(pWrite, pMsg); + taosWriteQitem(tsMWriteQueue, TAOS_QTYPE_RPC, pWrite); +} + +static void dnodeFreeMnodeWriteMsg(SMnodeMsg *pWrite) { + mnodeCleanupMsg(pWrite); + taosFreeQitem(pWrite); +} + +void dnodeSendRpcMnodeWriteRsp(void *pRaw, int32_t code) { + SMnodeMsg *pWrite = pRaw; + if (code == TSDB_CODE_ACTION_IN_PROGRESS) return; + if (code == TSDB_CODE_ACTION_NEED_REPROCESSED) { + dnodeReprocessMnodeWriteMsg(pWrite); + return; + } + + SRpcMsg rpcRsp = { + .handle = pWrite->rpcMsg.handle, + .pCont = pWrite->rpcRsp.rsp, + .contLen = pWrite->rpcRsp.len, + .code = code, + }; + + rpcSendResponse(&rpcRsp); + dnodeFreeMnodeWriteMsg(pWrite); +} + +static void *dnodeProcessMnodeWriteQueue(void *param) { + SMnodeMsg *pWriteMsg; + int32_t type; + void * unUsed; + + while (1) { + if (taosReadQitemFromQset(tsMWriteQset, &type, (void **)&pWriteMsg, &unUsed) == 0) { + dTrace("dnodeProcessMnodeWriteQueue: got no message from qset, exiting..."); + break; + } + + dTrace("%p, msg:%s will be processed in mwrite queue", pWriteMsg->rpcMsg.ahandle, taosMsg[pWriteMsg->rpcMsg.msgType]); + int32_t code = mnodeProcessWrite(pWriteMsg); + dnodeSendRpcMnodeWriteRsp(pWriteMsg, code); + } + + return NULL; +} + +void dnodeReprocessMnodeWriteMsg(void *pMsg) { + SMnodeMsg *pWrite = pMsg; + + if (!mnodeIsRunning() || tsMWriteQueue == NULL) { + dnodeSendRedirectMsg(pMsg, true); + dnodeFreeMnodeWriteMsg(pWrite); + } else { + taosWriteQitem(tsMWriteQueue, TAOS_QTYPE_RPC, pWrite); + } +} + +static void dnodeDoDelayReprocessMnodeWriteMsg(void *param, void *tmrId) { + dnodeReprocessMnodeWriteMsg(param); +} + +void dnodeDelayReprocessMnodeWriteMsg(void *pMsg) { + SMnodeMsg *mnodeMsg = pMsg; + void *unUsed = NULL; + taosTmrReset(dnodeDoDelayReprocessMnodeWriteMsg, 300, mnodeMsg, tsDnodeTmr, &unUsed); +} \ No newline at end of file diff --git a/src/dnode/src/dnodeMain.c b/src/dnode/src/dnodeMain.c index 68fe9869899edc42a1f6c251cce9749a800f43f6..76f9446e0e9deeb7a3c2d2085b1020241e38cf05 100644 --- a/src/dnode/src/dnodeMain.c +++ b/src/dnode/src/dnodeMain.c @@ -25,8 +25,11 @@ #include "dnodePeer.h" #include "dnodeModule.h" #include "dnodeVRead.h" -#include "dnodeShell.h" #include "dnodeVWrite.h" +#include "dnodeMRead.h" +#include "dnodeMWrite.h" +#include "dnodeMPeer.h" +#include "dnodeShell.h" static int32_t dnodeInitStorage(); static void dnodeCleanupStorage(); @@ -65,8 +68,11 @@ int32_t dnodeInitSystem() { dPrint("start to initialize TDengine on %s", tsLocalEp); if (dnodeInitStorage() != 0) return -1; - if (dnodeInitRead() != 0) return -1; - if (dnodeInitWrite() != 0) return -1; + if (dnodeInitVnodeRead() != 0) return -1; + if (dnodeInitVnodeWrite() != 0) return -1; + if (dnodeInitMnodeRead() != 0) return -1; + if (dnodeInitMnodeWrite() != 0) return -1; + if (dnodeInitMnodePeer() != 0) return -1; if (dnodeInitClient() != 0) return -1; if (dnodeInitServer() != 0) return -1; if (dnodeInitMgmt() != 0) return -1; @@ -89,8 +95,11 @@ void dnodeCleanUpSystem() { dnodeCleanupMgmt(); dnodeCleanupServer(); dnodeCleanupClient(); - dnodeCleanupWrite(); - dnodeCleanupRead(); + dnodeCleanupMnodePeer(); + dnodeCleanupMnodeWrite(); + dnodeCleanupMnodeRead(); + dnodeCleanupVnodeWrite(); + dnodeCleanupVnodeRead(); dnodeCleanupStorage(); taos_cleanup(); taosCloseLog(); diff --git a/src/dnode/src/dnodeMgmt.c b/src/dnode/src/dnodeMgmt.c index 4b28992aa492f41edfbbf6bd2ac4a795140b6f45..7c457defca9dd822b018221deeeb4646ed919988 100644 --- a/src/dnode/src/dnodeMgmt.c +++ b/src/dnode/src/dnodeMgmt.c @@ -20,9 +20,9 @@ #include "taosmsg.h" #include "ttime.h" #include "ttimer.h" -#include "trpc.h" #include "tsdb.h" #include "twal.h" +#include "tqueue.h" #include "tsync.h" #include "ttime.h" #include "ttimer.h" @@ -39,6 +39,17 @@ #define MPEER_CONTENT_LEN 2000 +void * tsDnodeTmr = NULL; +static void * tsStatusTimer = NULL; +static uint32_t tsRebootTime; + +static SRpcIpSet tsDMnodeIpSet = {0}; +static SDMMnodeInfos tsDMnodeInfos = {0}; +static SDMDnodeCfg tsDnodeCfg = {0}; +static taos_qset tsMgmtQset = NULL; +static taos_queue tsMgmtQueue = NULL; +static pthread_t tsQthread; + static void dnodeUpdateMnodeInfos(SDMMnodeInfos *pMnodes); static bool dnodeReadMnodeInfos(); static void dnodeSaveMnodeInfos(); @@ -47,14 +58,7 @@ static bool dnodeReadDnodeCfg(); static void dnodeSaveDnodeCfg(); static void dnodeProcessStatusRsp(SRpcMsg *pMsg); static void dnodeSendStatusMsg(void *handle, void *tmrId); - -static void *tsDnodeTmr = NULL; -static void *tsStatusTimer = NULL; -static uint32_t tsRebootTime; - -static SRpcIpSet tsMnodeIpSet = {0}; -static SDMMnodeInfos tsMnodeInfos = {0}; -static SDMDnodeCfg tsDnodeCfg = {0}; +static void *dnodeProcessMgmtQueue(void *param); static int32_t dnodeOpenVnodes(); static void dnodeCloseVnodes(); @@ -74,34 +78,64 @@ int32_t dnodeInitMgmt() { dnodeReadDnodeCfg(); tsRebootTime = taosGetTimestampSec(); - tsDnodeTmr = taosTmrInit(100, 200, 60000, "DND-DM"); - if (tsDnodeTmr == NULL) { - dError("failed to init dnode timer"); - return -1; - } - if (!dnodeReadMnodeInfos()) { - memset(&tsMnodeIpSet, 0, sizeof(SRpcIpSet)); - memset(&tsMnodeInfos, 0, sizeof(SDMMnodeInfos)); - tsMnodeIpSet.numOfIps = 1; - taosGetFqdnPortFromEp(tsFirst, tsMnodeIpSet.fqdn[0], &tsMnodeIpSet.port[0]); - tsMnodeIpSet.port[0] += TSDB_PORT_DNODEDNODE; + memset(&tsDMnodeIpSet, 0, sizeof(SRpcIpSet)); + memset(&tsDMnodeInfos, 0, sizeof(SDMMnodeInfos)); + + tsDMnodeIpSet.numOfIps = 1; + taosGetFqdnPortFromEp(tsFirst, tsDMnodeIpSet.fqdn[0], &tsDMnodeIpSet.port[0]); + if (strcmp(tsSecond, tsFirst) != 0) { - tsMnodeIpSet.numOfIps = 2; - taosGetFqdnPortFromEp(tsSecond, tsMnodeIpSet.fqdn[1], &tsMnodeIpSet.port[1]); - tsMnodeIpSet.port[1] += TSDB_PORT_DNODEDNODE; + tsDMnodeIpSet.numOfIps = 2; + taosGetFqdnPortFromEp(tsSecond, tsDMnodeIpSet.fqdn[1], &tsDMnodeIpSet.port[1]); } } else { - tsMnodeIpSet.inUse = tsMnodeInfos.inUse; - tsMnodeIpSet.numOfIps = tsMnodeInfos.nodeNum; - for (int32_t i = 0; i < tsMnodeInfos.nodeNum; i++) { - taosGetFqdnPortFromEp(tsMnodeInfos.nodeInfos[i].nodeEp, tsMnodeIpSet.fqdn[i], &tsMnodeIpSet.port[i]); - tsMnodeIpSet.port[i] += TSDB_PORT_DNODEDNODE; + tsDMnodeIpSet.inUse = tsDMnodeInfos.inUse; + tsDMnodeIpSet.numOfIps = tsDMnodeInfos.nodeNum; + for (int32_t i = 0; i < tsDMnodeInfos.nodeNum; i++) { + taosGetFqdnPortFromEp(tsDMnodeInfos.nodeInfos[i].nodeEp, tsDMnodeIpSet.fqdn[i], &tsDMnodeIpSet.port[i]); } } - int32_t code = dnodeOpenVnodes(); + // create the queue and thread to handle the message + tsMgmtQset = taosOpenQset(); + if (tsMgmtQset == NULL) { + dError("failed to create the mgmt queue set"); + dnodeCleanupMgmt(); + return -1; + } + + tsMgmtQueue = taosOpenQueue(); + if (tsMgmtQueue == NULL) { + dError("failed to create the mgmt queue"); + dnodeCleanupMgmt(); + return -1; + } + + taosAddIntoQset(tsMgmtQset, tsMgmtQueue, NULL); + + pthread_attr_t thAttr; + pthread_attr_init(&thAttr); + pthread_attr_setdetachstate(&thAttr, PTHREAD_CREATE_JOINABLE); + + int32_t code = pthread_create(&tsQthread, &thAttr, dnodeProcessMgmtQueue, NULL); + pthread_attr_destroy(&thAttr); + if (code != 0) { + dError("failed to create thread to process mgmt queue, reason:%s", strerror(errno)); + dnodeCleanupMgmt(); + return -1; + } + + code = dnodeOpenVnodes(); if (code != TSDB_CODE_SUCCESS) { + dnodeCleanupMgmt(); + return -1; + } + + tsDnodeTmr = taosTmrInit(100, 200, 60000, "DND-DM"); + if (tsDnodeTmr == NULL) { + dError("failed to init dnode timer"); + dnodeCleanupMgmt(); return -1; } @@ -124,22 +158,62 @@ void dnodeCleanupMgmt() { } dnodeCloseVnodes(); + + if (tsMgmtQset) taosQsetThreadResume(tsMgmtQset); + if (tsQthread) pthread_join(tsQthread, NULL); + + if (tsMgmtQueue) taosCloseQueue(tsMgmtQueue); + if (tsMgmtQset) taosCloseQset(tsMgmtQset); + tsMgmtQset = NULL; + tsMgmtQueue = NULL; + } -void dnodeDispatchToDnodeMgmt(SRpcMsg *pMsg) { - SRpcMsg rsp; +void dnodeDispatchToMgmtQueue(SRpcMsg *pMsg) { + void *item; - if (dnodeProcessMgmtMsgFp[pMsg->msgType]) { - rsp.code = (*dnodeProcessMgmtMsgFp[pMsg->msgType])(pMsg); + item = taosAllocateQitem(sizeof(SRpcMsg)); + if (item) { + memcpy(item, pMsg, sizeof(SRpcMsg)); + taosWriteQitem(tsMgmtQueue, 1, item); } else { - rsp.code = TSDB_CODE_MSG_NOT_PROCESSED; + SRpcMsg rsp; + rsp.handle = pMsg->handle; + rsp.pCont = NULL; + rsp.code = TSDB_CODE_SERV_OUT_OF_MEMORY; + rpcSendResponse(&rsp); + rpcFreeCont(pMsg->pCont); } +} - rsp.handle = pMsg->handle; - rsp.pCont = NULL; - rpcSendResponse(&rsp); +static void *dnodeProcessMgmtQueue(void *param) { + SRpcMsg *pMsg; + SRpcMsg rsp; + int type; + void *handle; - rpcFreeCont(pMsg->pCont); + while (1) { + if (taosReadQitemFromQset(tsMgmtQset, &type, (void **) &pMsg, &handle) == 0) { + dTrace("dnode mgmt got no message from qset, exit ..."); + break; + } + + dTrace("%p, msg:%s will be processed", pMsg->ahandle, taosMsg[pMsg->msgType]); + if (dnodeProcessMgmtMsgFp[pMsg->msgType]) { + rsp.code = (*dnodeProcessMgmtMsgFp[pMsg->msgType])(pMsg); + } else { + rsp.code = TSDB_CODE_MSG_NOT_PROCESSED; + } + + rsp.handle = pMsg->handle; + rsp.pCont = NULL; + rpcSendResponse(&rsp); + + rpcFreeCont(pMsg->pCont); + taosFreeQitem(pMsg); + } + + return NULL; } static int32_t dnodeGetVnodeList(int32_t vnodeList[], int32_t *numOfVnodes) { @@ -265,23 +339,27 @@ static int32_t dnodeProcessConfigDnodeMsg(SRpcMsg *pMsg) { return taosCfgDynamicOptions(pCfg->config); } -void dnodeUpdateIpSet(SRpcIpSet *pIpSet) { - dPrint("mnode IP list is changed, numOfIps:%d inUse:%d", pIpSet->numOfIps, pIpSet->inUse); +void dnodeUpdateMnodeIpSetForPeer(SRpcIpSet *pIpSet) { + dPrint("mnode IP list for is changed, numOfIps:%d inUse:%d", pIpSet->numOfIps, pIpSet->inUse); for (int i = 0; i < pIpSet->numOfIps; ++i) { + pIpSet->port[i] -= TSDB_PORT_DNODEDNODE; dPrint("mnode index:%d %s:%u", i, pIpSet->fqdn[i], pIpSet->port[i]) } - tsMnodeIpSet = *pIpSet; + tsDMnodeIpSet = *pIpSet; } -void dnodeGetMnodeDnodeIpSet(void *ipSetRaw) { +void dnodeGetMnodeIpSetForPeer(void *ipSetRaw) { SRpcIpSet *ipSet = ipSetRaw; - ipSet->numOfIps = tsMnodeInfos.nodeNum; - ipSet->inUse = tsMnodeInfos.inUse; - for (int32_t i = 0; i < tsMnodeInfos.nodeNum; ++i) { - taosGetFqdnPortFromEp(tsMnodeInfos.nodeInfos[i].nodeEp, ipSet->fqdn[i], &ipSet->port[i]); + *ipSet = tsDMnodeIpSet; + + for (int i=0; inumOfIps; ++i) ipSet->port[i] += TSDB_PORT_DNODEDNODE; - } +} + +void dnodeGetMnodeIpSetForShell(void *ipSetRaw) { + SRpcIpSet *ipSet = ipSetRaw; + *ipSet = tsDMnodeIpSet; } static void dnodeProcessStatusRsp(SRpcMsg *pMsg) { @@ -321,22 +399,20 @@ static void dnodeProcessStatusRsp(SRpcMsg *pMsg) { } static void dnodeUpdateMnodeInfos(SDMMnodeInfos *pMnodes) { - bool mnodesChanged = (memcmp(&tsMnodeInfos, pMnodes, sizeof(SDMMnodeInfos)) != 0); - bool mnodesNotInit = (tsMnodeInfos.nodeNum == 0); + bool mnodesChanged = (memcmp(&tsDMnodeInfos, pMnodes, sizeof(SDMMnodeInfos)) != 0); + bool mnodesNotInit = (tsDMnodeInfos.nodeNum == 0); if (!(mnodesChanged || mnodesNotInit)) return; - memcpy(&tsMnodeInfos, pMnodes, sizeof(SDMMnodeInfos)); - - tsMnodeIpSet.inUse = tsMnodeInfos.inUse; - tsMnodeIpSet.numOfIps = tsMnodeInfos.nodeNum; - for (int32_t i = 0; i < tsMnodeInfos.nodeNum; i++) { - taosGetFqdnPortFromEp(tsMnodeInfos.nodeInfos[i].nodeEp, tsMnodeIpSet.fqdn[i], &tsMnodeIpSet.port[i]); - tsMnodeIpSet.port[i] += TSDB_PORT_DNODEDNODE; + memcpy(&tsDMnodeInfos, pMnodes, sizeof(SDMMnodeInfos)); + dPrint("mnode infos is changed, nodeNum:%d inUse:%d", tsDMnodeInfos.nodeNum, tsDMnodeInfos.inUse); + for (int32_t i = 0; i < tsDMnodeInfos.nodeNum; i++) { + dPrint("mnode index:%d, %s", tsDMnodeInfos.nodeInfos[i].nodeId, tsDMnodeInfos.nodeInfos[i].nodeEp); } - dPrint("mnodes is changed, nodeNum:%d inUse:%d", tsMnodeInfos.nodeNum, tsMnodeInfos.inUse); - for (int32_t i = 0; i < tsMnodeInfos.nodeNum; i++) { - dPrint("mnode:%d, %s", tsMnodeInfos.nodeInfos[i].nodeId, tsMnodeInfos.nodeInfos[i].nodeEp); + tsDMnodeIpSet.inUse = tsDMnodeInfos.inUse; + tsDMnodeIpSet.numOfIps = tsDMnodeInfos.nodeNum; + for (int32_t i = 0; i < tsDMnodeInfos.nodeNum; i++) { + taosGetFqdnPortFromEp(tsDMnodeInfos.nodeInfos[i].nodeEp, tsDMnodeIpSet.fqdn[i], &tsDMnodeIpSet.port[i]); } dnodeSaveMnodeInfos(); @@ -344,11 +420,12 @@ static void dnodeUpdateMnodeInfos(SDMMnodeInfos *pMnodes) { } static bool dnodeReadMnodeInfos() { - char ipFile[TSDB_FILENAME_LEN] = {0}; - sprintf(ipFile, "%s/mgmtIpList.json", tsDnodeDir); + char ipFile[TSDB_FILENAME_LEN*2] = {0}; + + sprintf(ipFile, "%s/mnodeIpList.json", tsDnodeDir); FILE *fp = fopen(ipFile, "r"); if (!fp) { - dTrace("failed to read mnode mgmtIpList.json, file not exist"); + dTrace("failed to read mnodeIpList.json, file not exist"); return false; } @@ -359,39 +436,39 @@ static bool dnodeReadMnodeInfos() { if (len <= 0) { free(content); fclose(fp); - dError("failed to read mnode mgmtIpList.json, content is null"); + dError("failed to read mnodeIpList.json, content is null"); return false; } cJSON* root = cJSON_Parse(content); if (root == NULL) { - dError("failed to read mnode mgmtIpList.json, invalid json format"); + dError("failed to read mnodeIpList.json, invalid json format"); goto PARSE_OVER; } cJSON* inUse = cJSON_GetObjectItem(root, "inUse"); if (!inUse || inUse->type != cJSON_Number) { - dError("failed to read mnode mgmtIpList.json, inUse not found"); + dError("failed to read mnodeIpList.json, inUse not found"); goto PARSE_OVER; } - tsMnodeInfos.inUse = inUse->valueint; + tsDMnodeInfos.inUse = inUse->valueint; cJSON* nodeNum = cJSON_GetObjectItem(root, "nodeNum"); if (!nodeNum || nodeNum->type != cJSON_Number) { - dError("failed to read mnode mgmtIpList.json, nodeNum not found"); + dError("failed to read mnodeIpList.json, nodeNum not found"); goto PARSE_OVER; } - tsMnodeInfos.nodeNum = nodeNum->valueint; + tsDMnodeInfos.nodeNum = nodeNum->valueint; cJSON* nodeInfos = cJSON_GetObjectItem(root, "nodeInfos"); if (!nodeInfos || nodeInfos->type != cJSON_Array) { - dError("failed to read mnode mgmtIpList.json, nodeInfos not found"); + dError("failed to read mnodeIpList.json, nodeInfos not found"); goto PARSE_OVER; } int size = cJSON_GetArraySize(nodeInfos); - if (size != tsMnodeInfos.nodeNum) { - dError("failed to read mnode mgmtIpList.json, nodeInfos size not matched"); + if (size != tsDMnodeInfos.nodeNum) { + dError("failed to read mnodeIpList.json, nodeInfos size not matched"); goto PARSE_OVER; } @@ -401,24 +478,24 @@ static bool dnodeReadMnodeInfos() { cJSON *nodeId = cJSON_GetObjectItem(nodeInfo, "nodeId"); if (!nodeId || nodeId->type != cJSON_Number) { - dError("failed to read mnode mgmtIpList.json, nodeId not found"); + dError("failed to read mnodeIpList.json, nodeId not found"); goto PARSE_OVER; } - tsMnodeInfos.nodeInfos[i].nodeId = nodeId->valueint; + tsDMnodeInfos.nodeInfos[i].nodeId = nodeId->valueint; cJSON *nodeEp = cJSON_GetObjectItem(nodeInfo, "nodeEp"); if (!nodeEp || nodeEp->type != cJSON_String || nodeEp->valuestring == NULL) { - dError("failed to read mnode mgmtIpList.json, nodeName not found"); + dError("failed to read mnodeIpList.json, nodeName not found"); goto PARSE_OVER; } - strncpy(tsMnodeInfos.nodeInfos[i].nodeEp, nodeEp->valuestring, TSDB_EP_LEN); + strncpy(tsDMnodeInfos.nodeInfos[i].nodeEp, nodeEp->valuestring, TSDB_EP_LEN); } ret = true; - dPrint("read mnode iplist successed, numOfIps:%d inUse:%d", tsMnodeInfos.nodeNum, tsMnodeInfos.inUse); - for (int32_t i = 0; i < tsMnodeInfos.nodeNum; i++) { - dPrint("mnode:%d, %s", tsMnodeInfos.nodeInfos[i].nodeId, tsMnodeInfos.nodeInfos[i].nodeEp); + dPrint("read mnode iplist successed, numOfIps:%d inUse:%d", tsDMnodeInfos.nodeNum, tsDMnodeInfos.inUse); + for (int32_t i = 0; i < tsDMnodeInfos.nodeNum; i++) { + dPrint("mnode:%d, %s", tsDMnodeInfos.nodeInfos[i].nodeId, tsDMnodeInfos.nodeInfos[i].nodeEp); } PARSE_OVER: @@ -430,7 +507,7 @@ PARSE_OVER: static void dnodeSaveMnodeInfos() { char ipFile[TSDB_FILENAME_LEN] = {0}; - sprintf(ipFile, "%s/mgmtIpList.json", tsDnodeDir); + sprintf(ipFile, "%s/mnodeIpList.json", tsDnodeDir); FILE *fp = fopen(ipFile, "w"); if (!fp) return; @@ -439,13 +516,13 @@ static void dnodeSaveMnodeInfos() { char * content = calloc(1, maxLen + 1); len += snprintf(content + len, maxLen - len, "{\n"); - len += snprintf(content + len, maxLen - len, " \"inUse\": %d,\n", tsMnodeInfos.inUse); - len += snprintf(content + len, maxLen - len, " \"nodeNum\": %d,\n", tsMnodeInfos.nodeNum); + len += snprintf(content + len, maxLen - len, " \"inUse\": %d,\n", tsDMnodeInfos.inUse); + len += snprintf(content + len, maxLen - len, " \"nodeNum\": %d,\n", tsDMnodeInfos.nodeNum); len += snprintf(content + len, maxLen - len, " \"nodeInfos\": [{\n"); - for (int32_t i = 0; i < tsMnodeInfos.nodeNum; i++) { - len += snprintf(content + len, maxLen - len, " \"nodeId\": %d,\n", tsMnodeInfos.nodeInfos[i].nodeId); - len += snprintf(content + len, maxLen - len, " \"nodeEp\": \"%s\"\n", tsMnodeInfos.nodeInfos[i].nodeEp); - if (i < tsMnodeInfos.nodeNum -1) { + for (int32_t i = 0; i < tsDMnodeInfos.nodeNum; i++) { + len += snprintf(content + len, maxLen - len, " \"nodeId\": %d,\n", tsDMnodeInfos.nodeInfos[i].nodeId); + len += snprintf(content + len, maxLen - len, " \"nodeEp\": \"%s\"\n", tsDMnodeInfos.nodeInfos[i].nodeEp); + if (i < tsDMnodeInfos.nodeNum -1) { len += snprintf(content + len, maxLen - len, " },{\n"); } else { len += snprintf(content + len, maxLen - len, " }]\n"); @@ -461,11 +538,11 @@ static void dnodeSaveMnodeInfos() { } char *dnodeGetMnodeMasterEp() { - return tsMnodeInfos.nodeInfos[tsMnodeIpSet.inUse].nodeEp; + return tsDMnodeInfos.nodeInfos[tsDMnodeIpSet.inUse].nodeEp; } void* dnodeGetMnodeInfos() { - return &tsMnodeInfos; + return &tsDMnodeInfos; } static void dnodeSendStatusMsg(void *handle, void *tmrId) { @@ -508,11 +585,14 @@ static void dnodeSendStatusMsg(void *handle, void *tmrId) { .msgType = TSDB_MSG_TYPE_DM_STATUS }; - dnodeSendMsgToDnode(&tsMnodeIpSet, &rpcMsg); + SRpcIpSet ipSet; + dnodeGetMnodeIpSetForPeer(&ipSet); + dnodeSendMsgToDnode(&ipSet, &rpcMsg); } static bool dnodeReadDnodeCfg() { - char dnodeCfgFile[TSDB_FILENAME_LEN] = {0}; + char dnodeCfgFile[TSDB_FILENAME_LEN*2] = {0}; + sprintf(dnodeCfgFile, "%s/dnodeCfg.json", tsDnodeDir); FILE *fp = fopen(dnodeCfgFile, "r"); @@ -590,3 +670,24 @@ int32_t dnodeGetDnodeId() { return tsDnodeCfg.dnodeId; } +void dnodeSendRedirectMsg(SRpcMsg *rpcMsg, bool forShell) { + SRpcConnInfo connInfo; + rpcGetConnInfo(rpcMsg->handle, &connInfo); + + SRpcIpSet ipSet = {0}; + if (forShell) { + dnodeGetMnodeIpSetForShell(&ipSet); + } else { + dnodeGetMnodeIpSetForPeer(&ipSet); + } + + dTrace("msg:%s will be redirected, dnodeIp:%s user:%s, numOfIps:%d inUse:%d", taosMsg[rpcMsg->msgType], + taosIpStr(connInfo.clientIp), connInfo.user, ipSet.numOfIps, ipSet.inUse); + + for (int i = 0; i < ipSet.numOfIps; ++i) { + dTrace("mnode index:%d %s:%d", i, ipSet.fqdn[i], ipSet.port[i]); + ipSet.port[i] = htons(ipSet.port[i]); + } + + rpcSendRedirectRsp(rpcMsg->handle, &ipSet); +} diff --git a/src/dnode/src/dnodeModule.c b/src/dnode/src/dnodeModule.c index 2f3008c33e52b2c2265fd80d786e1312705468d7..16ede5353211c7acbb8834e035b34f1a69da7457 100644 --- a/src/dnode/src/dnodeModule.c +++ b/src/dnode/src/dnodeModule.c @@ -17,7 +17,6 @@ #include "os.h" #include "taosdef.h" #include "tglobal.h" -#include "trpc.h" #include "mnode.h" #include "http.h" #include "mqtt.h" @@ -46,12 +45,12 @@ static void dnodeUnSetModuleStatus(int32_t module) { } static void dnodeAllocModules() { - tsModule[TSDB_MOD_MGMT].enable = false; - tsModule[TSDB_MOD_MGMT].name = "mgmt"; - tsModule[TSDB_MOD_MGMT].initFp = mgmtInitSystem; - tsModule[TSDB_MOD_MGMT].cleanUpFp = mgmtCleanUpSystem; - tsModule[TSDB_MOD_MGMT].startFp = mgmtStartSystem; - tsModule[TSDB_MOD_MGMT].stopFp = mgmtStopSystem; + tsModule[TSDB_MOD_MNODE].enable = false; + tsModule[TSDB_MOD_MNODE].name = "mnode"; + tsModule[TSDB_MOD_MNODE].initFp = mnodeInitSystem; + tsModule[TSDB_MOD_MNODE].cleanUpFp = mnodeCleanupSystem; + tsModule[TSDB_MOD_MNODE].startFp = mnodeStartSystem; + tsModule[TSDB_MOD_MNODE].stopFp = mnodeStopSystem; tsModule[TSDB_MOD_HTTP].enable = (tsEnableHttpModule == 1); tsModule[TSDB_MOD_HTTP].name = "http"; @@ -94,8 +93,8 @@ void dnodeCleanUpModules() { } } - if (tsModule[TSDB_MOD_MGMT].enable && tsModule[TSDB_MOD_MGMT].cleanUpFp) { - (*tsModule[TSDB_MOD_MGMT].cleanUpFp)(); + if (tsModule[TSDB_MOD_MNODE].enable && tsModule[TSDB_MOD_MNODE].cleanUpFp) { + (*tsModule[TSDB_MOD_MNODE].cleanUpFp)(); } } @@ -125,7 +124,7 @@ void dnodeStartModules() { } void dnodeProcessModuleStatus(uint32_t moduleStatus) { - for (int32_t module = TSDB_MOD_MGMT; module < TSDB_MOD_HTTP; ++module) { + for (int32_t module = TSDB_MOD_MNODE; module < TSDB_MOD_HTTP; ++module) { bool enableModule = moduleStatus & (1 << module); if (!tsModule[module].enable && enableModule) { dPrint("module status:%u is received, start %s module", tsModuleStatus, tsModule[module].name); diff --git a/src/dnode/src/dnodePeer.c b/src/dnode/src/dnodePeer.c index c91da4953d2dd32aeeef480d11fc9520216f8a93..ea3af08d7146706c7fed52a88055ebdbdb07aaef 100644 --- a/src/dnode/src/dnodePeer.c +++ b/src/dnode/src/dnodePeer.c @@ -22,14 +22,14 @@ #include "os.h" #include "taosmsg.h" #include "tglobal.h" -#include "trpc.h" +#include "mnode.h" #include "dnode.h" #include "dnodeInt.h" #include "dnodeMgmt.h" #include "dnodeVWrite.h" -#include "mnode.h" +#include "dnodeMPeer.h" -extern void dnodeUpdateIpSet(SRpcIpSet *pIpSet); +extern void dnodeUpdateMnodeIpSetForPeer(SRpcIpSet *pIpSet); static void (*dnodeProcessReqMsgFp[TSDB_MSG_TYPE_MAX])(SRpcMsg *); static void dnodeProcessReqMsgFromDnode(SRpcMsg *pMsg, SRpcIpSet *); static void (*dnodeProcessRspMsgFp[TSDB_MSG_TYPE_MAX])(SRpcMsg *rpcMsg); @@ -43,16 +43,16 @@ int32_t dnodeInitServer() { dnodeProcessReqMsgFp[TSDB_MSG_TYPE_MD_ALTER_TABLE] = dnodeDispatchToVnodeWriteQueue; dnodeProcessReqMsgFp[TSDB_MSG_TYPE_MD_DROP_STABLE] = dnodeDispatchToVnodeWriteQueue; - dnodeProcessReqMsgFp[TSDB_MSG_TYPE_MD_CREATE_VNODE] = dnodeDispatchToDnodeMgmt; - dnodeProcessReqMsgFp[TSDB_MSG_TYPE_MD_DROP_VNODE] = dnodeDispatchToDnodeMgmt; - dnodeProcessReqMsgFp[TSDB_MSG_TYPE_MD_ALTER_STREAM] = dnodeDispatchToDnodeMgmt; - dnodeProcessReqMsgFp[TSDB_MSG_TYPE_MD_CONFIG_DNODE] = dnodeDispatchToDnodeMgmt; + dnodeProcessReqMsgFp[TSDB_MSG_TYPE_MD_CREATE_VNODE] = dnodeDispatchToMgmtQueue; + dnodeProcessReqMsgFp[TSDB_MSG_TYPE_MD_DROP_VNODE] = dnodeDispatchToMgmtQueue; + dnodeProcessReqMsgFp[TSDB_MSG_TYPE_MD_ALTER_STREAM] = dnodeDispatchToMgmtQueue; + dnodeProcessReqMsgFp[TSDB_MSG_TYPE_MD_CONFIG_DNODE] = dnodeDispatchToMgmtQueue; - dnodeProcessReqMsgFp[TSDB_MSG_TYPE_DM_CONFIG_TABLE] = mgmtProcessReqMsgFromDnode; - dnodeProcessReqMsgFp[TSDB_MSG_TYPE_DM_CONFIG_VNODE] = mgmtProcessReqMsgFromDnode; - dnodeProcessReqMsgFp[TSDB_MSG_TYPE_DM_GRANT] = mgmtProcessReqMsgFromDnode; - dnodeProcessReqMsgFp[TSDB_MSG_TYPE_DM_STATUS] = mgmtProcessReqMsgFromDnode; - dnodeProcessReqMsgFp[TSDB_MSG_TYPE_DM_AUTH] = mgmtProcessReqMsgFromDnode; + dnodeProcessReqMsgFp[TSDB_MSG_TYPE_DM_CONFIG_TABLE] = dnodeDispatchToMnodePeerQueue; + dnodeProcessReqMsgFp[TSDB_MSG_TYPE_DM_CONFIG_VNODE] = dnodeDispatchToMnodePeerQueue; + dnodeProcessReqMsgFp[TSDB_MSG_TYPE_DM_AUTH] = dnodeDispatchToMnodePeerQueue; + dnodeProcessReqMsgFp[TSDB_MSG_TYPE_DM_GRANT] = dnodeDispatchToMnodePeerQueue; + dnodeProcessReqMsgFp[TSDB_MSG_TYPE_DM_STATUS] = dnodeDispatchToMnodePeerQueue; SRpcInit rpcInit; memset(&rpcInit, 0, sizeof(rpcInit)); @@ -101,16 +101,14 @@ static void dnodeProcessReqMsgFromDnode(SRpcMsg *pMsg, SRpcIpSet *pIpSet) { rpcSendResponse(&rspMsg); return; } - + if (dnodeProcessReqMsgFp[pMsg->msgType]) { (*dnodeProcessReqMsgFp[pMsg->msgType])(pMsg); } else { + dTrace("RPC %p, message:%s not processed", pMsg->handle, taosMsg[pMsg->msgType]); rspMsg.code = TSDB_CODE_MSG_NOT_PROCESSED; rpcSendResponse(&rspMsg); rpcFreeCont(pMsg->pCont); - dTrace("RPC %p, message:%s not processed", pMsg->handle, taosMsg[pMsg->msgType]); - return; - } } @@ -146,12 +144,14 @@ void dnodeCleanupClient() { } static void dnodeProcessRspFromDnode(SRpcMsg *pMsg, SRpcIpSet *pIpSet) { + if (pMsg->msgType == TSDB_MSG_TYPE_DM_STATUS_RSP && pIpSet) { + dnodeUpdateMnodeIpSetForPeer(pIpSet); + } - if (dnodeProcessRspMsgFp[pMsg->msgType]) { - if (pMsg->msgType == TSDB_MSG_TYPE_DM_STATUS_RSP && pIpSet) dnodeUpdateIpSet(pIpSet); + if (dnodeProcessRspMsgFp[pMsg->msgType]) { (*dnodeProcessRspMsgFp[pMsg->msgType])(pMsg); } else { - dError("RPC %p, msg:%s is not processed", pMsg->handle, taosMsg[pMsg->msgType]); + mnodeProcessPeerRsp(pMsg); } rpcFreeCont(pMsg->pCont); @@ -167,6 +167,6 @@ void dnodeSendMsgToDnode(SRpcIpSet *ipSet, SRpcMsg *rpcMsg) { void dnodeSendMsgToDnodeRecv(SRpcMsg *rpcMsg, SRpcMsg *rpcRsp) { SRpcIpSet ipSet = {0}; - dnodeGetMnodeDnodeIpSet(&ipSet); + dnodeGetMnodeIpSetForPeer(&ipSet); rpcSendRecv(tsDnodeClientRpc, &ipSet, rpcMsg, rpcRsp); } diff --git a/src/dnode/src/dnodeShell.c b/src/dnode/src/dnodeShell.c index fbed16483996304471a0dbad1df5638bead01920..b09e14e2830960d860f2a7c448a2bd1aedf7e7d2 100644 --- a/src/dnode/src/dnodeShell.c +++ b/src/dnode/src/dnodeShell.c @@ -18,7 +18,6 @@ #include "taoserror.h" #include "taosdef.h" #include "taosmsg.h" -#include "trpc.h" #include "tglobal.h" #include "http.h" #include "mnode.h" @@ -26,6 +25,8 @@ #include "dnodeInt.h" #include "dnodeVRead.h" #include "dnodeVWrite.h" +#include "dnodeMRead.h" +#include "dnodeMWrite.h" #include "dnodeShell.h" static void (*dnodeProcessShellMsgFp[TSDB_MSG_TYPE_MAX])(SRpcMsg *); @@ -35,43 +36,41 @@ static void * tsDnodeShellRpc = NULL; static int32_t tsDnodeQueryReqNum = 0; static int32_t tsDnodeSubmitReqNum = 0; -void mgmtProcessMsgFromShell(SRpcMsg *rpcMsg); - int32_t dnodeInitShell() { dnodeProcessShellMsgFp[TSDB_MSG_TYPE_SUBMIT] = dnodeDispatchToVnodeWriteQueue; dnodeProcessShellMsgFp[TSDB_MSG_TYPE_QUERY] = dnodeDispatchToVnodeReadQueue; dnodeProcessShellMsgFp[TSDB_MSG_TYPE_FETCH] = dnodeDispatchToVnodeReadQueue; // the following message shall be treated as mnode write - dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_CONNECT] = mgmtProcessMsgFromShell; - dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_CREATE_ACCT] = mgmtProcessMsgFromShell; - dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_ALTER_ACCT] = mgmtProcessMsgFromShell; - dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_DROP_ACCT] = mgmtProcessMsgFromShell; - dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_CREATE_USER] = mgmtProcessMsgFromShell; - dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_ALTER_USER] = mgmtProcessMsgFromShell; - dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_DROP_USER] = mgmtProcessMsgFromShell; - dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_CREATE_DNODE]= mgmtProcessMsgFromShell; - dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_DROP_DNODE] = mgmtProcessMsgFromShell; - dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_CREATE_DB] = mgmtProcessMsgFromShell; - dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_DROP_DB] = mgmtProcessMsgFromShell; - dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_ALTER_DB] = mgmtProcessMsgFromShell; - dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_CREATE_TABLE]= mgmtProcessMsgFromShell; - dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_DROP_TABLE] = mgmtProcessMsgFromShell; - dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_ALTER_TABLE] = mgmtProcessMsgFromShell; - dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_ALTER_STREAM]= mgmtProcessMsgFromShell; - dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_KILL_QUERY] = mgmtProcessMsgFromShell; - dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_KILL_STREAM] = mgmtProcessMsgFromShell; - dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_KILL_CONN] = mgmtProcessMsgFromShell; - dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_HEARTBEAT] = mgmtProcessMsgFromShell; - + dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_CREATE_ACCT] = dnodeDispatchToMnodeWriteQueue; + dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_ALTER_ACCT] = dnodeDispatchToMnodeWriteQueue; + dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_DROP_ACCT] = dnodeDispatchToMnodeWriteQueue; + dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_CREATE_USER] = dnodeDispatchToMnodeWriteQueue; + dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_ALTER_USER] = dnodeDispatchToMnodeWriteQueue; + dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_DROP_USER] = dnodeDispatchToMnodeWriteQueue; + dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_CREATE_DNODE]= dnodeDispatchToMnodeWriteQueue; + dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_DROP_DNODE] = dnodeDispatchToMnodeWriteQueue; + dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_CREATE_DB] = dnodeDispatchToMnodeWriteQueue; + dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_DROP_DB] = dnodeDispatchToMnodeWriteQueue; + dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_ALTER_DB] = dnodeDispatchToMnodeWriteQueue; + dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_CREATE_TABLE]= dnodeDispatchToMnodeWriteQueue; + dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_DROP_TABLE] = dnodeDispatchToMnodeWriteQueue; + dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_ALTER_TABLE] = dnodeDispatchToMnodeWriteQueue; + dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_ALTER_STREAM]= dnodeDispatchToMnodeWriteQueue; + dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_KILL_QUERY] = dnodeDispatchToMnodeWriteQueue; + dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_KILL_STREAM] = dnodeDispatchToMnodeWriteQueue; + dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_KILL_CONN] = dnodeDispatchToMnodeWriteQueue; + dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_CONFIG_DNODE]= dnodeDispatchToMnodeWriteQueue; + // the following message shall be treated as mnode query - dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_USE_DB] = mgmtProcessMsgFromShell; - dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_TABLE_META] = mgmtProcessMsgFromShell; - dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_STABLE_VGROUP]= mgmtProcessMsgFromShell; - dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_TABLES_META] = mgmtProcessMsgFromShell; - dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_SHOW] = mgmtProcessMsgFromShell; - dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_RETRIEVE] = mgmtProcessMsgFromShell; - dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_CONFIG_DNODE]= mgmtProcessMsgFromShell; + dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_HEARTBEAT] = dnodeDispatchToMnodeReadQueue; + dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_CONNECT] = dnodeDispatchToMnodeReadQueue; + dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_USE_DB] = dnodeDispatchToMnodeReadQueue; + dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_TABLE_META] = dnodeDispatchToMnodeReadQueue; + dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_STABLE_VGROUP]= dnodeDispatchToMnodeReadQueue; + dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_TABLES_META] = dnodeDispatchToMnodeReadQueue; + dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_SHOW] = dnodeDispatchToMnodeReadQueue; + dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_RETRIEVE] = dnodeDispatchToMnodeReadQueue; int32_t numOfThreads = tsNumOfCores * tsNumOfThreadsPerCore; numOfThreads = (int32_t) ((1.0 - tsRatioOfQueryThreads) * numOfThreads / 2.0); @@ -85,7 +84,7 @@ int32_t dnodeInitShell() { rpcInit.label = "SHELL"; rpcInit.numOfThreads = numOfThreads; rpcInit.cfp = dnodeProcessMsgFromShell; - rpcInit.sessions = TSDB_SESSIONS_PER_DNODE; + rpcInit.sessions = tsMaxShellConns; rpcInit.connType = TAOS_CONN_SERVER; rpcInit.idleTime = tsShellActivityTimer * 1000; rpcInit.afp = dnodeRetrieveUserAuthInfo; @@ -139,7 +138,7 @@ void dnodeProcessMsgFromShell(SRpcMsg *pMsg, SRpcIpSet *pIpSet) { } static int dnodeRetrieveUserAuthInfo(char *user, char *spi, char *encrypt, char *secret, char *ckey) { - int code = mgmtRetriveAuth(user, spi, encrypt, secret, ckey); + int code = mnodeRetriveAuth(user, spi, encrypt, secret, ckey); if (code != TSDB_CODE_NOT_READY) return code; SDMAuthMsg *pMsg = rpcMallocCont(sizeof(SDMAuthMsg)); @@ -169,6 +168,44 @@ static int dnodeRetrieveUserAuthInfo(char *user, char *spi, char *encrypt, char return rpcRsp.code; } +void *dnodeSendCfgTableToRecv(int32_t vgId, int32_t sid) { + dTrace("vgId:%d, sid:%d send config table msg to mnode", vgId, sid); + + int32_t contLen = sizeof(SDMConfigTableMsg); + SDMConfigTableMsg *pMsg = rpcMallocCont(contLen); + + pMsg->dnodeId = htonl(dnodeGetDnodeId()); + pMsg->vgId = htonl(vgId); + pMsg->sid = htonl(sid); + + SRpcMsg rpcMsg = {0}; + rpcMsg.pCont = pMsg; + rpcMsg.contLen = contLen; + rpcMsg.msgType = TSDB_MSG_TYPE_DM_CONFIG_TABLE; + + SRpcMsg rpcRsp = {0}; + dnodeSendMsgToDnodeRecv(&rpcMsg, &rpcRsp); + terrno = rpcRsp.code; + + if (rpcRsp.code != 0) { + rpcFreeCont(rpcRsp.pCont); + dError("vgId:%d, sid:%d failed to config table from mnode", vgId, sid); + return NULL; + } else { + dPrint("vgId:%d, sid:%d config table msg is received", vgId, sid); + + // delete this after debug finished + SMDCreateTableMsg *pTable = rpcRsp.pCont; + int16_t numOfColumns = htons(pTable->numOfColumns); + int16_t numOfTags = htons(pTable->numOfTags); + int32_t sid = htonl(pTable->sid); + uint64_t uid = htobe64(pTable->uid); + dPrint("table:%s, numOfColumns:%d numOfTags:%d sid:%d uid:%d", pTable->tableId, numOfColumns, numOfTags, sid, uid); + + return rpcRsp.pCont; + } +} + SDnodeStatisInfo dnodeGetStatisInfo() { SDnodeStatisInfo info = {0}; if (dnodeGetRunStatus() == TSDB_DNODE_RUN_STATUS_RUNING) { diff --git a/src/dnode/src/dnodeSystem.c b/src/dnode/src/dnodeSystem.c index a7bfc2d7d2224287dec353b7a40b239307d93a74..0f8dabd75a7b0e10263a721b77e765971d5272c0 100644 --- a/src/dnode/src/dnodeSystem.c +++ b/src/dnode/src/dnodeSystem.c @@ -28,8 +28,12 @@ int32_t main(int32_t argc, char *argv[]) { // Set global configuration file for (int32_t i = 1; i < argc; ++i) { if (strcmp(argv[i], "-c") == 0) { - if (i < argc - 1) { - strcpy(configDir, argv[++i]); + if (i < argc - 1) { + if (strlen(argv[++i]) > TSDB_FILENAME_LEN - 1) { + printf("config file path overflow"); + exit(EXIT_FAILURE); + } + strcpy(configDir, argv[i]); } else { printf("'-c' requires a parameter, default:%s\n", configDir); exit(EXIT_FAILURE); diff --git a/src/dnode/src/dnodeVRead.c b/src/dnode/src/dnodeVRead.c index 22505f2780bd220740b8523bca769ca8cb699a54..1ff868d0ff5e7ae341579265d52f1deec020a5a7 100644 --- a/src/dnode/src/dnodeVRead.c +++ b/src/dnode/src/dnodeVRead.c @@ -19,7 +19,6 @@ #include "taosmsg.h" #include "tutil.h" #include "tqueue.h" -#include "trpc.h" #include "twal.h" #include "tglobal.h" #include "dnodeInt.h" @@ -53,7 +52,7 @@ static void dnodeHandleIdleReadWorker(SReadWorker *); static SReadWorkerPool readPool; static taos_qset readQset; -int32_t dnodeInitRead() { +int32_t dnodeInitVnodeRead() { readQset = taosOpenQset(); readPool.min = 2; @@ -71,7 +70,7 @@ int32_t dnodeInitRead() { return 0; } -void dnodeCleanupRead() { +void dnodeCleanupVnodeRead() { for (int i=0; i < readPool.max; ++i) { SReadWorker *pWorker = readPool.readWorker + i; if (pWorker->thread) { @@ -142,7 +141,7 @@ void dnodeDispatchToVnodeReadQueue(SRpcMsg *pMsg) { } } -void *dnodeAllocateRqueue(void *pVnode) { +void *dnodeAllocateVnodeRqueue(void *pVnode) { taos_queue queue = taosOpenQueue(); if (queue == NULL) return NULL; @@ -172,7 +171,7 @@ void *dnodeAllocateRqueue(void *pVnode) { return queue; } -void dnodeFreeRqueue(void *rqueue) { +void dnodeFreeVnodeRqueue(void *rqueue) { taosCloseQueue(rqueue); // dynamically adjust the number of threads diff --git a/src/dnode/src/dnodeVWrite.c b/src/dnode/src/dnodeVWrite.c index a1531433ef4569634ebc530f87dd7b8b7dd640d9..20c50c5f8c3528d8be2773486d859d3d007841b6 100644 --- a/src/dnode/src/dnodeVWrite.c +++ b/src/dnode/src/dnodeVWrite.c @@ -22,9 +22,9 @@ #include "trpc.h" #include "tsdb.h" #include "twal.h" +#include "tdataformat.h" #include "tglobal.h" #include "vnode.h" -#include "tdataformat.h" #include "dnodeInt.h" #include "dnodeVWrite.h" #include "dnodeMgmt.h" @@ -54,7 +54,7 @@ static void dnodeHandleIdleWorker(SWriteWorker *pWorker); SWriteWorkerPool wWorkerPool; -int32_t dnodeInitWrite() { +int32_t dnodeInitVnodeWrite() { wWorkerPool.max = tsNumOfCores; wWorkerPool.writeWorker = (SWriteWorker *)calloc(sizeof(SWriteWorker), wWorkerPool.max); if (wWorkerPool.writeWorker == NULL) return -1; @@ -67,7 +67,7 @@ int32_t dnodeInitWrite() { return 0; } -void dnodeCleanupWrite() { +void dnodeCleanupVnodeWrite() { for (int32_t i = 0; i < wWorkerPool.max; ++i) { SWriteWorker *pWorker = wWorkerPool.writeWorker + i; if (pWorker->thread) { @@ -122,14 +122,17 @@ void dnodeDispatchToVnodeWriteQueue(SRpcMsg *pMsg) { } } -void *dnodeAllocateWqueue(void *pVnode) { +void *dnodeAllocateVnodeWqueue(void *pVnode) { SWriteWorker *pWorker = wWorkerPool.writeWorker + wWorkerPool.nextId; void *queue = taosOpenQueue(); if (queue == NULL) return NULL; if (pWorker->qset == NULL) { pWorker->qset = taosOpenQset(); - if (pWorker->qset == NULL) return NULL; + if (pWorker->qset == NULL) { + taosCloseQueue(queue); + return NULL; + } taosAddIntoQset(pWorker->qset, queue, pVnode); pWorker->qall = taosAllocateQall(); @@ -157,13 +160,13 @@ void *dnodeAllocateWqueue(void *pVnode) { return queue; } -void dnodeFreeWqueue(void *wqueue) { +void dnodeFreeVnodeWqueue(void *wqueue) { taosCloseQueue(wqueue); // dynamically adjust the number of threads } -void dnodeSendRpcWriteRsp(void *pVnode, void *param, int32_t code) { +void dnodeSendRpcVnodeWriteRsp(void *pVnode, void *param, int32_t code) { SWriteMsg *pWrite = (SWriteMsg *)param; if (code > 0) return; @@ -223,7 +226,7 @@ static void *dnodeProcessWriteQueue(void *param) { taosGetQitem(pWorker->qall, &type, &item); if (type == TAOS_QTYPE_RPC) { pWrite = (SWriteMsg *)item; - dnodeSendRpcWriteRsp(pVnode, item, pWrite->rpcMsg.code); + dnodeSendRpcVnodeWriteRsp(pVnode, item, pWrite->rpcMsg.code); } else { taosFreeQitem(item); vnodeRelease(pVnode); diff --git a/src/inc/dnode.h b/src/inc/dnode.h index 9884cf28707109e62fd8b8057f4becceaca6cb20..b561c407a3415d7db27333d96e21a72d4f159d8b 100644 --- a/src/inc/dnode.h +++ b/src/inc/dnode.h @@ -37,22 +37,33 @@ typedef enum { SDnodeRunStatus dnodeGetRunStatus(); SDnodeStatisInfo dnodeGetStatisInfo(); -void *dnodeAllocateWqueue(void *pVnode); -void dnodeFreeWqueue(void *queue); -void *dnodeAllocateRqueue(void *pVnode); -void dnodeFreeRqueue(void *rqueue); -void dnodeSendRpcWriteRsp(void *pVnode, void *param, int32_t code); - -bool dnodeIsFirstDeploy(); -char *dnodeGetMnodeMasterEp(); -void dnodeGetMnodeDnodeIpSet(void *ipSet); -void * dnodeGetMnodeInfos(); -int32_t dnodeGetDnodeId(); - -void dnodeAddClientRspHandle(uint8_t msgType, void (*fp)(SRpcMsg *rpcMsg)); -void dnodeAddServerMsgHandle(uint8_t msgType, void (*fp)(SRpcMsg *rpcMsg)); -void dnodeSendMsgToDnode(SRpcIpSet *ipSet, SRpcMsg *rpcMsg); -void dnodeSendMsgToDnodeRecv(SRpcMsg *rpcMsg, SRpcMsg *rpcRsp); +bool dnodeIsFirstDeploy(); +char * dnodeGetMnodeMasterEp(); +void dnodeGetMnodeIpSetForPeer(void *ipSet); +void dnodeGetMnodeIpSetForShell(void *ipSet); +void * dnodeGetMnodeInfos(); +int32_t dnodeGetDnodeId(); + +void dnodeAddClientRspHandle(uint8_t msgType, void (*fp)(SRpcMsg *rpcMsg)); +void dnodeSendMsgToDnode(SRpcIpSet *ipSet, SRpcMsg *rpcMsg); +void dnodeSendMsgToDnodeRecv(SRpcMsg *rpcMsg, SRpcMsg *rpcRsp); +void *dnodeSendCfgTableToRecv(int32_t vgId, int32_t sid); + +void *dnodeAllocateVnodeWqueue(void *pVnode); +void dnodeFreeVnodeWqueue(void *queue); +void *dnodeAllocateVnodeRqueue(void *pVnode); +void dnodeFreeVnodeRqueue(void *rqueue); +void dnodeSendRpcVnodeWriteRsp(void *pVnode, void *param, int32_t code); + +int32_t dnodeAllocateMnodePqueue(); +void dnodeFreeMnodePqueue(); +int32_t dnodeAllocateMnodeRqueue(); +void dnodeFreeMnodeRqueue(); +int32_t dnodeAllocateMnodeWqueue(); +void dnodeFreeMnodeWqueue(); +void dnodeSendRpcMnodeWriteRsp(void *pMsg, int32_t code); +void dnodeReprocessMnodeWriteMsg(void *pMsg); +void dnodeDelayReprocessMnodeWriteMsg(void *pMsg); #ifdef __cplusplus } diff --git a/src/inc/mnode.h b/src/inc/mnode.h index 48b1ac97bdb146985ff87d812e8f80a603f4ace4..bbb6fc53851700c70e192d4e368183acd833ea28 100644 --- a/src/inc/mnode.h +++ b/src/inc/mnode.h @@ -13,23 +13,58 @@ * along with this program. If not, see . */ -#ifndef TDENGINE_MGMT_H -#define TDENGINE_MGMT_H +#ifndef TDENGINE_MNODE_H +#define TDENGINE_MNODE_H #ifdef __cplusplus extern "C" { #endif -int32_t mgmtInitSystem(); -int32_t mgmtStartSystem(); -void mgmtCleanUpSystem(); -void mgmtStopSystem(); -void sdbUpdateSync(); +#include "trpc.h" + +struct SAcctObj; +struct SDnodeObj; +struct SUserObj; +struct SDbObj; +struct SVgObj; +struct STableObj; + +typedef struct { + int32_t len; + void * rsp; +} SMnodeRsp; -int32_t mgmtRetriveAuth(char *user, char *spi, char *encrypt, char *secret, char *ckey); -void mgmtProcessMsgFromShell(SRpcMsg *rpcMsg); -void mgmtProcessReqMsgFromDnode(SRpcMsg *rpcMsg); +typedef struct SMnodeMsg { + SRpcMsg rpcMsg; + SMnodeRsp rpcRsp; + int8_t received; + int8_t successed; + int8_t expected; + int8_t retry; + int32_t code; + struct SAcctObj * pAcct; + struct SDnodeObj *pDnode; + struct SUserObj * pUser; + struct SDbObj * pDb; + struct SVgObj * pVgroup; + struct STableObj *pTable; +} SMnodeMsg; +void mnodeCreateMsg(SMnodeMsg *pMsg, SRpcMsg *rpcMsg); +int32_t mnodeInitMsg(SMnodeMsg *pMsg); +void mnodeCleanupMsg(SMnodeMsg *pMsg); + +int32_t mnodeInitSystem(); +int32_t mnodeStartSystem(); +void mnodeCleanupSystem(); +void mnodeStopSystem(); +void sdbUpdateSync(); +bool mnodeIsRunning(); +int32_t mnodeProcessRead(SMnodeMsg *pMsg); +int32_t mnodeProcessWrite(SMnodeMsg *pMsg); +int32_t mnodeProcessPeerReq(SMnodeMsg *pMsg); +void mnodeProcessPeerRsp(SRpcMsg *pMsg); +int32_t mnodeRetriveAuth(char *user, char *spi, char *encrypt, char *secret, char *ckey); #ifdef __cplusplus } diff --git a/src/inc/taosdef.h b/src/inc/taosdef.h index b87a6b3118304010f89c52943f5ec636a15bc069..548a39ad4245d8cb36866f4c142d66bb523ce7df 100644 --- a/src/inc/taosdef.h +++ b/src/inc/taosdef.h @@ -203,6 +203,7 @@ void tsDataSwap(void *pLeft, void *pRight, int32_t type, int32_t size); #define TSDB_COL_NAME_LEN 64 #define TSDB_MAX_SAVED_SQL_LEN TSDB_MAX_COLUMNS * 64 #define TSDB_MAX_SQL_LEN TSDB_PAYLOAD_SIZE +#define TSDB_MAX_SQL_SHOW_LEN 256 #define TSDB_MAX_ALLOWED_SQL_LEN (8*1024*1024U) // sql length should be less than 6mb #define TSDB_MAX_BYTES_PER_ROW TSDB_MAX_COLUMNS * 64 @@ -337,9 +338,6 @@ void tsDataSwap(void *pLeft, void *pRight, int32_t type, int32_t size); #define TSDB_ORDER_ASC 1 #define TSDB_ORDER_DESC 2 -#define TSDB_SESSIONS_PER_VNODE (300) -#define TSDB_SESSIONS_PER_DNODE (TSDB_SESSIONS_PER_VNODE * TSDB_MAX_VNODES) - #define TSDB_DEFAULT_MNODES_HASH_SIZE 5 #define TSDB_DEFAULT_DNODES_HASH_SIZE 10 #define TSDB_DEFAULT_ACCOUNTS_HASH_SIZE 10 @@ -373,7 +371,7 @@ typedef enum { } ETableType; typedef enum { - TSDB_MOD_MGMT, + TSDB_MOD_MNODE, TSDB_MOD_HTTP, TSDB_MOD_MONITOR, TSDB_MOD_MQTT, diff --git a/src/inc/taoserror.h b/src/inc/taoserror.h index ae30efd93eb1bb58ddcf7b94ecf710bb17a3ae91..db910c9caea9331c888fc5181823682a91441fe7 100644 --- a/src/inc/taoserror.h +++ b/src/inc/taoserror.h @@ -46,145 +46,138 @@ static STaosError errors[] = { #endif // rpc -TAOS_DEFINE_ERROR(TSDB_CODE_ACTION_IN_PROGRESS, 0, 1, "action in progress") -TAOS_DEFINE_ERROR(TSDB_CODE_ACTION_NEED_REPROCESSED, 0, 3, "action need to be reprocessed") -TAOS_DEFINE_ERROR(TSDB_CODE_MSG_NOT_PROCESSED, 0, 4, "message not processed") -TAOS_DEFINE_ERROR(TSDB_CODE_ALREADY_PROCESSED, 0, 5, "message already processed") -TAOS_DEFINE_ERROR(TSDB_CODE_REDIRECT, 0, 6, "redirect") -TAOS_DEFINE_ERROR(TSDB_CODE_LAST_SESSION_NOT_FINISHED, 0, 7, "last session not finished") -TAOS_DEFINE_ERROR(TSDB_CODE_MAX_SESSIONS, 0, 8, "max sessions") // too many sessions -TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_SESSION_ID, 0, 9, "invalid session id") -TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_TRAN_ID, 0, 10, "invalid transaction id") -TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_MSG_TYPE, 0, 11, "invalid message type") -TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_MSG_LEN, 0, 12, "invalid message length") -TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_MSG_CONTENT, 0, 13, "invalid message content") -TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_MSG_VERSION, 0, 14, "invalid message version") -TAOS_DEFINE_ERROR(TSDB_CODE_UNEXPECTED_RESPONSE, 0, 15, "unexpected response") -TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_RESPONSE_TYPE, 0, 16, "invalid response type") -TAOS_DEFINE_ERROR(TSDB_CODE_MISMATCHED_METER_ID, 0, 17, "mismatched meter id") -TAOS_DEFINE_ERROR(TSDB_CODE_DISCONNECTED, 0, 18, "disconnected") -TAOS_DEFINE_ERROR(TSDB_CODE_NOT_READY, 0, 19, "not ready") // peer is not ready to process data -TAOS_DEFINE_ERROR(TSDB_CODE_TOO_SLOW, 0, 20, "too slow") -TAOS_DEFINE_ERROR(TSDB_CODE_OTHERS, 0, 21, "others") -TAOS_DEFINE_ERROR(TSDB_CODE_APP_ERROR, 0, 22, "app error") -TAOS_DEFINE_ERROR(TSDB_CODE_ALREADY_THERE, 0, 23, "already there") -TAOS_DEFINE_ERROR(TSDB_CODE_NO_RESOURCE, 0, 14, "no resource") -TAOS_DEFINE_ERROR(TSDB_CODE_OPS_NOT_SUPPORT, 0, 25, "operations not support") -TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_OPTION, 0, 26, "invalid option") -TAOS_DEFINE_ERROR(TSDB_CODE_NOT_CONFIGURED, 0, 27, "not configured") -TAOS_DEFINE_ERROR(TSDB_CODE_NODE_OFFLINE, 0, 28, "node offline") -TAOS_DEFINE_ERROR(TSDB_CODE_NETWORK_UNAVAIL, 0, 29, "network unavailable") -TAOS_DEFINE_ERROR(TSDB_CODE_AUTH_REQUIRED, 0, 30, "auth required") +TAOS_DEFINE_ERROR(TSDB_CODE_ACTION_IN_PROGRESS, 0, 0x0001, "action in progress") +TAOS_DEFINE_ERROR(TSDB_CODE_ACTION_NEED_REPROCESSED, 0, 0x0003, "action need to be reprocessed") +TAOS_DEFINE_ERROR(TSDB_CODE_MSG_NOT_PROCESSED, 0, 0x0004, "message not processed") +TAOS_DEFINE_ERROR(TSDB_CODE_ALREADY_PROCESSED, 0, 0x0005, "message already processed") +TAOS_DEFINE_ERROR(TSDB_CODE_REDIRECT, 0, 0x0006, "redirect") +TAOS_DEFINE_ERROR(TSDB_CODE_LAST_SESSION_NOT_FINISHED, 0, 0x0007, "last session not finished") +TAOS_DEFINE_ERROR(TSDB_CODE_MAX_SESSIONS, 0, 0x0008, "max sessions") // too many sessions +TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_SESSION_ID, 0, 0x0009, "invalid session id") +TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_TRAN_ID, 0, 0x000A, "invalid transaction id") +TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_MSG_TYPE, 0, 0x000B, "invalid message type") +TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_MSG_LEN, 0, 0x000C, "invalid message length") +TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_MSG_CONTENT, 0, 0x000D, "invalid message content") +TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_MSG_VERSION, 0, 0x000E, "invalid message version") +TAOS_DEFINE_ERROR(TSDB_CODE_UNEXPECTED_RESPONSE, 0, 0x000F, "unexpected response") +TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_RESPONSE_TYPE, 0, 0x0010, "invalid response type") +TAOS_DEFINE_ERROR(TSDB_CODE_MISMATCHED_METER_ID, 0, 0x0011, "mismatched meter id") +TAOS_DEFINE_ERROR(TSDB_CODE_DISCONNECTED, 0, 0x0012, "disconnected") +TAOS_DEFINE_ERROR(TSDB_CODE_NOT_READY, 0, 0x0013, "not ready") // peer is not ready to process data +TAOS_DEFINE_ERROR(TSDB_CODE_TOO_SLOW, 0, 0x0014, "too slow") +TAOS_DEFINE_ERROR(TSDB_CODE_OTHERS, 0, 0x0015, "others") +TAOS_DEFINE_ERROR(TSDB_CODE_APP_ERROR, 0, 0x0016, "app error") +TAOS_DEFINE_ERROR(TSDB_CODE_ALREADY_THERE, 0, 0x0017, "already there") +TAOS_DEFINE_ERROR(TSDB_CODE_NO_RESOURCE, 0, 0x0018, "no resource") +TAOS_DEFINE_ERROR(TSDB_CODE_OPS_NOT_SUPPORT, 0, 0x0019, "operations not support") +TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_OPTION, 0, 0x001A, "invalid option") +TAOS_DEFINE_ERROR(TSDB_CODE_NOT_CONFIGURED, 0, 0x001B, "not configured") +TAOS_DEFINE_ERROR(TSDB_CODE_NETWORK_UNAVAIL, 0, 0x001C, "network unavailable") +TAOS_DEFINE_ERROR(TSDB_CODE_AUTH_REQUIRED, 0, 0x001D, "auth required") // db -TAOS_DEFINE_ERROR(TSDB_CODE_DB_NOT_SELECTED, 0, 100, "db not selected") -TAOS_DEFINE_ERROR(TSDB_CODE_DB_ALREADY_EXIST, 0, 101, "database aleady exist") -TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_DB, 0, 102, "invalid database") -TAOS_DEFINE_ERROR(TSDB_CODE_MONITOR_DB_FORBIDDEN, 0, 103, "monitor db forbidden") +TAOS_DEFINE_ERROR(TSDB_CODE_DB_NOT_SELECTED, 0, 0x0100, "db not selected") +TAOS_DEFINE_ERROR(TSDB_CODE_DB_ALREADY_EXIST, 0, 0x0101, "database aleady exist") +TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_DB, 0, 0x0102, "invalid database") +TAOS_DEFINE_ERROR(TSDB_CODE_MONITOR_DB_FORBIDDEN, 0, 0x0103, "monitor db forbidden") // user -TAOS_DEFINE_ERROR(TSDB_CODE_USER_ALREADY_EXIST, 0, 150, "user already exist") -TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_USER, 0, 151, "invalid user") -TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_PASS, 0, 152, "invalid password") -TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_USER_FORMAT, 0, 153, "invalid user format") -TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_PASS_FORMAT, 0, 154, "invalid password format") -TAOS_DEFINE_ERROR(TSDB_CODE_NO_USER_FROM_CONN, 0, 155, "can not get user from conn") +TAOS_DEFINE_ERROR(TSDB_CODE_USER_ALREADY_EXIST, 0, 0x0180, "user already exist") +TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_USER, 0, 0x0181, "invalid user") +TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_PASS, 0, 0x0182, "invalid password") +TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_USER_FORMAT, 0, 0x0183, "invalid user format") +TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_PASS_FORMAT, 0, 0x0184, "invalid password format") +TAOS_DEFINE_ERROR(TSDB_CODE_NO_USER_FROM_CONN, 0, 0x0185, "can not get user from conn") // table -TAOS_DEFINE_ERROR(TSDB_CODE_TABLE_ALREADY_EXIST, 0, 200, "table already exist") -TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_TABLE_ID, 0, 201, "invalid table id") -TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_TABLE_TYPE, 0, 202, "invalid table typee") -TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_TABLE, 0, 203, "invalid table name") -TAOS_DEFINE_ERROR(TSDB_CODE_NOT_SUPER_TABLE, 0, 204, "no super table") // operation only available for super table -TAOS_DEFINE_ERROR(TSDB_CODE_NOT_ACTIVE_TABLE, 0, 205, "not active table") -TAOS_DEFINE_ERROR(TSDB_CODE_TABLE_ID_MISMATCH, 0, 206, "table id mismatch") -TAOS_DEFINE_ERROR(TSDB_CODE_TAG_ALREAY_EXIST, 0, 207, "tag already exist") -TAOS_DEFINE_ERROR(TSDB_CODE_TAG_NOT_EXIST, 0, 208, "tag not exist") -TAOS_DEFINE_ERROR(TSDB_CODE_FIELD_ALREAY_EXIST, 0, 209, "field already exist") -TAOS_DEFINE_ERROR(TSDB_CODE_FIELD_NOT_EXIST, 0, 210, "field not exist") -TAOS_DEFINE_ERROR(TSDB_CODE_COL_NAME_TOO_LONG, 0, 211, "column name too long") -TAOS_DEFINE_ERROR(TSDB_CODE_TOO_MANY_TAGS, 0, 211, "too many tags") +TAOS_DEFINE_ERROR(TSDB_CODE_TABLE_ALREADY_EXIST, 0, 0x0200, "table already exist") +TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_TABLE_ID, 0, 0x0201, "invalid table id") +TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_TABLE_TYPE, 0, 0x0202, "invalid table typee") +TAOS_DEFINE_ERROR(TSDB_CODE_NOT_SUPER_TABLE, 0, 0x0203, "no super table") // operation only available for super table +TAOS_DEFINE_ERROR(TSDB_CODE_TAG_ALREAY_EXIST, 0, 0x0204, "tag already exist") +TAOS_DEFINE_ERROR(TSDB_CODE_TAG_NOT_EXIST, 0, 0x0205, "tag not exist") +TAOS_DEFINE_ERROR(TSDB_CODE_FIELD_ALREAY_EXIST, 0, 0x0206, "field already exist") +TAOS_DEFINE_ERROR(TSDB_CODE_FIELD_NOT_EXIST, 0, 0x0207, "field not exist") +TAOS_DEFINE_ERROR(TSDB_CODE_COL_NAME_TOO_LONG, 0, 0x0209, "column name too long") +TAOS_DEFINE_ERROR(TSDB_CODE_TOO_MANY_TAGS, 0, 0x0209, "too many tags") // dnode & mnode -TAOS_DEFINE_ERROR(TSDB_CODE_NO_ENOUGH_DNODES, 0, 250, "no enough dnodes") -TAOS_DEFINE_ERROR(TSDB_CODE_DNODE_ALREADY_EXIST, 0, 251, "dnode already exist") -TAOS_DEFINE_ERROR(TSDB_CODE_DNODE_NOT_EXIST, 0, 252, "dnode not exist") -TAOS_DEFINE_ERROR(TSDB_CODE_NO_MASTER, 0, 253, "no master") -TAOS_DEFINE_ERROR(TSDB_CODE_NO_REMOVE_MASTER, 0, 254, "no remove master") -TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_QUERY_ID, 0, 255, "invalid query id") -TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_STREAM_ID, 0, 256, "invalid stream id") -TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_CONNECTION, 0, 257, "invalid connection") -TAOS_DEFINE_ERROR(TSDB_CODE_SDB_ERROR, 0, 258, "sdb error") -TAOS_DEFINE_ERROR(TSDB_CODE_TIMESTAMP_OUT_OF_RANGE, 0, 259, "timestamp is out of range") +TAOS_DEFINE_ERROR(TSDB_CODE_NO_ENOUGH_DNODES, 0, 0x0280, "no enough dnodes") +TAOS_DEFINE_ERROR(TSDB_CODE_DNODE_ALREADY_EXIST, 0, 0x0281, "dnode already exist") +TAOS_DEFINE_ERROR(TSDB_CODE_DNODE_NOT_EXIST, 0, 0x0282, "dnode not exist") +TAOS_DEFINE_ERROR(TSDB_CODE_NO_MASTER, 0, 0x0283, "no master") +TAOS_DEFINE_ERROR(TSDB_CODE_NO_REMOVE_MASTER, 0, 0x0284, "no remove master") +TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_QUERY_ID, 0, 0x0285, "invalid query id") +TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_STREAM_ID, 0, 0x0286, "invalid stream id") +TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_CONNECTION, 0, 0x0287, "invalid connection") +TAOS_DEFINE_ERROR(TSDB_CODE_SDB_ERROR, 0, 0x0288, "sdb error") +TAOS_DEFINE_ERROR(TSDB_CODE_TIMESTAMP_OUT_OF_RANGE, 0, 0x0289, "timestamp is out of range") // acct -TAOS_DEFINE_ERROR(TSDB_CODE_ACCT_ALREADY_EXIST, 0, 300, "accounts already exist") -TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_ACCT, 0, 301, "invalid account") -TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_ACCT_PARAMETER, 0, 302, "invalid account parameter") -TAOS_DEFINE_ERROR(TSDB_CODE_TOO_MANY_ACCTS, 0, 303, "too many accounts") -TAOS_DEFINE_ERROR(TSDB_CODE_TOO_MANY_USERS, 0, 304, "too many users") -TAOS_DEFINE_ERROR(TSDB_CODE_TOO_MANY_TABLES, 0, 305, "too many tables") -TAOS_DEFINE_ERROR(TSDB_CODE_TOO_MANY_DATABASES, 0, 306, "too many databases") -TAOS_DEFINE_ERROR(TSDB_CODE_TOO_MANY_TIME_SERIES, 0, 307, "not enough time series") +TAOS_DEFINE_ERROR(TSDB_CODE_ACCT_ALREADY_EXIST, 0, 0x0300, "accounts already exist") +TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_ACCT, 0, 0x0301, "invalid account") +TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_ACCT_PARAMETER, 0, 0x0302, "invalid account parameter") +TAOS_DEFINE_ERROR(TSDB_CODE_TOO_MANY_ACCTS, 0, 0x0303, "too many accounts") +TAOS_DEFINE_ERROR(TSDB_CODE_TOO_MANY_USERS, 0, 0x0304, "too many users") +TAOS_DEFINE_ERROR(TSDB_CODE_TOO_MANY_TABLES, 0, 0x0305, "too many tables") +TAOS_DEFINE_ERROR(TSDB_CODE_TOO_MANY_DATABASES, 0, 0x0306, "too many databases") +TAOS_DEFINE_ERROR(TSDB_CODE_TOO_MANY_TIME_SERIES, 0, 0x0307, "not enough time series") // grant -TAOS_DEFINE_ERROR(TSDB_CODE_AUTH_FAILURE, 0, 350, "auth failure") -TAOS_DEFINE_ERROR(TSDB_CODE_NO_RIGHTS, 0, 351, "no rights") -TAOS_DEFINE_ERROR(TSDB_CODE_NO_WRITE_ACCESS, 0, 352, "no write access") -TAOS_DEFINE_ERROR(TSDB_CODE_NO_READ_ACCESS, 0, 353, "no read access") -TAOS_DEFINE_ERROR(TSDB_CODE_GRANT_EXPIRED, 0, 354, "grant expired") -TAOS_DEFINE_ERROR(TSDB_CODE_GRANT_DNODE_LIMITED, 0, 355, "grant dnode limited") -TAOS_DEFINE_ERROR(TSDB_CODE_GRANT_ACCT_LIMITED, 0, 356, "grant account limited") -TAOS_DEFINE_ERROR(TSDB_CODE_GRANT_TIMESERIES_LIMITED, 0, 357, "grant timeseries limited") -TAOS_DEFINE_ERROR(TSDB_CODE_GRANT_DB_LIMITED, 0, 358, "grant db limited") -TAOS_DEFINE_ERROR(TSDB_CODE_GRANT_USER_LIMITED, 0, 359, "grant user limited") -TAOS_DEFINE_ERROR(TSDB_CODE_GRANT_CONN_LIMITED, 0, 360, "grant conn limited") -TAOS_DEFINE_ERROR(TSDB_CODE_GRANT_STREAM_LIMITED, 0, 361, "grant stream limited") -TAOS_DEFINE_ERROR(TSDB_CODE_GRANT_SPEED_LIMITED, 0, 362, "grant speed limited") -TAOS_DEFINE_ERROR(TSDB_CODE_GRANT_STORAGE_LIMITED, 0, 363, "grant storage limited") -TAOS_DEFINE_ERROR(TSDB_CODE_GRANT_QUERYTIME_LIMITED, 0, 364, "grant query time limited") -TAOS_DEFINE_ERROR(TSDB_CODE_GRANT_CPU_LIMITED, 0, 365, "grant cpu limited") +TAOS_DEFINE_ERROR(TSDB_CODE_AUTH_FAILURE, 0, 0x0380, "auth failure") +TAOS_DEFINE_ERROR(TSDB_CODE_NO_RIGHTS, 0, 0x0381, "no rights") +TAOS_DEFINE_ERROR(TSDB_CODE_NO_WRITE_ACCESS, 0, 0x0382, "no write access") +TAOS_DEFINE_ERROR(TSDB_CODE_NO_READ_ACCESS, 0, 0x0383, "no read access") +TAOS_DEFINE_ERROR(TSDB_CODE_GRANT_EXPIRED, 0, 0x0384, "grant expired") +TAOS_DEFINE_ERROR(TSDB_CODE_GRANT_DNODE_LIMITED, 0, 0x0385, "grant dnode limited") +TAOS_DEFINE_ERROR(TSDB_CODE_GRANT_ACCT_LIMITED, 0, 0x0386, "grant account limited") +TAOS_DEFINE_ERROR(TSDB_CODE_GRANT_TIMESERIES_LIMITED, 0, 0x0387, "grant timeseries limited") +TAOS_DEFINE_ERROR(TSDB_CODE_GRANT_DB_LIMITED, 0, 0x0388, "grant db limited") +TAOS_DEFINE_ERROR(TSDB_CODE_GRANT_USER_LIMITED, 0, 0x0389, "grant user limited") +TAOS_DEFINE_ERROR(TSDB_CODE_GRANT_CONN_LIMITED, 0, 0x038A, "grant conn limited") +TAOS_DEFINE_ERROR(TSDB_CODE_GRANT_STREAM_LIMITED, 0, 0x038B, "grant stream limited") +TAOS_DEFINE_ERROR(TSDB_CODE_GRANT_SPEED_LIMITED, 0, 0x038C, "grant speed limited") +TAOS_DEFINE_ERROR(TSDB_CODE_GRANT_STORAGE_LIMITED, 0, 0x038D, "grant storage limited") +TAOS_DEFINE_ERROR(TSDB_CODE_GRANT_QUERYTIME_LIMITED, 0, 0x038E, "grant query time limited") +TAOS_DEFINE_ERROR(TSDB_CODE_GRANT_CPU_LIMITED, 0, 0x038F, "grant cpu limited") // server -TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_VGROUP_ID, 0, 400, "invalid vgroup id") -TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_VNODE_ID, 0, 401, "invalid vnode id") -TAOS_DEFINE_ERROR(TSDB_CODE_NOT_ACTIVE_VNODE, 0, 402, "not active vnode") -TAOS_DEFINE_ERROR(TSDB_CODE_VG_INIT_FAILED, 0, 403, "vg init failed") -TAOS_DEFINE_ERROR(TSDB_CODE_SERV_NO_DISKSPACE, 0, 404, "server no diskspace") -TAOS_DEFINE_ERROR(TSDB_CODE_SERV_OUT_OF_MEMORY, 0, 405, "server out of memory") -TAOS_DEFINE_ERROR(TSDB_CODE_NO_DISK_PERMISSIONS, 0, 406, "no disk permissions") -TAOS_DEFINE_ERROR(TSDB_CODE_FILE_CORRUPTED, 0, 407, "file corrupted") -TAOS_DEFINE_ERROR(TSDB_CODE_MEMORY_CORRUPTED, 0, 408, "memory corrupted") -TAOS_DEFINE_ERROR(TSDB_CODE_NOT_SUCH_FILE_OR_DIR, 0, 409, "no such file or directory") +TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_VGROUP_ID, 0, 0x0400, "invalid vgroup id") +TAOS_DEFINE_ERROR(TSDB_CODE_VG_INIT_FAILED, 0, 0x0402, "vgroup init failed") +TAOS_DEFINE_ERROR(TSDB_CODE_SERV_NO_DISKSPACE, 0, 0x0403, "server no diskspace") +TAOS_DEFINE_ERROR(TSDB_CODE_SERV_OUT_OF_MEMORY, 0, 0x0404, "server out of memory") +TAOS_DEFINE_ERROR(TSDB_CODE_NO_DISK_PERMISSIONS, 0, 0x0405, "no disk permissions") +TAOS_DEFINE_ERROR(TSDB_CODE_FILE_CORRUPTED, 0, 0x0406, "file corrupted") +TAOS_DEFINE_ERROR(TSDB_CODE_MEMORY_CORRUPTED, 0, 0x0407, "memory corrupted") +TAOS_DEFINE_ERROR(TSDB_CODE_NOT_SUCH_FILE_OR_DIR, 0, 0x0408, "no such file or directory") // client -TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_CLIENT_VERSION, 0, 451, "invalid client version") -TAOS_DEFINE_ERROR(TSDB_CODE_CLI_OUT_OF_MEMORY, 0, 452, "client out of memory") -TAOS_DEFINE_ERROR(TSDB_CODE_CLI_NO_DISKSPACE, 0, 453, "client no disk space") -TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_TIME_STAMP, 0, 454, "invalid timestamp") -TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_SQL, 0, 455, "invalid sql") -TAOS_DEFINE_ERROR(TSDB_CODE_QUERY_CACHE_ERASED, 0, 456, "query cache erased") -TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_QUERY_MSG, 0, 457, "invalid query message") // failed to validate the sql expression msg by vnode -TAOS_DEFINE_ERROR(TSDB_CODE_SORTED_RES_TOO_MANY, 0, 458, "sorted res too many") // too many result for ordered super table projection query -TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_QHANDLE, 0, 459, "invalid handle") -TAOS_DEFINE_ERROR(TSDB_CODE_QUERY_CANCELLED, 0, 460, "query cancelled") -TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_IE, 0, 461, "invalid ie") -TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_VALUE, 0, 462, "invalid value") -TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_FQDN, 0, 463, "invalid FQDN") +TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_CLIENT_VERSION, 0, 0x0481, "invalid client version") +TAOS_DEFINE_ERROR(TSDB_CODE_CLI_OUT_OF_MEMORY, 0, 0x0482, "client out of memory") +TAOS_DEFINE_ERROR(TSDB_CODE_CLI_NO_DISKSPACE, 0, 0x0483, "client no disk space") +TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_TIME_STAMP, 0, 0x0484, "invalid timestamp") +TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_SQL, 0, 0x0485, "invalid sql") +TAOS_DEFINE_ERROR(TSDB_CODE_QUERY_CACHE_ERASED, 0, 0x0486, "query cache erased") +TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_QUERY_MSG, 0, 0x0487, "invalid query message") // failed to validate the sql expression msg by vnode +TAOS_DEFINE_ERROR(TSDB_CODE_SORTED_RES_TOO_MANY, 0, 0x0488, "sorted res too many") // too many result for ordered super table projection query +TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_QHANDLE, 0, 0x0489, "invalid handle") +TAOS_DEFINE_ERROR(TSDB_CODE_QUERY_CANCELLED, 0, 0x048A, "query cancelled") +TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_IE, 0, 0x048B, "invalid ie") +TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_VALUE, 0, 0x048C, "invalid value") +TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_FQDN, 0, 0x048D, "invalid FQDN") // others -TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_FILE_FORMAT, 0, 500, "invalid file format") +TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_FILE_FORMAT, 0, 0x0500, "invalid file format") // TSDB -TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_CONFIG, 0, 550, "invalid TSDB configuration") +TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_CONFIG, 0, 0x0580, "invalid TSDB configuration") #ifdef TAOS_ERROR_C }; #endif -#define TSDB_CODE_MAX_ERROR_CODE 120 #ifdef __cplusplus } diff --git a/src/inc/taosmsg.h b/src/inc/taosmsg.h index 17b975c193d20b9c3ce1b129a507588c20f0a634..ba95560f7d5804e0b24245bbcba6970db9b77af3 100644 --- a/src/inc/taosmsg.h +++ b/src/inc/taosmsg.h @@ -676,9 +676,9 @@ typedef struct { } SCMCreateDnodeMsg, SCMDropDnodeMsg; typedef struct { - uint32_t dnode; - int32_t vnode; - int32_t sid; + int32_t dnodeId; + int32_t vgId; + int32_t sid; } SDMConfigTableMsg; typedef struct { diff --git a/src/inc/tbalance.h b/src/inc/tbalance.h index fee045254534b7f15016b610a772dd30b1b76971..63437021fd1542471a37f0ad168507398ddd8192 100644 --- a/src/inc/tbalance.h +++ b/src/inc/tbalance.h @@ -26,7 +26,7 @@ struct SDnodeObj; int32_t balanceInit(); void balanceCleanUp(); void balanceNotify(); -void balanceUpdateMgmt(); +void balanceUpdateMnode(); void balanceReset(); int32_t balanceAllocVnodes(struct SVgObj *pVgroup); int32_t balanceDropDnode(struct SDnodeObj *pDnode); diff --git a/src/inc/tcq.h b/src/inc/tcq.h index d0a2097c05d2501e169e3f4347acc122c4f1430f..e025afaa0ae428b9545b5a23308830f8aa455002 100644 --- a/src/inc/tcq.h +++ b/src/inc/tcq.h @@ -19,6 +19,7 @@ extern "C" { #endif +#include "tdataformat.h" typedef int (*FCqWrite)(void *ahandle, void *pHead, int type); @@ -40,7 +41,7 @@ void cqStart(void *handle); void cqStop(void *handle); // cqCreate is called by TSDB to start an instance of CQ -void *cqCreate(void *handle, int sid, char *sqlStr, SSchema *pSchema, int columns); +void *cqCreate(void *handle, int sid, char *sqlStr, STSchema *pSchema); // cqDrop is called by TSDB to stop an instance of CQ, handle is the return value of cqCreate void cqDrop(void *handle); diff --git a/src/inc/trpc.h b/src/inc/trpc.h index 16223b813a1a738f6b0213e8d6c4f62cd0c2290c..5c5c77c251779cc4632065dd9d4468ff5c4cadb1 100644 --- a/src/inc/trpc.h +++ b/src/inc/trpc.h @@ -28,21 +28,21 @@ extern "C" { extern int tsRpcHeadSize; -typedef struct { +typedef struct SRpcIpSet { int8_t inUse; int8_t numOfIps; uint16_t port[TSDB_MAX_REPLICA]; char fqdn[TSDB_MAX_REPLICA][TSDB_FQDN_LEN]; } SRpcIpSet; -typedef struct { +typedef struct SRpcConnInfo { uint32_t clientIp; uint16_t clientPort; uint32_t serverIp; char user[TSDB_USER_LEN]; } SRpcConnInfo; -typedef struct { +typedef struct SRpcMsg { uint8_t msgType; void *pCont; int contLen; @@ -51,7 +51,7 @@ typedef struct { void *ahandle; //app handle set by client, for debug purpose } SRpcMsg; -typedef struct { +typedef struct SRpcInit { uint16_t localPort; // local port char *label; // for debug purpose int numOfThreads; // number of threads to handle connections diff --git a/src/inc/tsdb.h b/src/inc/tsdb.h index bee68b81f567bf0259edea33e944a4aa074ccdaa..c758d3aea4340f57140eea0d8f3f5ae95f94a2c6 100644 --- a/src/inc/tsdb.h +++ b/src/inc/tsdb.h @@ -43,6 +43,8 @@ typedef struct { void *cqH; int (*notifyStatus)(void *, int status); int (*eventCallBack)(void *); + void *(*cqCreateFunc)(void *handle, int sid, char *sqlStr, STSchema *pSchema); + void (*cqDropFunc)(void *handle); } STsdbAppH; // --------- TSDB REPOSITORY CONFIGURATION DEFINITION @@ -71,7 +73,7 @@ typedef void TsdbRepoT; // use void to hide implementation details from outside int tsdbCreateRepo(char *rootDir, STsdbCfg *pCfg, void *limiter); int32_t tsdbDropRepo(TsdbRepoT *repo); -TsdbRepoT *tsdbOpenRepo(char *tsdbDir, STsdbAppH *pAppH); +TsdbRepoT *tsdbOpenRepo(char *rootDir, STsdbAppH *pAppH); int32_t tsdbCloseRepo(TsdbRepoT *repo, int toCommit); int32_t tsdbConfigRepo(TsdbRepoT *repo, STsdbCfg *pCfg); @@ -92,6 +94,7 @@ typedef struct { STSchema * schema; STSchema * tagSchema; SDataRow tagValues; + char * sql; } STableCfg; int tsdbInitTableCfg(STableCfg *config, ETableType type, uint64_t uid, int32_t tid); @@ -101,6 +104,7 @@ int tsdbTableSetTagSchema(STableCfg *config, STSchema *pSchema, bool dup); int tsdbTableSetTagValue(STableCfg *config, SDataRow row, bool dup); int tsdbTableSetName(STableCfg *config, char *name, bool dup); int tsdbTableSetSName(STableCfg *config, char *sname, bool dup); +int tsdbTableSetStreamSql(STableCfg *config, char *sql, bool dup); void tsdbClearTableCfg(STableCfg *config); int32_t tsdbGetTableTagVal(TsdbRepoT *repo, STableId* id, int32_t colId, int16_t *type, int16_t *bytes, char **val); diff --git a/src/kit/shell/inc/shell.h b/src/kit/shell/inc/shell.h index 5400d9c5bafff20ecc0a42eacd7ac09ccdd2c08e..54ac45583c525ecfae690a999a70183f289151c3 100644 --- a/src/kit/shell/inc/shell.h +++ b/src/kit/shell/inc/shell.h @@ -68,7 +68,6 @@ void get_history_path(char* history); void cleanup_handler(void* arg); void exitShell(); int shellDumpResult(TAOS* con, char* fname, int* error_no, bool printMode); -void shellPrintNChar(const char* str, int length, int width); void shellGetGrantInfo(void *con); int isCommentLine(char *line); diff --git a/src/kit/shell/src/shellDarwin.c b/src/kit/shell/src/shellDarwin.c index e4ef06c4036ebd8f8c1638b63407889686c4c2d4..439ca6edada6b9ffd1f2d2a0385e94ec7831913e 100644 --- a/src/kit/shell/src/shellDarwin.c +++ b/src/kit/shell/src/shellDarwin.c @@ -96,8 +96,12 @@ void shellParseArgument(int argc, char *argv[], struct arguments *arguments) { exit(EXIT_FAILURE); } } else if (strcmp(argv[i], "-c") == 0) { - if (i < argc - 1) { - strcpy(configDir, argv[++i]); + if (i < argc - 1) { + if (strlen(argv[++i]) > TSDB_FILENAME_LEN - 1) { + fprintf(stderr, "config file path: %s overflow max len %d\n", argv[i], TSDB_FILENAME_LEN - 1); + exit(EXIT_FAILURE); + } + strcpy(configDir, argv[i]); } else { fprintf(stderr, "Option -c requires an argument\n"); exit(EXIT_FAILURE); @@ -335,48 +339,20 @@ void *shellLoopQuery(void *arg) { tscError("failed to malloc command"); return NULL; } - while (1) { - // Read command from shell. + do { + // Read command from shell. memset(command, 0, MAX_COMMAND_SIZE); set_terminal_mode(); shellReadCommand(con, command); reset_terminal_mode(); - - // Run the command - shellRunCommand(con, command); - } + } while (shellRunCommand(con, command) == 0); pthread_cleanup_pop(1); return NULL; } -void shellPrintNChar(const char *str, int length, int width) { - int pos = 0, cols = 0; - while (pos < length) { - wchar_t wc; - pos += mbtowc(&wc, str + pos, MB_CUR_MAX); - if (pos > length) { - break; - } - - int w = wcwidth(wc); - if (w > 0) { - if (width > 0 && cols + w > width) { - break; - } - printf("%lc", wc); - cols += w; - } - } - - for (; cols < width; cols++) { - putchar(' '); - } -} - - int get_old_terminal_mode(struct termios *tio) { /* Make sure stdin is a terminal. */ if (!isatty(STDIN_FILENO)) { diff --git a/src/kit/shell/src/shellEngine.c b/src/kit/shell/src/shellEngine.c index 93818d7d73ce29391e1260a5d95f41ad1b0c4fb9..5b4da875de38672df9f1b76ce6c9174b06f4318f 100644 --- a/src/kit/shell/src/shellEngine.c +++ b/src/kit/shell/src/shellEngine.c @@ -82,20 +82,15 @@ TAOS *shellInit(SShellArguments *args) { // Check if it is temperory run if (args->commands != NULL || args->file[0] != 0) { if (args->commands != NULL) { - char *token; - token = strtok(args->commands, ";"); - while (token != NULL) { - printf("%s%s\n", PROMPT_HEADER, token); - shellRunCommand(con, token); - token = strtok(NULL, ";"); - } + printf("%s%s\n", PROMPT_HEADER, args->commands); + shellRunCommand(con, args->commands); } if (args->file[0] != 0) { source_file(con, args->file); } - taos_close(con); + taos_close(con); write_history(); exit(EXIT_SUCCESS); } @@ -111,96 +106,37 @@ TAOS *shellInit(SShellArguments *args) { return con; } -void shellReplaceCtrlChar(char *str) { - _Bool ctrlOn = false; - char *pstr = NULL; - char quote = 0; - for (pstr = str; *str != '\0'; ++str) { - if (ctrlOn) { - switch (*str) { - case 'n': - *pstr = '\n'; - pstr++; - break; - case 'r': - *pstr = '\r'; - pstr++; - break; - case 't': - *pstr = '\t'; - pstr++; - break; - case 'G': - *pstr++ = '\\'; - *pstr++ = *str; - break; - case '\\': - *pstr = '\\'; - pstr++; - break; - case '\'': - case '"': - if (quote) { - *pstr++ = '\\'; - *pstr++ = *str; - } - break; - default: - *pstr = *str; - pstr++; - break; - } - ctrlOn = false; - } else { - if (*str == '\\') { - ctrlOn = true; - } else { - if (quote == *str) { - quote = 0; - } else if (*str == '\'' || *str == '"') { - quote = *str; - } - *pstr = *str; - pstr++; - } +static bool isEmptyCommand(const char* cmd) { + for (char c = *cmd++; c != 0; c = *cmd++) { + if (c != ' ' && c != '\t' && c != ';') { + return false; } } - *pstr = '\0'; + return true; } -int32_t shellRunCommand(TAOS *con, char *command) { + +static int32_t shellRunSingleCommand(TAOS *con, char *command) { /* If command is empty just return */ - if (regex_match(command, "^[ \t;]*$", REG_EXTENDED)) { + if (isEmptyCommand(command)) { return 0; } - /* Update the history vector. */ - if (history.hstart == history.hend || - history.hist[(history.hend + MAX_HISTORY_SIZE - 1) % MAX_HISTORY_SIZE] == NULL || - strcmp(command, history.hist[(history.hend + MAX_HISTORY_SIZE - 1) % MAX_HISTORY_SIZE]) != 0) { - if (history.hist[history.hend] != NULL) { - tfree(history.hist[history.hend]); - } - history.hist[history.hend] = strdup(command); - - history.hend = (history.hend + 1) % MAX_HISTORY_SIZE; - if (history.hend == history.hstart) { - history.hstart = (history.hstart + 1) % MAX_HISTORY_SIZE; - } - } - - shellReplaceCtrlChar(command); - // Analyse the command. if (regex_match(command, "^[ \t]*(quit|q|exit)[ \t;]*$", REG_EXTENDED | REG_ICASE)) { taos_close(con); write_history(); return -1; - } else if (regex_match(command, "^[\t ]*clear[ \t;]*$", REG_EXTENDED | REG_ICASE)) { + } + + if (regex_match(command, "^[\t ]*clear[ \t;]*$", REG_EXTENDED | REG_ICASE)) { // If clear the screen. system("clear"); - } else if (regex_match(command, "^[\t ]*set[ \t]+max_binary_display_width[ \t]+(default|[1-9][0-9]*)[ \t;]*$", REG_EXTENDED | REG_ICASE)) { + return 0; + } + + if (regex_match(command, "^[\t ]*set[ \t]+max_binary_display_width[ \t]+(default|[1-9][0-9]*)[ \t;]*$", REG_EXTENDED | REG_ICASE)) { strtok(command, " \t"); strtok(NULL, " \t"); char* p = strtok(NULL, " \t"); @@ -209,21 +145,102 @@ int32_t shellRunCommand(TAOS *con, char *command) { } else { tsMaxBinaryDisplayWidth = atoi(p); } - } else if (regex_match(command, "^[ \t]*source[\t ]+[^ ]+[ \t;]*$", REG_EXTENDED | REG_ICASE)) { + return 0; + } + + if (regex_match(command, "^[ \t]*source[\t ]+[^ ]+[ \t;]*$", REG_EXTENDED | REG_ICASE)) { /* If source file. */ char *c_ptr = strtok(command, " ;"); assert(c_ptr != NULL); c_ptr = strtok(NULL, " ;"); assert(c_ptr != NULL); - source_file(con, c_ptr); - } else { - shellRunCommandOnServer(con, command); + return 0; } - + + shellRunCommandOnServer(con, command); return 0; } + +int32_t shellRunCommand(TAOS* con, char* command) { + /* If command is empty just return */ + if (isEmptyCommand(command)) { + return 0; + } + + /* Update the history vector. */ + if (history.hstart == history.hend || + history.hist[(history.hend + MAX_HISTORY_SIZE - 1) % MAX_HISTORY_SIZE] == NULL || + strcmp(command, history.hist[(history.hend + MAX_HISTORY_SIZE - 1) % MAX_HISTORY_SIZE]) != 0) { + if (history.hist[history.hend] != NULL) { + tfree(history.hist[history.hend]); + } + history.hist[history.hend] = strdup(command); + + history.hend = (history.hend + 1) % MAX_HISTORY_SIZE; + if (history.hend == history.hstart) { + history.hstart = (history.hstart + 1) % MAX_HISTORY_SIZE; + } + } + + bool esc = false; + char quote = 0, *cmd = command, *p = command; + for (char c = *command++; c != 0; c = *command++) { + if (esc) { + switch (c) { + case 'n': + c = '\n'; + break; + case 'r': + c = '\r'; + break; + case 't': + c = '\t'; + break; + case 'G': + *p++ = '\\'; + break; + case '\'': + case '"': + if (quote) { + *p++ = '\\'; + } + break; + } + *p++ = c; + esc = false; + continue; + } + + if (c == '\\') { + esc = true; + continue; + } + + if (quote == c) { + quote = 0; + } else if (c == '\'' || c == '"') { + quote = c; + } + + *p++ = c; + if (c == ';') { + c = *p; + *p = 0; + if (shellRunSingleCommand(con, cmd) < 0) { + return -1; + } + *p = c; + p = cmd; + } + } + + *p = 0; + return shellRunSingleCommand(con, cmd); +} + + void shellRunCommandOnServer(TAOS *con, char command[]) { int64_t st, et; wordexp_t full_path; @@ -418,7 +435,6 @@ static int dumpResultToFile(const char* fname, TAOS_RES* result) { int num_fields = taos_num_fields(result); TAOS_FIELD *fields = taos_fetch_fields(result); - int32_t* length = taos_fetch_lengths(result); int precision = taos_result_precision(result); for (int col = 0; col < num_fields; col++) { @@ -431,6 +447,7 @@ static int dumpResultToFile(const char* fname, TAOS_RES* result) { int numOfRows = 0; do { + int32_t* length = taos_fetch_lengths(result); for (int i = 0; i < num_fields; i++) { if (i > 0) { fputc(',', fp); @@ -448,6 +465,39 @@ static int dumpResultToFile(const char* fname, TAOS_RES* result) { } +static void shellPrintNChar(const char *str, int length, int width) { + int pos = 0, cols = 0; + while (pos < length) { + wchar_t wc; + int bytes = mbtowc(&wc, str + pos, MB_CUR_MAX); + if (bytes == 0) { + break; + } + pos += bytes; + if (pos > length) { + break; + } + +#ifdef WINDOWS + int w = bytes; +#else + int w = wcwidth(wc); +#endif + if (w > 0) { + if (width > 0 && cols + w > width) { + break; + } + printf("%lc", wc); + cols += w; + } + } + + for (; cols < width; cols++) { + putchar(' '); + } +} + + static void printField(const char* val, TAOS_FIELD* field, int width, int32_t length, int precision) { if (val == NULL) { int w = width; @@ -506,7 +556,6 @@ static int verticalPrintResult(TAOS_RES* result) { int num_fields = taos_num_fields(result); TAOS_FIELD *fields = taos_fetch_fields(result); - int32_t* length = taos_fetch_lengths(result); int precision = taos_result_precision(result); int maxColNameLen = 0; @@ -520,6 +569,7 @@ static int verticalPrintResult(TAOS_RES* result) { int numOfRows = 0; do { printf("*************************** %d.row ***************************\n", numOfRows + 1); + int32_t* length = taos_fetch_lengths(result); for (int i = 0; i < num_fields; i++) { TAOS_FIELD* field = fields + i; @@ -614,7 +664,6 @@ static int horizontalPrintResult(TAOS_RES* result) { int num_fields = taos_num_fields(result); TAOS_FIELD *fields = taos_fetch_fields(result); - int32_t* length = taos_fetch_lengths(result); int precision = taos_result_precision(result); int width[TSDB_MAX_COLUMNS]; @@ -626,6 +675,7 @@ static int horizontalPrintResult(TAOS_RES* result) { int numOfRows = 0; do { + int32_t* length = taos_fetch_lengths(result); for (int i = 0; i < num_fields; i++) { putchar(' '); printField(row[i], fields + i, width[i], length[i], precision); diff --git a/src/kit/shell/src/shellLinux.c b/src/kit/shell/src/shellLinux.c index 856e011a78bb63b112239f644d1423c0d21b8d04..7d035126c00351444220ab66ea340bc362219ebc 100644 --- a/src/kit/shell/src/shellLinux.c +++ b/src/kit/shell/src/shellLinux.c @@ -80,6 +80,11 @@ static error_t parse_opt(int key, char *arg, struct argp_state *state) { if (wordexp(arg, &full_path, 0) != 0) { fprintf(stderr, "Invalid path %s\n", arg); return -1; + } + if (strlen(full_path.we_wordv[0]) > TSDB_FILENAME_LEN - 1) { + fprintf(stderr, "config file path: %s overflow max len %d\n", full_path.we_wordv[0], TSDB_FILENAME_LEN - 1); + wordfree(&full_path); + return -1; } strcpy(configDir, full_path.we_wordv[0]); wordfree(&full_path); @@ -307,19 +312,13 @@ void *shellLoopQuery(void *arg) { return NULL; } - while (1) { + do { // Read command from shell. - memset(command, 0, MAX_COMMAND_SIZE); set_terminal_mode(); shellReadCommand(con, command); reset_terminal_mode(); - - // Run the command - if (shellRunCommand(con, command) != 0) { - break; - } - } + } while (shellRunCommand(con, command) == 0); tfree(command); exitShell(); @@ -329,30 +328,6 @@ void *shellLoopQuery(void *arg) { return NULL; } -void shellPrintNChar(const char *str, int length, int width) { - int pos = 0, cols = 0; - while (pos < length) { - wchar_t wc; - pos += mbtowc(&wc, str + pos, MB_CUR_MAX); - if (pos > length) { - break; - } - - int w = wcwidth(wc); - if (w > 0) { - if (width > 0 && cols + w > width) { - break; - } - printf("%lc", wc); - cols += w; - } - } - - for (; cols < width; cols++) { - putchar(' '); - } -} - int get_old_terminal_mode(struct termios *tio) { /* Make sure stdin is a terminal. */ if (!isatty(STDIN_FILENO)) { diff --git a/src/kit/shell/src/shellWindows.c b/src/kit/shell/src/shellWindows.c index c4466772766863bccd66869d434b38a1b9a47222..cf96bce5a89d76216535b68d745ea9b826cd0b01 100644 --- a/src/kit/shell/src/shellWindows.c +++ b/src/kit/shell/src/shellWindows.c @@ -76,8 +76,12 @@ void shellParseArgument(int argc, char *argv[], struct arguments *arguments) { exit(EXIT_FAILURE); } } else if (strcmp(argv[i], "-c") == 0) { - if (i < argc - 1) { - strcpy(configDir, argv[++i]); + if (i < argc - 1) { + if (strlen(argv[++i]) > TSDB_FILENAME_LEN - 1) { + fprintf(stderr, "config file path: %s overflow max len %d\n", argv[i], TSDB_FILENAME_LEN - 1); + exit(EXIT_FAILURE); + } + strcpy(configDir, argv[i]); } else { fprintf(stderr, "Option -c requires an argument\n"); exit(EXIT_FAILURE); @@ -203,46 +207,17 @@ void *shellLoopQuery(void *arg) { char *command = malloc(MAX_COMMAND_SIZE); if (command == NULL) return NULL; - while (1) { + do { memset(command, 0, MAX_COMMAND_SIZE); shellPrintPrompt(); // Read command from shell. shellReadCommand(con, command); - - // Run the command - shellRunCommand(con, command); - } + } while (shellRunCommand(con, command) == 0); return NULL; } -void shellPrintNChar(const char *str, int length, int width) { - int pos = 0, cols = 0; - while (pos < length) { - wchar_t wc; - int bytes = mbtowc(&wc, str + pos, MB_CUR_MAX); - pos += bytes; - if (pos > length) { - break; - } - - int w = bytes; - if (w > 0) { - if (width > 0 && cols + w > width) { - break; - } - printf("%lc", wc); - cols += w; - } - } - - for (; cols < width; cols++) { - putchar(' '); - } -} - - void get_history_path(char *history) { sprintf(history, "%s/%s", ".", HISTORY_FILE); } void exitShell() { exit(EXIT_SUCCESS); } diff --git a/src/kit/taosdemo/taosdemo.c b/src/kit/taosdemo/taosdemo.c index 04194c6127f2b825735dcc7cf102c0a321f3079b..cf84ecad43b953a2ba0432fee66992866eced324 100644 --- a/src/kit/taosdemo/taosdemo.c +++ b/src/kit/taosdemo/taosdemo.c @@ -49,7 +49,7 @@ static struct argp_option options[] = { {0, 'h', "host", 0, "The host to connect to TDEngine. Default is localhost.", 0}, {0, 'p', "port", 0, "The TCP/IP port number to use for the connection. Default is 0.", 1}, {0, 'u', "user", 0, "The TDEngine user name to use when connecting to the server. Default is 'root'.", 2}, - {0, 'a', "password", 0, "The password to use when connecting to the server. Default is 'taosdata'.", 3}, + {0, 'P', "password", 0, "The password to use when connecting to the server. Default is 'taosdata'.", 3}, {0, 'd', "database", 0, "Destination database. Default is 'test'.", 3}, {0, 'm', "table_prefix", 0, "Table prefix name. Default is 't'.", 3}, {0, 'M', 0, 0, "Use metric flag.", 13}, @@ -58,12 +58,15 @@ static struct argp_option options[] = { {0, 'b', "type_of_cols", 0, "The data_type of columns: 'INT', 'TINYINT', 'SMALLINT', 'BIGINT', 'FLOAT', 'DOUBLE', 'BINARY'. Default is 'INT'.", 7}, {0, 'w', "length_of_binary", 0, "The length of data_type 'BINARY'. Only applicable when type of cols is 'BINARY'. Default is 8", 8}, {0, 'l', "num_of_cols_per_record", 0, "The number of columns per record. Default is 3.", 8}, - {0, 'c', "num_of_conns", 0, "The number of connections. Default is 10.", 9}, + {0, 'T', "num_of_threads", 0, "The number of threads. Default is 10.", 9}, {0, 'r', "num_of_records_per_req", 0, "The number of records per request. Default is 1000.", 10}, {0, 't', "num_of_tables", 0, "The number of tables. Default is 10000.", 11}, {0, 'n', "num_of_records_per_table", 0, "The number of records per table. Default is 100000.", 12}, - {0, 'f', "config_directory", 0, "Configuration directory. Default is '/etc/taos/'.", 14}, + {0, 'c', "config_directory", 0, "Configuration directory. Default is '/etc/taos/'.", 14}, {0, 'x', 0, 0, "Insert only flag.", 13}, + {0, 'O', "order", 0, "Insert mode--0: In order, 1: Out of order. Default is in order.", 14}, + {0, 'R', "rate", 0, "Out of order data's rate--if order=1 Default 10, min: 0, max: 50.", 14}, + {0, 'D', "delete table", 0, "Delete data methods——0: don't delete, 1: delete by table, 2: delete by stable, 3: delete by database", 14}, {0}}; /* Used by main to communicate with parse_opt. */ @@ -81,11 +84,14 @@ typedef struct DemoArguments { char *datatype[MAX_NUM_DATATYPE]; int len_of_binary; int num_of_CPR; - int num_of_connections; + int num_of_threads; int num_of_RPR; int num_of_tables; int num_of_DPT; int abort; + int order; + int rate; + int method_of_delete; char **arg_list; } SDemoArguments; @@ -106,7 +112,7 @@ static error_t parse_opt(int key, char *arg, struct argp_state *state) { case 'u': arguments->user = arg; break; - case 'a': + case 'P': arguments->password = arg; break; case 'o': @@ -115,8 +121,8 @@ static error_t parse_opt(int key, char *arg, struct argp_state *state) { case 'q': arguments->mode = atoi(arg); break; - case 'c': - arguments->num_of_connections = atoi(arg); + case 'T': + arguments->num_of_threads = atoi(arg); break; case 'r': arguments->num_of_RPR = atoi(arg); @@ -176,7 +182,7 @@ static error_t parse_opt(int key, char *arg, struct argp_state *state) { case 'x': arguments->insert_only = true; break; - case 'f': + case 'c': if (wordexp(arg, &full_path, 0) != 0) { fprintf(stderr, "Invalid path %s\n", arg); return -1; @@ -184,6 +190,30 @@ static error_t parse_opt(int key, char *arg, struct argp_state *state) { taos_options(TSDB_OPTION_CONFIGDIR, full_path.we_wordv[0]); wordfree(&full_path); break; + case 'O': + arguments->order = atoi(arg); + if (arguments->order > 1 || arguments->order < 0) + { + arguments->order = 0; + } else if (arguments->order == 1) + { + arguments->rate = 10; + } + break; + case 'R': + arguments->rate = atoi(arg); + if (arguments->order == 1 && (arguments->rate > 50 || arguments->rate <= 0)) + { + arguments->rate = 10; + } + break; + case 'D': + arguments->method_of_delete = atoi(arg); + if (arguments->method_of_delete < 0 || arguments->method_of_delete > 3) + { + arguments->method_of_delete = 0; + } + break; case OPT_ABORT: arguments->abort = 1; break; @@ -217,6 +247,8 @@ typedef struct { int ncols_per_record; int nrecords_per_table; int nrecords_per_request; + int data_of_order; + int data_of_rate; int64_t start_time; bool do_aggreFunc; @@ -236,6 +268,8 @@ typedef struct { int ncols_per_record; char **data_type; int len_of_binary; + int data_of_order; + int data_of_rate; sem_t *mutex_sem; int *notFinished; @@ -258,6 +292,8 @@ void *readMetric(void *sarg); void *syncWrite(void *sarg); +void *deleteTable(); + void *asyncWrite(void *sarg); void generateData(char *res, char **data_type, int num_of_cols, int64_t timestamp, int len_of_binary); @@ -291,11 +327,14 @@ int main(int argc, char *argv[]) { }, 8, // len_of_binary 1, // num_of_CPR - 1, // num_of_connections + 1, // num_of_connections/thread 1, // num_of_RPR 1, // num_of_tables 50000, // num_of_DPT 0, // abort + 0, // order + 0, // rate + 0, // method_of_delete NULL // arg_list }; @@ -304,7 +343,7 @@ int main(int argc, char *argv[]) { // For demo use, change default values for some parameters; arguments.num_of_tables = 10000; arguments.num_of_CPR = 3; - arguments.num_of_connections = 10; + arguments.num_of_threads = 10; arguments.num_of_DPT = 100000; arguments.num_of_RPR = 1000; arguments.use_metric = true; @@ -330,8 +369,11 @@ int main(int argc, char *argv[]) { char *tb_prefix = arguments.tb_prefix; int len_of_binary = arguments.len_of_binary; int ncols_per_record = arguments.num_of_CPR; + int order = arguments.order; + int rate = arguments.rate; + int method_of_delete = arguments.method_of_delete; int ntables = arguments.num_of_tables; - int nconnections = arguments.num_of_connections; + int threads = arguments.num_of_threads; int nrecords_per_table = arguments.num_of_DPT; int nrecords_per_request = arguments.num_of_RPR; bool use_metric = arguments.use_metric; @@ -371,12 +413,19 @@ int main(int argc, char *argv[]) { printf("# Binary Length(If applicable): %d\n", (strcasestr(dataString, "BINARY") != NULL) ? len_of_binary : -1); printf("# Number of Columns per record: %d\n", ncols_per_record); - printf("# Number of Connections: %d\n", nconnections); + printf("# Number of Threads: %d\n", threads); printf("# Number of Tables: %d\n", ntables); printf("# Number of Data per Table: %d\n", nrecords_per_table); printf("# Records/Request: %d\n", nrecords_per_request); printf("# Database name: %s\n", db_name); printf("# Table prefix: %s\n", tb_prefix); + if (order == 1) + { + printf("# Data order: %d\n", order); + printf("# Data out of order rate: %d\n", rate); + + } + printf("# Delete method: %d\n", method_of_delete); printf("# Test time: %d-%02d-%02d %02d:%02d:%02d\n", tm.tm_year + 1900, tm.tm_mon + 1, tm.tm_mday, tm.tm_hour, tm.tm_min, tm.tm_sec); printf("###################################################################\n\n"); @@ -392,12 +441,18 @@ int main(int argc, char *argv[]) { fprintf(fp, "# Binary Length(If applicable): %d\n", (strcasestr(dataString, "BINARY") != NULL) ? len_of_binary : -1); fprintf(fp, "# Number of Columns per record: %d\n", ncols_per_record); - fprintf(fp, "# Number of Connections: %d\n", nconnections); + fprintf(fp, "# Number of Threads: %d\n", threads); fprintf(fp, "# Number of Tables: %d\n", ntables); fprintf(fp, "# Number of Data per Table: %d\n", nrecords_per_table); fprintf(fp, "# Records/Request: %d\n", nrecords_per_request); fprintf(fp, "# Database name: %s\n", db_name); fprintf(fp, "# Table prefix: %s\n", tb_prefix); + if (order == 1) + { + printf("# Data order: %d\n", order); + printf("# Data out of order rate: %d\n", rate); + + } fprintf(fp, "# Test time: %d-%02d-%02d %02d:%02d:%02d\n", tm.tm_year + 1900, tm.tm_mon + 1, tm.tm_mday, tm.tm_hour, tm.tm_min, tm.tm_sec); fprintf(fp, "###################################################################\n\n"); @@ -414,7 +469,7 @@ int main(int argc, char *argv[]) { sprintf(command, "drop database %s;", db_name); taos_query(taos, command); - sleep(3); + sprintf(command, "create database %s;", db_name); taos_query(taos, command); @@ -479,22 +534,22 @@ int main(int argc, char *argv[]) { taos_close(taos); } /* Wait for table to create */ - sleep(5); + /* Insert data */ double ts = getCurrentTime(); printf("Inserting data......\n"); - pthread_t *pids = malloc(nconnections * sizeof(pthread_t)); - info *infos = malloc(nconnections * sizeof(info)); + pthread_t *pids = malloc(threads * sizeof(pthread_t)); + info *infos = malloc(threads * sizeof(info)); - int a = ntables / nconnections; + int a = ntables / threads; if (a < 1) { - nconnections = ntables; + threads = ntables; a = 1; } - int b = ntables % nconnections; + int b = ntables % threads; int last = 0; - for (int i = 0; i < nconnections; i++) { + for (int i = 0; i < threads; i++) { info *t_info = infos + i; t_info->threadID = i; strcpy(t_info->db_name, db_name); @@ -507,6 +562,8 @@ int main(int argc, char *argv[]) { t_info->len_of_binary = len_of_binary; t_info->nrecords_per_request = nrecords_per_request; t_info->start_table_id = last; + t_info->data_of_order = order; + t_info->data_of_rate = rate; t_info->end_table_id = i < b ? last + a : last + a - 1; last = t_info->end_table_id + 1; @@ -520,15 +577,15 @@ int main(int argc, char *argv[]) { pthread_create(pids + i, NULL, asyncWrite, t_info); } } - for (int i = 0; i < nconnections; i++) { + for (int i = 0; i < threads; i++) { pthread_join(pids[i], NULL); } double t = getCurrentTime() - ts; if (query_mode == SYNC) { - printf("SYNC Insert with %d connections:\n", nconnections); + printf("SYNC Insert with %d connections:\n", threads); } else { - printf("ASYNC Insert with %d connections:\n", nconnections); + printf("ASYNC Insert with %d connections:\n", threads); } fprintf(fp, "|%10.d | %10.2f | %10.2f | %10.4f |\n\n", @@ -540,7 +597,7 @@ int main(int argc, char *argv[]) { t, ntables * nrecords_per_table, nrecords_per_request, ntables * nrecords_per_table / t); - for (int i = 0; i < nconnections; i++) { + for (int i = 0; i < threads; i++) { info *t_info = infos + i; taos_close(t_info->taos); sem_destroy(&(t_info->mutex_sem)); @@ -551,6 +608,55 @@ int main(int argc, char *argv[]) { free(infos); fclose(fp); + if (method_of_delete != 0) + { + TAOS *dtaos = taos_connect(ip_addr, user, pass, db_name, port); + double dts = getCurrentTime(); + printf("Deleteing %d table(s)......\n", ntables); + + switch (method_of_delete) + { + case 1: + // delete by table + /* Create all the tables; */ + for (int i = 0; i < ntables; i++) { + sprintf(command, "drop table %s.%s%d;", db_name, tb_prefix, i); + queryDB(dtaos, command); + } + break; + case 2: + // delete by stable + if (!use_metric) { + break; + } + else + { + sprintf(command, "drop table %s.meters;", db_name); + queryDB(dtaos, command); + } + break; + case 3: + // delete by database + sprintf(command, "drop database %s;", db_name); + queryDB(dtaos, command); + break; + default: + break; + } + + printf("Table(s) droped!\n"); + taos_close(dtaos); + + double dt = getCurrentTime() - dts; + printf("Spent %.4f seconds to drop %d tables\n", dt, ntables); + + FILE *fp = fopen(arguments.output_file, "a"); + fprintf(fp, "Spent %.4f seconds to drop %d tables\n", dt, ntables); + fclose(fp); + + } + + if (!insert_only) { // query data pthread_t read_id; @@ -735,7 +841,15 @@ void *syncWrite(void *sarg) { pstr += sprintf(pstr, "insert into %s.%s%d values", winfo->db_name, winfo->tb_prefix, tID); int k; for (k = 0; k < winfo->nrecords_per_request;) { - generateData(data, data_type, ncols_per_record, tmp_time++, len_of_binary); + int rand_num = rand() % 100; + if (winfo->data_of_order ==1 && rand_num < winfo->data_of_rate) + { + long d = tmp_time - rand() % 1000000 + rand_num; + generateData(data, data_type, ncols_per_record, d, len_of_binary); + } else + { + generateData(data, data_type, ncols_per_record, tmp_time += 1000, len_of_binary); + } pstr += sprintf(pstr, " %s", data); inserted++; k++; @@ -774,6 +888,8 @@ void *asyncWrite(void *sarg) { tb_info->mutex_sem = &(winfo->mutex_sem); tb_info->notFinished = &(winfo->notFinished); tb_info->lock_sem = &(winfo->lock_sem); + tb_info->data_of_order = winfo->data_of_order; + tb_info->data_of_rate = winfo->data_of_rate; /* char buff[BUFFER_SIZE] = "\0"; */ /* sprintf(buff, "insert into %s values (0, 0)", tb_info->tb_name); */ @@ -815,7 +931,15 @@ void callBack(void *param, TAOS_RES *res, int code) { pstr += sprintf(pstr, "insert into %s values", tb_info->tb_name); for (int i = 0; i < tb_info->nrecords_per_request; i++) { - generateData(data, datatype, ncols_per_record, tmp_time++, len_of_binary); + int rand_num = rand() % 100; + if (tb_info->data_of_order ==1 && rand_num < tb_info->data_of_rate) + { + long d = tmp_time - rand() % 1000000 + rand_num; + generateData(data, datatype, ncols_per_record, d, len_of_binary); + } else + { + generateData(data, datatype, ncols_per_record, tmp_time += 1000, len_of_binary); + } pstr += sprintf(pstr, "%s", data); tb_info->counter++; diff --git a/src/mnode/inc/mgmtDb.h b/src/mnode/inc/mgmtDb.h deleted file mode 100644 index 4ad46eee7baff5c57d8733584e027320eb5b8c84..0000000000000000000000000000000000000000 --- a/src/mnode/inc/mgmtDb.h +++ /dev/null @@ -1,56 +0,0 @@ -/* - * Copyright (c) 2019 TAOS Data, Inc. - * - * This program is free software: you can use, redistribute, and/or modify - * it under the terms of the GNU Affero General Public License, version 3 - * or later ("AGPL"), as published by the Free Software Foundation. - * - * This program is distributed in the hope that it will be useful, but WITHOUT - * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or - * FITNESS FOR A PARTICULAR PURPOSE. - * - * You should have received a copy of the GNU Affero General Public License - * along with this program. If not, see . - */ - -#ifndef TDENGINE_MGMT_DB_H -#define TDENGINE_MGMT_DB_H - -#ifdef __cplusplus -extern "C" { -#endif - -#include "mgmtDef.h" - -enum _TSDB_DB_STATUS { - TSDB_DB_STATUS_READY, - TSDB_DB_STATUS_DROPPING -}; - -// api -int32_t mgmtInitDbs(); -void mgmtCleanUpDbs(); -SDbObj *mgmtGetDb(char *db); -SDbObj *mgmtGetDbByTableId(char *db); -void * mgmtGetNextDb(void *pIter, SDbObj **pDb); -void mgmtIncDbRef(SDbObj *pDb); -void mgmtDecDbRef(SDbObj *pDb); -bool mgmtCheckIsMonitorDB(char *db, char *monitordb); -void mgmtDropAllDbs(SAcctObj *pAcct); - -// util func -void mgmtAddSuperTableIntoDb(SDbObj *pDb); -void mgmtRemoveSuperTableFromDb(SDbObj *pDb); -void mgmtAddTableIntoDb(SDbObj *pDb); -void mgmtRemoveTableFromDb(SDbObj *pDb); -void mgmtAddVgroupIntoDb(SVgObj *pVgroup); -void mgmtAddVgroupIntoDbTail(SVgObj *pVgroup); -void mgmtRemoveVgroupFromDb(SVgObj *pVgroup); -void mgmtMoveVgroupToTail(SVgObj *pVgroup); -void mgmtMoveVgroupToHead(SVgObj *pVgroup); - -#ifdef __cplusplus -} -#endif - -#endif diff --git a/src/mnode/inc/mgmtShell.h b/src/mnode/inc/mgmtShell.h deleted file mode 100644 index c3ae3e96e885fd55e6517c59d7293d341eaaba9a..0000000000000000000000000000000000000000 --- a/src/mnode/inc/mgmtShell.h +++ /dev/null @@ -1,49 +0,0 @@ -/* - * Copyright (c) 2019 TAOS Data, Inc. - * - * This program is free software: you can use, redistribute, and/or modify - * it under the terms of the GNU Affero General Public License, version 3 - * or later ("AGPL"), as published by the Free Software Foundation. - * - * This program is distributed in the hope that it will be useful, but WITHOUT - * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or - * FITNESS FOR A PARTICULAR PURPOSE. - * - * You should have received a copy of the GNU Affero General Public License - * along with this program. If not, see . - */ - -#ifndef TDENGINE_MGMT_SHELL_H -#define TDENGINE_MGMT_SHELL_H - -#ifdef __cplusplus -extern "C" { -#endif -#include "mgmtDef.h" - -int32_t mgmtInitShell(); -void mgmtCleanUpShell(); -void mgmtAddShellMsgHandle(uint8_t msgType, void (*fp)(SQueuedMsg *queuedMsg)); - -typedef int32_t (*SShowMetaFp)(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn); -typedef int32_t (*SShowRetrieveFp)(SShowObj *pShow, char *data, int32_t rows, void *pConn); -void mgmtAddShellShowMetaHandle(uint8_t showType, SShowMetaFp fp); -void mgmtAddShellShowRetrieveHandle(uint8_t showType, SShowRetrieveFp fp); - -void mgmtAddToShellQueue(SQueuedMsg *queuedMsg); -void mgmtDealyedAddToShellQueue(SQueuedMsg *queuedMsg); -void mgmtSendSimpleResp(void *thandle, int32_t code); - -bool mgmtCheckQhandle(uint64_t qhandle); -void *mgmtSaveQhandle(void *qhandle, int32_t size); -void mgmtFreeQhandle(void *qhandle, bool forceRemove); - -void *mgmtMallocQueuedMsg(SRpcMsg *rpcMsg); -void *mgmtCloneQueuedMsg(SQueuedMsg *pSrcMsg); -void mgmtFreeQueuedMsg(SQueuedMsg *pMsg); - -#ifdef __cplusplus -} -#endif - -#endif diff --git a/src/mnode/inc/mgmtVgroup.h b/src/mnode/inc/mgmtVgroup.h deleted file mode 100644 index ab0345cd2037ef92e3285bbb2880129824434b1f..0000000000000000000000000000000000000000 --- a/src/mnode/inc/mgmtVgroup.h +++ /dev/null @@ -1,56 +0,0 @@ -/* - * Copyright (c) 2019 TAOS Data, Inc. - * - * This program is free software: you can use, redistribute, and/or modify - * it under the terms of the GNU Affero General Public License, version 3 - * or later ("AGPL"), as published by the Free Software Foundation. - * - * This program is distributed in the hope that it will be useful, but WITHOUT - * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or - * FITNESS FOR A PARTICULAR PURPOSE. - * - * You should have received a copy of the GNU Affero General Public License - * along with this program. If not, see . - */ - -#ifndef TDENGINE_MGMT_VGROUP_H -#define TDENGINE_MGMT_VGROUP_H - -#ifdef __cplusplus -extern "C" { -#endif - -#include "mgmtDef.h" - -int32_t mgmtInitVgroups(); -void mgmtCleanUpVgroups(); -SVgObj *mgmtGetVgroup(int32_t vgId); -void mgmtIncVgroupRef(SVgObj *pVgroup); -void mgmtDecVgroupRef(SVgObj *pVgroup); -void mgmtDropAllDbVgroups(SDbObj *pDropDb, bool sendMsg); -void mgmtDropAllDnodeVgroups(SDnodeObj *pDropDnode); -void mgmtUpdateAllDbVgroups(SDbObj *pAlterDb); - -void * mgmtGetNextVgroup(void *pIter, SVgObj **pVgroup); -void mgmtUpdateVgroup(SVgObj *pVgroup); -void mgmtUpdateVgroupStatus(SVgObj *pVgroup, SDnodeObj *dnodeId, SVnodeLoad *pVload); - -void mgmtCreateVgroup(SQueuedMsg *pMsg, SDbObj *pDb); -void mgmtDropVgroup(SVgObj *pVgroup, void *ahandle); -void mgmtAlterVgroup(SVgObj *pVgroup, void *ahandle); -SVgObj *mgmtGetAvailableVgroup(SDbObj *pDb); - -void mgmtAddTableIntoVgroup(SVgObj *pVgroup, SChildTableObj *pTable); -void mgmtRemoveTableFromVgroup(SVgObj *pVgroup, SChildTableObj *pTable); -void mgmtSendCreateVnodeMsg(SVgObj *pVgroup, SRpcIpSet *ipSet, void *ahandle); -void mgmtSendDropVnodeMsg(int32_t vgId, SRpcIpSet *ipSet, void *ahandle); -void mgmtSendCreateVgroupMsg(SVgObj *pVgroup, void *ahandle); - -SRpcIpSet mgmtGetIpSetFromVgroup(SVgObj *pVgroup); -SRpcIpSet mgmtGetIpSetFromIp(char *ep); - -#ifdef __cplusplus -} -#endif - -#endif diff --git a/src/mnode/inc/mgmtAcct.h b/src/mnode/inc/mnodeAcct.h similarity index 57% rename from src/mnode/inc/mgmtAcct.h rename to src/mnode/inc/mnodeAcct.h index 136c28f2ae4b28deff2b8be938e4e76797f7c82b..44c3fc3cb8144c9e341e52819f67244dde1f2950 100644 --- a/src/mnode/inc/mgmtAcct.h +++ b/src/mnode/inc/mnodeAcct.h @@ -13,8 +13,8 @@ * along with this program. If not, see . */ -#ifndef TDENGINE_MGMT_ACCT_H -#define TDENGINE_MGMT_ACCT_H +#ifndef TDENGINE_MNODE_ACCT_H +#define TDENGINE_MNODE_ACCT_H #ifdef __cplusplus extern "C" { @@ -22,16 +22,16 @@ extern "C" { #include "tacct.h" -int32_t mgmtInitAccts(); -void mgmtCleanUpAccts(); -void * mgmtGetAcct(char *acctName); -void * mgmtGetNextAcct(void *pIter, SAcctObj **pAcct); -void mgmtIncAcctRef(SAcctObj *pAcct); -void mgmtDecAcctRef(SAcctObj *pAcct); -void mgmtAddDbToAcct(SAcctObj *pAcct, SDbObj *pDb); -void mgmtDropDbFromAcct(SAcctObj *pAcct, SDbObj *pDb); -void mgmtAddUserToAcct(SAcctObj *pAcct, SUserObj *pUser); -void mgmtDropUserFromAcct(SAcctObj *pAcct, SUserObj *pUser); +int32_t mnodeInitAccts(); +void mnodeCleanupAccts(); +void * mnodeGetAcct(char *acctName); +void * mnodeGetNextAcct(void *pIter, SAcctObj **pAcct); +void mnodeIncAcctRef(SAcctObj *pAcct); +void mnodeDecAcctRef(SAcctObj *pAcct); +void mnodeAddDbToAcct(SAcctObj *pAcct, SDbObj *pDb); +void mnodeDropDbFromAcct(SAcctObj *pAcct, SDbObj *pDb); +void mnodeAddUserToAcct(SAcctObj *pAcct, SUserObj *pUser); +void mnodeDropUserFromAcct(SAcctObj *pAcct, SUserObj *pUser); #ifdef __cplusplus } diff --git a/src/mnode/inc/mnodeDb.h b/src/mnode/inc/mnodeDb.h new file mode 100644 index 0000000000000000000000000000000000000000..c8557af1667016c9f5bc455175a195651fd77532 --- /dev/null +++ b/src/mnode/inc/mnodeDb.h @@ -0,0 +1,56 @@ +/* + * Copyright (c) 2019 TAOS Data, Inc. + * + * This program is free software: you can use, redistribute, and/or modify + * it under the terms of the GNU Affero General Public License, version 3 + * or later ("AGPL"), as published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ + +#ifndef TDENGINE_MNODE_DB_H +#define TDENGINE_MNODE_DB_H + +#ifdef __cplusplus +extern "C" { +#endif + +#include "mnodeDef.h" + +enum _TSDB_DB_STATUS { + TSDB_DB_STATUS_READY, + TSDB_DB_STATUS_DROPPING +}; + +// api +int32_t mnodeInitDbs(); +void mnodeCleanupDbs(); +SDbObj *mnodeGetDb(char *db); +SDbObj *mnodeGetDbByTableId(char *db); +void * mnodeGetNextDb(void *pIter, SDbObj **pDb); +void mnodeIncDbRef(SDbObj *pDb); +void mnodeDecDbRef(SDbObj *pDb); +bool mnodeCheckIsMonitorDB(char *db, char *monitordb); +void mnodeDropAllDbs(SAcctObj *pAcct); + +// util func +void mnodeAddSuperTableIntoDb(SDbObj *pDb); +void mnodeRemoveSuperTableFromDb(SDbObj *pDb); +void mnodeAddTableIntoDb(SDbObj *pDb); +void mnodeRemoveTableFromDb(SDbObj *pDb); +void mnodeAddVgroupIntoDb(SVgObj *pVgroup); +void mnodeAddVgroupIntoDbTail(SVgObj *pVgroup); +void mnodeRemoveVgroupFromDb(SVgObj *pVgroup); +void mnodeMoveVgroupToTail(SVgObj *pVgroup); +void mnodeMoveVgroupToHead(SVgObj *pVgroup); + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/src/mnode/inc/mgmtDef.h b/src/mnode/inc/mnodeDef.h similarity index 92% rename from src/mnode/inc/mgmtDef.h rename to src/mnode/inc/mnodeDef.h index 9d3e46205d1ed21d673db72997164274371b3c64..594fd3fd50b0adede5e5ff759a43e8f2ddba9ec2 100644 --- a/src/mnode/inc/mgmtDef.h +++ b/src/mnode/inc/mnodeDef.h @@ -13,8 +13,8 @@ * along with this program. If not, see . */ -#ifndef TDENGINE_MGMT_DEF_H -#define TDENGINE_MGMT_DEF_H +#ifndef TDENGINE_MNODE_DEF_H +#define TDENGINE_MNODE_DEF_H #ifdef __cplusplus extern "C" { @@ -65,7 +65,7 @@ typedef struct SMnodeObj { int8_t role; } SMnodeObj; -typedef struct { +typedef struct STableObj { char *tableId; int8_t type; } STableObj; @@ -220,6 +220,7 @@ typedef struct SAcctObj { typedef struct { int8_t type; + int32_t index; char db[TSDB_DB_NAME_LEN + 1]; void * pIter; int16_t numOfColumns; @@ -228,31 +229,10 @@ typedef struct { int32_t numOfReads; int16_t offset[TSDB_MAX_COLUMNS]; int16_t bytes[TSDB_MAX_COLUMNS]; - void * signature; uint16_t payloadLen; char payload[]; } SShowObj; -typedef struct { - uint8_t msgType; - int8_t received; - int8_t successed; - int8_t expected; - int8_t retry; - int8_t maxRetry; - int32_t contLen; - int32_t code; - void *ahandle; - void *thandle; - void *pCont; - SAcctObj *pAcct; - SDnodeObj*pDnode; - SUserObj *pUser; - SDbObj *pDb; - SVgObj *pVgroup; - STableObj *pTable; -} SQueuedMsg; - #ifdef __cplusplus } #endif diff --git a/src/mnode/inc/mgmtDnode.h b/src/mnode/inc/mnodeDnode.h similarity index 56% rename from src/mnode/inc/mgmtDnode.h rename to src/mnode/inc/mnodeDnode.h index 14960c5701b358259f695002808b876ce8b39fe4..f95a163d838fc10ea9bb9acadef7bd9df7a83769 100644 --- a/src/mnode/inc/mgmtDnode.h +++ b/src/mnode/inc/mnodeDnode.h @@ -13,8 +13,8 @@ * along with this program. If not, see . */ -#ifndef TDENGINE_CLUSTER_H -#define TDENGINE_CLUSTER_H +#ifndef TDENGINE_MNODE_DNODE_H +#define TDENGINE_MNODE_DNODE_H #ifdef __cplusplus extern "C" { @@ -27,21 +27,27 @@ typedef enum { TAOS_DN_STATUS_READY } EDnodeStatus; -int32_t mgmtInitDnodes(); -void mgmtCleanupDnodes(); +typedef enum { + TAOS_DN_ALTERNATIVE_ROLE_ANY, + TAOS_DN_ALTERNATIVE_ROLE_MNODE, + TAOS_DN_ALTERNATIVE_ROLE_VNODE +} EDnodeAlternativeRole; + +int32_t mnodeInitDnodes(); +void mnodeCleanupDnodes(); -char* mgmtGetDnodeStatusStr(int32_t dnodeStatus); +char* mnodeGetDnodeStatusStr(int32_t dnodeStatus); void mgmtMonitorDnodeModule(); -int32_t mgmtGetDnodesNum(); -int32_t mgmtGetOnlinDnodesNum(); -void * mgmtGetNextDnode(void *pIter, SDnodeObj **pDnode); -void mgmtIncDnodeRef(SDnodeObj *pDnode); -void mgmtDecDnodeRef(SDnodeObj *pDnode); -void * mgmtGetDnode(int32_t dnodeId); -void * mgmtGetDnodeByEp(char *ep); -void mgmtUpdateDnode(SDnodeObj *pDnode); -int32_t mgmtDropDnode(SDnodeObj *pDnode); +int32_t mnodeGetDnodesNum(); +int32_t mnodeGetOnlinDnodesNum(); +void * mnodeGetNextDnode(void *pIter, SDnodeObj **pDnode); +void mnodeIncDnodeRef(SDnodeObj *pDnode); +void mnodeDecDnodeRef(SDnodeObj *pDnode); +void * mnodeGetDnode(int32_t dnodeId); +void * mnodeGetDnodeByEp(char *ep); +void mnodeUpdateDnode(SDnodeObj *pDnode); +int32_t mnodeDropDnode(SDnodeObj *pDnode); extern int32_t tsAccessSquence; diff --git a/src/mnode/inc/mgmtInt.h b/src/mnode/inc/mnodeInt.h similarity index 92% rename from src/mnode/inc/mgmtInt.h rename to src/mnode/inc/mnodeInt.h index 240392f68497464d8ef53f17e6c518141d2e9a14..7405cef6f3ce9786864db8ff69a9e248c7dfc53b 100644 --- a/src/mnode/inc/mgmtInt.h +++ b/src/mnode/inc/mnodeInt.h @@ -13,8 +13,8 @@ * along with this program. If not, see . */ -#ifndef TDENGINE_MGMT_LOG_H -#define TDENGINE_MGMT_LOG_H +#ifndef TDENGINE_MNODE_LOG_H +#define TDENGINE_MNODE_LOG_H #ifdef __cplusplus extern "C" { @@ -28,7 +28,7 @@ extern int32_t sdbDebugFlag; // mnode log function #define mError(...) { if (mDebugFlag & DEBUG_ERROR) { taosPrintLog("ERROR MND ", 255, __VA_ARGS__); }} -#define mWarn(...) { if (mDebugFlag & DEBUG_WARN) { taosPrintLog("WARN MND ", mDebugFlag, __VA_ARGS__); }} +#define mWarn(...) { if (mDebugFlag & DEBUG_WARN) { taosPrintLog("WARN MND ", mDebugFlag, __VA_ARGS__); }} #define mTrace(...) { if (mDebugFlag & DEBUG_TRACE) { taosPrintLog("MND ", mDebugFlag, __VA_ARGS__); }} #define mPrint(...) { taosPrintLog("MND ", 255, __VA_ARGS__); } @@ -37,7 +37,7 @@ extern int32_t sdbDebugFlag; #define mLPrint(...) { monitorSaveLog(0, __VA_ARGS__); mPrint(__VA_ARGS__) } #define sdbError(...) { if (sdbDebugFlag & DEBUG_ERROR) { taosPrintLog("ERROR MND-SDB ", 255, __VA_ARGS__); }} -#define sdbWarn(...) { if (sdbDebugFlag & DEBUG_WARN) { taosPrintLog("WARN MND-SDB ", sdbDebugFlag, __VA_ARGS__); }} +#define sdbWarn(...) { if (sdbDebugFlag & DEBUG_WARN) { taosPrintLog("WARN MND-SDB ", sdbDebugFlag, __VA_ARGS__); }} #define sdbTrace(...) { if (sdbDebugFlag & DEBUG_TRACE) { taosPrintLog("MND-SDB ", sdbDebugFlag, __VA_ARGS__);}} #define sdbPrint(...) { taosPrintLog("MND-SDB ", 255, __VA_ARGS__); } diff --git a/src/mnode/inc/mgmtMnode.h b/src/mnode/inc/mnodeMnode.h similarity index 54% rename from src/mnode/inc/mgmtMnode.h rename to src/mnode/inc/mnodeMnode.h index 0973aa6ea67985e5736c1b90f3d7c2ba1286e642..c75deac5947f81b13687356324eb29d1b57c4362 100644 --- a/src/mnode/inc/mgmtMnode.h +++ b/src/mnode/inc/mnodeMnode.h @@ -13,8 +13,8 @@ * along with this program. If not, see . */ -#ifndef TDENGINE_MGMT_MNODE_H -#define TDENGINE_MGMT_MNODE_H +#ifndef TDENGINE_MNODE_MNODE_H +#define TDENGINE_MNODE_MNODE_H #ifdef __cplusplus extern "C" { @@ -28,23 +28,25 @@ typedef enum { TAOS_MN_STATUS_READY } EMnodeStatus; -int32_t mgmtInitMnodes(); -void mgmtCleanupMnodes(); +int32_t mnodeInitMnodes(); +void mnodeCleanupMnodes(); -int32_t mgmtAddMnode(int32_t dnodeId); -int32_t mgmtDropMnode(int32_t dnodeId); -void mgmtDropMnodeLocal(int32_t dnodeId); +int32_t mnodeAddMnode(int32_t dnodeId); +int32_t mnodeDropMnode(int32_t dnodeId); +void mnodeDropMnodeLocal(int32_t dnodeId); -void * mgmtGetMnode(int32_t mnodeId); -int32_t mgmtGetMnodesNum(); -void * mgmtGetNextMnode(void *pIter, struct SMnodeObj **pMnode); -void mgmtIncMnodeRef(struct SMnodeObj *pMnode); -void mgmtDecMnodeRef(struct SMnodeObj *pMnode); +void * mnodeGetMnode(int32_t mnodeId); +int32_t mnodeGetMnodesNum(); +void * mnodeGetNextMnode(void *pIter, struct SMnodeObj **pMnode); +void mnodeIncMnodeRef(struct SMnodeObj *pMnode); +void mnodeDecMnodeRef(struct SMnodeObj *pMnode); -char * mgmtGetMnodeRoleStr(); -void mgmtGetMnodeIpSet(SRpcIpSet *ipSet); -void mgmtGetMnodeInfos(void *mnodes); -void mgmtUpdateMnodeIpSet(); +char * mnodeGetMnodeRoleStr(); +void mnodeGetMnodeIpSetForPeer(SRpcIpSet *ipSet); +void mnodeGetMnodeIpSetForShell(SRpcIpSet *ipSet); + +void mnodeGetMnodeInfos(void *mnodes); +void mnodeUpdateMnodeIpSet(); #ifdef __cplusplus } diff --git a/src/mnode/inc/mnodePeer.h b/src/mnode/inc/mnodePeer.h new file mode 100644 index 0000000000000000000000000000000000000000..e409d90de96c4e6f0e37ca48ce212ec38efb5706 --- /dev/null +++ b/src/mnode/inc/mnodePeer.h @@ -0,0 +1,33 @@ +/* + * Copyright (c) 2019 TAOS Data, Inc. + * + * This program is free software: you can use, redistribute, and/or modify + * it under the terms of the GNU Affero General Public License, version 3 + * or later ("AGPL"), as published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ + +#ifndef TDENGINE_MNODE_PEER_H +#define TDENGINE_MNODE_PEER_H + +#ifdef __cplusplus +extern "C" { +#endif +#include "mnodeDef.h" + +void mnodeAddPeerRspHandle(uint8_t msgType, void (*fp)(SRpcMsg *rpcMsg)); +void mnodeAddPeerMsgHandle(uint8_t msgType, int32_t (*fp)(SMnodeMsg *mnodeMsg)); +int32_t mnodeProcessPeerReq(SMnodeMsg *pMsg); +void mnodeProcessPeerRsp(SRpcMsg *pMsg); + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/src/mnode/inc/mgmtProfile.h b/src/mnode/inc/mnodeProfile.h similarity index 82% rename from src/mnode/inc/mgmtProfile.h rename to src/mnode/inc/mnodeProfile.h index f33ff9c3facf8e89b825ca984f66ea0c17b60c75..0311983a3812b68bfe8253fcc94733f65c124119 100644 --- a/src/mnode/inc/mgmtProfile.h +++ b/src/mnode/inc/mnodeProfile.h @@ -13,16 +13,16 @@ * along with this program. If not, see . */ -#ifndef TDENGINE_MGMT_PROFILE_H -#define TDENGINE_MGMT_PROFILE_H +#ifndef TDENGINE_MNODE_PROFILE_H +#define TDENGINE_MNODE_PROFILE_H #ifdef __cplusplus extern "C" { #endif -#include "mgmtDef.h" +#include "mnodeDef.h" -int32_t mgmtInitProfile(); -void mgmtCleanUpProfile(); +int32_t mnodeInitProfile(); +void mnodeCleanupProfile(); #ifdef __cplusplus } diff --git a/src/mnode/inc/mgmtServer.h b/src/mnode/inc/mnodeRead.h similarity index 77% rename from src/mnode/inc/mgmtServer.h rename to src/mnode/inc/mnodeRead.h index 08e4463ad83632d8142c2eb35034eb8523494d26..5e7131f585a9ef62c64bac7462d12cde7c093173 100644 --- a/src/mnode/inc/mgmtServer.h +++ b/src/mnode/inc/mnodeRead.h @@ -13,15 +13,16 @@ * along with this program. If not, see . */ -#ifndef TDENGINE_MGMT_DSERVER_H -#define TDENGINE_MGMT_DSERVER_H +#ifndef TDENGINE_MNODE_READ_H +#define TDENGINE_MNODE_READ_H #ifdef __cplusplus extern "C" { #endif +#include "mnodeDef.h" -int32_t mgmtInitServer(); -void mgmtCleanupServer(); +void mnodeAddReadMsgHandle(uint8_t msgType, int32_t (*fp)(SMnodeMsg *mnodeMsg)); +int32_t mnodeProcessRead(SMnodeMsg *pMsg); #ifdef __cplusplus } diff --git a/src/mnode/inc/mgmtSdb.h b/src/mnode/inc/mnodeSdb.h similarity index 100% rename from src/mnode/inc/mgmtSdb.h rename to src/mnode/inc/mnodeSdb.h diff --git a/src/mnode/inc/mnodeShow.h b/src/mnode/inc/mnodeShow.h new file mode 100644 index 0000000000000000000000000000000000000000..d571eabfd8a6853ef7311d358600994cd701d9e1 --- /dev/null +++ b/src/mnode/inc/mnodeShow.h @@ -0,0 +1,36 @@ +/* + * Copyright (c) 2019 TAOS Data, Inc. + * + * This program is free software: you can use, redistribute, and/or modify + * it under the terms of the GNU Affero General Public License, version 3 + * or later ("AGPL"), as published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ + +#ifndef TDENGINE_MNODE_SHELL_H +#define TDENGINE_MNODE_SHELL_H + +#ifdef __cplusplus +extern "C" { +#endif +#include "mnodeDef.h" + +int32_t mnodeInitShow(); +void mnodeCleanUpShow(); + +typedef int32_t (*SShowMetaFp)(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn); +typedef int32_t (*SShowRetrieveFp)(SShowObj *pShow, char *data, int32_t rows, void *pConn); +void mnodeAddShowMetaHandle(uint8_t showType, SShowMetaFp fp); +void mnodeAddShowRetrieveHandle(uint8_t showType, SShowRetrieveFp fp); + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/src/mnode/inc/mgmtTable.h b/src/mnode/inc/mnodeTable.h similarity index 60% rename from src/mnode/inc/mgmtTable.h rename to src/mnode/inc/mnodeTable.h index 3ef4c6d45309a681d585f7e2a0cb2e48c35b4842..78ef0e37e8a870a01b687235cb355794a906ef06 100644 --- a/src/mnode/inc/mgmtTable.h +++ b/src/mnode/inc/mnodeTable.h @@ -20,18 +20,18 @@ extern "C" { #endif -#include "mgmtDef.h" +#include "mnodeDef.h" -int32_t mgmtInitTables(); -void mgmtCleanUpTables(); -void * mgmtGetTable(char *tableId); -void mgmtIncTableRef(void *pTable); -void mgmtDecTableRef(void *pTable); -void * mgmtGetNextChildTable(void *pIter, SChildTableObj **pTable); -void * mgmtGetNextSuperTable(void *pIter, SSuperTableObj **pTable); -void mgmtDropAllChildTables(SDbObj *pDropDb); -void mgmtDropAllSuperTables(SDbObj *pDropDb); -void mgmtDropAllChildTablesInVgroups(SVgObj *pVgroup); +int32_t mnodeInitTables(); +void mnodeCleanupTables(); +void * mnodeGetTable(char *tableId); +void mnodeIncTableRef(void *pTable); +void mnodeDecTableRef(void *pTable); +void * mnodeGetNextChildTable(void *pIter, SChildTableObj **pTable); +void * mnodeGetNextSuperTable(void *pIter, SSuperTableObj **pTable); +void mnodeDropAllChildTables(SDbObj *pDropDb); +void mnodeDropAllSuperTables(SDbObj *pDropDb); +void mnodeDropAllChildTablesInVgroups(SVgObj *pVgroup); #ifdef __cplusplus } diff --git a/src/mnode/inc/mnodeUser.h b/src/mnode/inc/mnodeUser.h new file mode 100644 index 0000000000000000000000000000000000000000..338b43b47de070d2ebe76fe23499b55f3d42d318 --- /dev/null +++ b/src/mnode/inc/mnodeUser.h @@ -0,0 +1,38 @@ +/* + * Copyright (c) 2019 TAOS Data, Inc. + * + * This program is free software: you can use, redistribute, and/or modify + * it under the terms of the GNU Affero General Public License, version 3 + * or later ("AGPL"), as published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ + +#ifndef TDENGINE_MNODE_USER_H +#define TDENGINE_MNODE_USER_H + +#ifdef __cplusplus +extern "C" { +#endif +#include "mnodeDef.h" + +int32_t mnodeInitUsers(); +void mnodeCleanupUsers(); +SUserObj *mnodeGetUser(char *name); +void * mnodeGetNextUser(void *pIter, SUserObj **pUser); +void mnodeIncUserRef(SUserObj *pUser); +void mnodeDecUserRef(SUserObj *pUser); +SUserObj *mnodeGetUserFromConn(void *pConn); +int32_t mnodeCreateUser(SAcctObj *pAcct, char *name, char *pass); +void mnodeDropAllUsers(SAcctObj *pAcct); + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/src/mnode/inc/mnodeVgroup.h b/src/mnode/inc/mnodeVgroup.h new file mode 100644 index 0000000000000000000000000000000000000000..d61145d9b82ffa643a52c0e4feb6c75443079153 --- /dev/null +++ b/src/mnode/inc/mnodeVgroup.h @@ -0,0 +1,57 @@ +/* + * Copyright (c) 2019 TAOS Data, Inc. + * + * This program is free software: you can use, redistribute, and/or modify + * it under the terms of the GNU Affero General Public License, version 3 + * or later ("AGPL"), as published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ + +#ifndef TDENGINE_MNODE_VGROUP_H +#define TDENGINE_MNODE_VGROUP_H + +#ifdef __cplusplus +extern "C" { +#endif + +struct SMnodeMsg; + +int32_t mnodeInitVgroups(); +void mnodeCleanupVgroups(); +SVgObj *mnodeGetVgroup(int32_t vgId); +void mnodeIncVgroupRef(SVgObj *pVgroup); +void mnodeDecVgroupRef(SVgObj *pVgroup); +void mnodeDropAllDbVgroups(SDbObj *pDropDb); +void mnodeSendDropAllDbVgroupsMsg(SDbObj *pDropDb); +void mnodeDropAllDnodeVgroups(SDnodeObj *pDropDnode); +void mnodeUpdateAllDbVgroups(SDbObj *pAlterDb); + +void * mnodeGetNextVgroup(void *pIter, SVgObj **pVgroup); +void mnodeUpdateVgroup(SVgObj *pVgroup); +void mnodeUpdateVgroupStatus(SVgObj *pVgroup, SDnodeObj *dnodeId, SVnodeLoad *pVload); + +int32_t mnodeCreateVgroup(struct SMnodeMsg *pMsg, SDbObj *pDb); +void mnodeDropVgroup(SVgObj *pVgroup, void *ahandle); +void mnodeAlterVgroup(SVgObj *pVgroup, void *ahandle); +SVgObj *mnodeGetAvailableVgroup(SDbObj *pDb); + +void mnodeAddTableIntoVgroup(SVgObj *pVgroup, SChildTableObj *pTable); +void mnodeRemoveTableFromVgroup(SVgObj *pVgroup, SChildTableObj *pTable); +void mnodeSendCreateVnodeMsg(SVgObj *pVgroup, SRpcIpSet *ipSet, void *ahandle); +void mnodeSendDropVnodeMsg(int32_t vgId, SRpcIpSet *ipSet, void *ahandle); +void mnodeSendCreateVgroupMsg(SVgObj *pVgroup, void *ahandle); + +SRpcIpSet mnodeGetIpSetFromVgroup(SVgObj *pVgroup); +SRpcIpSet mnodeGetIpSetFromIp(char *ep); + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/src/mnode/inc/mnodeWrite.h b/src/mnode/inc/mnodeWrite.h new file mode 100644 index 0000000000000000000000000000000000000000..ced8eeb7c053bacbf786ad4bcf765f8c6b7e21b3 --- /dev/null +++ b/src/mnode/inc/mnodeWrite.h @@ -0,0 +1,31 @@ +/* + * Copyright (c) 2019 TAOS Data, Inc. + * + * This program is free software: you can use, redistribute, and/or modify + * it under the terms of the GNU Affero General Public License, version 3 + * or later ("AGPL"), as published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ + +#ifndef TDENGINE_MNODE_WRITE_H +#define TDENGINE_MNODE_WRITE_H + +#ifdef __cplusplus +extern "C" { +#endif +#include "mnodeDef.h" + +void mnodeAddWriteMsgHandle(uint8_t msgType, int32_t (*fp)(SMnodeMsg *mnodeMsg)); +int32_t mnodeProcessRead(SMnodeMsg *pMsg); + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/src/mnode/src/mgmtServer.c b/src/mnode/src/mgmtServer.c deleted file mode 100644 index 7810189e342a1aff53d23929b6678a362baccf76..0000000000000000000000000000000000000000 --- a/src/mnode/src/mgmtServer.c +++ /dev/null @@ -1,106 +0,0 @@ -/* - * Copyright (c) 2019 TAOS Data, Inc. - * - * This program is free software: you can use, redistribute, and/or modify - * it under the terms of the GNU Affero General Public License, version 3 - * or later ("AGPL"), as published by the Free Software Foundation. - * - * This program is distributed in the hope that it will be useful, but WITHOUT - * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or - * FITNESS FOR A PARTICULAR PURPOSE. - * - * You should have received a copy of the GNU Affero General Public License - * along with this program. If not, see . - */ - -#define _DEFAULT_SOURCE -#include "os.h" -#include "taoserror.h" -#include "trpc.h" -#include "tsched.h" -#include "tsystem.h" -#include "tutil.h" -#include "tgrant.h" -#include "tbalance.h" -#include "tglobal.h" -#include "dnode.h" -#include "mgmtDef.h" -#include "mgmtInt.h" -#include "mgmtDb.h" -#include "mgmtMnode.h" -#include "mgmtProfile.h" -#include "mgmtShell.h" -#include "mgmtSdb.h" -#include "mgmtTable.h" -#include "mgmtVgroup.h" - -static void (*mgmtProcessDnodeMsgFp[TSDB_MSG_TYPE_MAX])(SRpcMsg *rpcMsg); -static void *tsMgmtServerQhandle = NULL; - -int32_t mgmtInitServer() { - - tsMgmtServerQhandle = taosInitScheduler(tsMaxShellConns, 1, "MS"); - - mPrint("server connection to dnode is opened"); - return 0; -} - -void mgmtCleanupServer() { - if (tsMgmtServerQhandle) { - taosCleanUpScheduler(tsMgmtServerQhandle); - tsMgmtServerQhandle = NULL; - } -} - -void dnodeAddServerMsgHandle(uint8_t msgType, void (*fp)(SRpcMsg *rpcMsg)) { - mgmtProcessDnodeMsgFp[msgType] = fp; -} - -static void mgmtProcessRequestFromDnode(SSchedMsg *sched) { - SRpcMsg *pMsg = sched->msg; - (*mgmtProcessDnodeMsgFp[pMsg->msgType])(pMsg); - rpcFreeCont(pMsg->pCont); - free(pMsg); -} - -static void mgmtAddToServerQueue(SRpcMsg *pMsg) { - SSchedMsg schedMsg; - schedMsg.msg = pMsg; - schedMsg.fp = mgmtProcessRequestFromDnode; - taosScheduleTask(tsMgmtServerQhandle, &schedMsg); -} - -void mgmtProcessReqMsgFromDnode(SRpcMsg *rpcMsg) { - if (mgmtProcessDnodeMsgFp[rpcMsg->msgType] == NULL) { - mError("%s is not processed in mnode", taosMsg[rpcMsg->msgType]); - mgmtSendSimpleResp(rpcMsg->handle, TSDB_CODE_MSG_NOT_PROCESSED); - rpcFreeCont(rpcMsg->pCont); - } - - if (rpcMsg->pCont == NULL) { - mgmtSendSimpleResp(rpcMsg->handle, TSDB_CODE_INVALID_MSG_LEN); - return; - } - - if (!sdbIsMaster()) { - SRpcConnInfo connInfo; - rpcGetConnInfo(rpcMsg->handle, &connInfo); - - SRpcIpSet ipSet = {0}; - dnodeGetMnodeDnodeIpSet(&ipSet); - for (int i = 0; i < ipSet.numOfIps; ++i) - ipSet.port[i] = htons(ipSet.port[i]); - - mTrace("conn from dnode ip:%s user:%s redirect msg, inUse:%d", taosIpStr(connInfo.clientIp), connInfo.user, ipSet.inUse); - for (int32_t i = 0; i < ipSet.numOfIps; ++i) { - mTrace("mnode index:%d %s:%d", i, ipSet.fqdn[i], htons(ipSet.port[i])); - } - rpcSendRedirectRsp(rpcMsg->handle, &ipSet); - return; - } - - SRpcMsg *pMsg = malloc(sizeof(SRpcMsg)); - memcpy(pMsg, rpcMsg, sizeof(SRpcMsg)); - mgmtAddToServerQueue(pMsg); -} - diff --git a/src/mnode/src/mgmtShell.c b/src/mnode/src/mgmtShell.c deleted file mode 100644 index ccbed350dc3b22ebbfde250ef2edfd0c887a1cde..0000000000000000000000000000000000000000 --- a/src/mnode/src/mgmtShell.c +++ /dev/null @@ -1,561 +0,0 @@ -/* - * Copyright (c) 2019 TAOS Data, Inc. - * - * This program is free software: you can use, redistribute, and/or modify - * it under the terms of the GNU Affero General Public License, version 3 - * or later ("AGPL"), as published by the Free Software Foundation. - * - * This program is distributed in the hope that it will be useful, but WITHOUT - * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or - * FITNESS FOR A PARTICULAR PURPOSE. - * - * You should have received a copy of the GNU Affero General Public License - * along with this program. If not, see . - */ - -#define _DEFAULT_SOURCE -#include "os.h" -#include "taosmsg.h" -#include "taoserror.h" -#include "trpc.h" -#include "tsched.h" -#include "tutil.h" -#include "ttimer.h" -#include "tgrant.h" -#include "tglobal.h" -#include "tcache.h" -#include "dnode.h" -#include "mgmtDef.h" -#include "mgmtInt.h" -#include "mgmtAcct.h" -#include "mgmtDb.h" -#include "mgmtDnode.h" -#include "mgmtMnode.h" -#include "mgmtProfile.h" -#include "mgmtSdb.h" -#include "mgmtShell.h" -#include "mgmtTable.h" -#include "mgmtUser.h" -#include "mgmtVgroup.h" - -typedef int32_t (*SShowMetaFp)(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn); -typedef int32_t (*SShowRetrieveFp)(SShowObj *pShow, char *data, int32_t rows, void *pConn); - -static bool mgmtCheckMsgReadOnly(SQueuedMsg *pMsg); -static void mgmtProcessUnSupportMsg(SRpcMsg *rpcMsg); -static void mgmtProcessShowMsg(SQueuedMsg *queuedMsg); -static void mgmtProcessRetrieveMsg(SQueuedMsg *queuedMsg); -static void mgmtProcessHeartBeatMsg(SQueuedMsg *queuedMsg); -static void mgmtProcessConnectMsg(SQueuedMsg *queuedMsg); -static void mgmtProcessUseMsg(SQueuedMsg *queuedMsg); -static void mgmtFreeShowObj(void *data); - -void *tsMgmtTmr; -static void *tsMgmtTranQhandle = NULL; -static void (*tsMgmtProcessShellMsgFp[TSDB_MSG_TYPE_MAX])(SQueuedMsg *) = {0}; -static void *tsQhandleCache = NULL; -static SShowMetaFp tsMgmtShowMetaFp[TSDB_MGMT_TABLE_MAX] = {0}; -static SShowRetrieveFp tsMgmtShowRetrieveFp[TSDB_MGMT_TABLE_MAX] = {0}; - -int32_t mgmtInitShell() { - mgmtAddShellMsgHandle(TSDB_MSG_TYPE_CM_SHOW, mgmtProcessShowMsg); - mgmtAddShellMsgHandle(TSDB_MSG_TYPE_CM_RETRIEVE, mgmtProcessRetrieveMsg); - mgmtAddShellMsgHandle(TSDB_MSG_TYPE_CM_HEARTBEAT, mgmtProcessHeartBeatMsg); - mgmtAddShellMsgHandle(TSDB_MSG_TYPE_CM_CONNECT, mgmtProcessConnectMsg); - mgmtAddShellMsgHandle(TSDB_MSG_TYPE_CM_USE_DB, mgmtProcessUseMsg); - - tsMgmtTmr = taosTmrInit((tsMaxShellConns) * 3, 200, 3600000, "MND"); - tsMgmtTranQhandle = taosInitScheduler(tsMaxShellConns, 1, "mnodeT"); - tsQhandleCache = taosCacheInitWithCb(tsMgmtTmr, 10, mgmtFreeShowObj); - - return 0; -} - -void mgmtCleanUpShell() { - if (tsMgmtTmr != NULL){ - taosTmrCleanUp(tsMgmtTmr); - tsMgmtTmr = NULL; - } - - if (tsQhandleCache != NULL) { - taosCacheCleanup(tsQhandleCache); - tsQhandleCache = NULL; - } - - if (tsMgmtTranQhandle != NULL) { - taosCleanUpScheduler(tsMgmtTranQhandle); - tsMgmtTranQhandle = NULL; - } -} - -void mgmtAddShellMsgHandle(uint8_t showType, void (*fp)(SQueuedMsg *queuedMsg)) { - tsMgmtProcessShellMsgFp[showType] = fp; -} - -void mgmtAddShellShowMetaHandle(uint8_t showType, SShowMetaFp fp) { - tsMgmtShowMetaFp[showType] = fp; -} - -void mgmtAddShellShowRetrieveHandle(uint8_t msgType, SShowRetrieveFp fp) { - tsMgmtShowRetrieveFp[msgType] = fp; -} - -void mgmtProcessTranRequest(SSchedMsg *sched) { - SQueuedMsg *queuedMsg = sched->msg; - (*tsMgmtProcessShellMsgFp[queuedMsg->msgType])(queuedMsg); - mgmtFreeQueuedMsg(queuedMsg); -} - -void mgmtAddToShellQueue(SQueuedMsg *queuedMsg) { - SSchedMsg schedMsg; - schedMsg.msg = queuedMsg; - schedMsg.fp = mgmtProcessTranRequest; - taosScheduleTask(tsMgmtTranQhandle, &schedMsg); -} - -static void mgmtDoDealyedAddToShellQueue(void *param, void *tmrId) { - mgmtAddToShellQueue(param); -} - -void mgmtDealyedAddToShellQueue(SQueuedMsg *queuedMsg) { - void *unUsed = NULL; - taosTmrReset(mgmtDoDealyedAddToShellQueue, 300, queuedMsg, tsMgmtTmr, &unUsed); -} - -void mgmtProcessMsgFromShell(SRpcMsg *rpcMsg) { - - mTrace("%p, msg:%s will be processed", rpcMsg->ahandle, taosMsg[rpcMsg->msgType]); - - if (rpcMsg->pCont == NULL) { - mgmtSendSimpleResp(rpcMsg->handle, TSDB_CODE_INVALID_MSG_LEN); - return; - } - - if (!sdbIsMaster()) { - SRpcConnInfo connInfo; - rpcGetConnInfo(rpcMsg->handle, &connInfo); - - SRpcIpSet ipSet = {0}; - mgmtGetMnodeIpSet(&ipSet); - mTrace("conn from shell ip:%s user:%s redirect msg, inUse:%d", taosIpStr(connInfo.clientIp), connInfo.user, ipSet.inUse); - for (int32_t i = 0; i < ipSet.numOfIps; ++i) { - mTrace("mnode index:%d ip:%s:%d", i, ipSet.fqdn[i], htons(ipSet.port[i])); - } - - rpcSendRedirectRsp(rpcMsg->handle, &ipSet); - return; - } - - if (grantCheck(TSDB_GRANT_TIME) != TSDB_CODE_SUCCESS) { - mgmtSendSimpleResp(rpcMsg->handle, TSDB_CODE_GRANT_EXPIRED); - rpcFreeCont(rpcMsg->pCont); - return; - } - - if (tsMgmtProcessShellMsgFp[rpcMsg->msgType] == NULL) { - mgmtProcessUnSupportMsg(rpcMsg); - rpcFreeCont(rpcMsg->pCont); - return; - } - - SQueuedMsg *pMsg = mgmtMallocQueuedMsg(rpcMsg); - if (pMsg == NULL) { - mgmtSendSimpleResp(rpcMsg->handle, TSDB_CODE_INVALID_USER); - rpcFreeCont(rpcMsg->pCont); - return; - } - - if (mgmtCheckMsgReadOnly(pMsg)) { - (*tsMgmtProcessShellMsgFp[rpcMsg->msgType])(pMsg); - mgmtFreeQueuedMsg(pMsg); - } else { - if (!pMsg->pUser->writeAuth) { - mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_NO_RIGHTS); - mgmtFreeQueuedMsg(pMsg); - } else { - mgmtAddToShellQueue(pMsg); - } - } -} - -char *mgmtGetShowTypeStr(int32_t showType) { - switch (showType) { - case TSDB_MGMT_TABLE_ACCT: return "show accounts"; - case TSDB_MGMT_TABLE_USER: return "show users"; - case TSDB_MGMT_TABLE_DB: return "show databases"; - case TSDB_MGMT_TABLE_TABLE: return "show tables"; - case TSDB_MGMT_TABLE_DNODE: return "show dnodes"; - case TSDB_MGMT_TABLE_MNODE: return "show mnodes"; - case TSDB_MGMT_TABLE_VGROUP: return "show vgroups"; - case TSDB_MGMT_TABLE_METRIC: return "show stables"; - case TSDB_MGMT_TABLE_MODULE: return "show modules"; - case TSDB_MGMT_TABLE_QUERIES: return "show queries"; - case TSDB_MGMT_TABLE_STREAMS: return "show streams"; - case TSDB_MGMT_TABLE_CONFIGS: return "show configs"; - case TSDB_MGMT_TABLE_CONNS: return "show connections"; - case TSDB_MGMT_TABLE_SCORES: return "show scores"; - case TSDB_MGMT_TABLE_GRANTS: return "show grants"; - case TSDB_MGMT_TABLE_VNODES: return "show vnodes"; - default: return "undefined"; - } -} - -static void mgmtProcessShowMsg(SQueuedMsg *pMsg) { - SCMShowMsg *pShowMsg = pMsg->pCont; - if (pShowMsg->type >= TSDB_MGMT_TABLE_MAX) { - mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_INVALID_MSG_TYPE); - return; - } - - if (!tsMgmtShowMetaFp[pShowMsg->type] || !tsMgmtShowRetrieveFp[pShowMsg->type]) { - mError("show type:%s is not support", mgmtGetShowTypeStr(pShowMsg->type)); - mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_OPS_NOT_SUPPORT); - return; - } - - int32_t size = sizeof(SCMShowRsp) + sizeof(SSchema) * TSDB_MAX_COLUMNS + TSDB_EXTRA_PAYLOAD_SIZE; - SCMShowRsp *pShowRsp = rpcMallocCont(size); - if (pShowRsp == NULL) { - mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_SERV_OUT_OF_MEMORY); - return; - } - - int32_t showObjSize = sizeof(SShowObj) + htons(pShowMsg->payloadLen); - SShowObj *pShow = (SShowObj *) calloc(1, showObjSize); - pShow->signature = pShow; - pShow->type = pShowMsg->type; - pShow->payloadLen = htons(pShowMsg->payloadLen); - strcpy(pShow->db, pShowMsg->db); - memcpy(pShow->payload, pShowMsg->payload, pShow->payloadLen); - - pShow = mgmtSaveQhandle(pShow, showObjSize); - pShowRsp->qhandle = htobe64((uint64_t) pShow); - - mTrace("show:%p, type:%s, start to get meta", pShow, mgmtGetShowTypeStr(pShowMsg->type)); - int32_t code = (*tsMgmtShowMetaFp[pShowMsg->type])(&pShowRsp->tableMeta, pShow, pMsg->thandle); - if (code == 0) { - SRpcMsg rpcRsp = { - .handle = pMsg->thandle, - .pCont = pShowRsp, - .contLen = sizeof(SCMShowRsp) + sizeof(SSchema) * pShow->numOfColumns, - .code = code - }; - rpcSendResponse(&rpcRsp); - } else { - mError("show:%p, type:%s, failed to get meta, reason:%s", pShow, mgmtGetShowTypeStr(pShowMsg->type), tstrerror(code)); - mgmtFreeQhandle(pShow, false); - SRpcMsg rpcRsp = { - .handle = pMsg->thandle, - .code = code - }; - rpcSendResponse(&rpcRsp); - } -} - -static void mgmtProcessRetrieveMsg(SQueuedMsg *pMsg) { - int32_t rowsToRead = 0; - int32_t size = 0; - int32_t rowsRead = 0; - SRetrieveTableMsg *pRetrieve = pMsg->pCont; - pRetrieve->qhandle = htobe64(pRetrieve->qhandle); - - /* - * in case of server restart, apps may hold qhandle created by server before - * restart, which is actually invalid, therefore, signature check is required. - */ - if (!mgmtCheckQhandle(pRetrieve->qhandle)) { - mError("retrieve:%p, qhandle:%p is invalid", pRetrieve, pRetrieve->qhandle); - mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_INVALID_QHANDLE); - return; - } - - SShowObj *pShow = (SShowObj *)pRetrieve->qhandle; - mTrace("show:%p, type:%s, retrieve data", pShow, mgmtGetShowTypeStr(pShow->type)); - - if ((pRetrieve->free & TSDB_QUERY_TYPE_FREE_RESOURCE) != TSDB_QUERY_TYPE_FREE_RESOURCE) { - rowsToRead = pShow->numOfRows - pShow->numOfReads; - } - - /* return no more than 100 meters in one round trip */ - if (rowsToRead > 100) rowsToRead = 100; - - /* - * the actual number of table may be larger than the value of pShow->numOfRows, if a query is - * issued during a continuous create table operation. Therefore, rowToRead may be less than 0. - */ - if (rowsToRead < 0) rowsToRead = 0; - size = pShow->rowSize * rowsToRead; - - size += 100; - SRetrieveTableRsp *pRsp = rpcMallocCont(size); - - // if free flag is set, client wants to clean the resources - if ((pRetrieve->free & TSDB_QUERY_TYPE_FREE_RESOURCE) != TSDB_QUERY_TYPE_FREE_RESOURCE) - rowsRead = (*tsMgmtShowRetrieveFp[pShow->type])(pShow, pRsp->data, rowsToRead, pMsg->thandle); - - if (rowsRead < 0) { // TSDB_CODE_ACTION_IN_PROGRESS; - rpcFreeCont(pRsp); - mgmtFreeQhandle(pShow, false); - return; - } - - pRsp->numOfRows = htonl(rowsRead); - pRsp->precision = htonl(TSDB_TIME_PRECISION_MILLI); // millisecond time precision - - SRpcMsg rpcRsp = { - .handle = pMsg->thandle, - .pCont = pRsp, - .contLen = size, - .code = 0, - .msgType = 0 - }; - rpcSendResponse(&rpcRsp); - - if (rowsToRead == 0) { - mgmtFreeQhandle(pShow, true); - } else { - mgmtFreeQhandle(pShow, false); - } -} - -static void mgmtProcessHeartBeatMsg(SQueuedMsg *pMsg) { - SCMHeartBeatRsp *pHBRsp = (SCMHeartBeatRsp *) rpcMallocCont(sizeof(SCMHeartBeatRsp)); - if (pHBRsp == NULL) { - mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_SERV_OUT_OF_MEMORY); - return; - } - - pHBRsp->onlineDnodes = htonl(mgmtGetOnlinDnodesNum()); - pHBRsp->totalDnodes = htonl(mgmtGetDnodesNum()); - mgmtGetMnodeIpSet(&pHBRsp->ipList); - - /* - * TODO - * Dispose kill stream or kill query message - */ - pHBRsp->queryId = 0; - pHBRsp->streamId = 0; - pHBRsp->killConnection = 0; - - SRpcMsg rpcRsp = { - .handle = pMsg->thandle, - .pCont = pHBRsp, - .contLen = sizeof(SCMHeartBeatRsp), - .code = 0, - .msgType = 0 - }; - rpcSendResponse(&rpcRsp); -} - -static void mgmtProcessConnectMsg(SQueuedMsg *pMsg) { - SRpcMsg rpcRsp = {.handle = pMsg->thandle, .pCont = NULL, .contLen = 0, .code = 0, .msgType = 0}; - SCMConnectMsg *pConnectMsg = pMsg->pCont; - - SRpcConnInfo connInfo; - if (rpcGetConnInfo(pMsg->thandle, &connInfo) != 0) { - mError("thandle:%p is already released while process connect msg", pMsg->thandle); - return; - } - - int32_t code; - if (grantCheck(TSDB_GRANT_TIME) != TSDB_CODE_SUCCESS) { - code = TSDB_CODE_GRANT_EXPIRED; - goto connect_over; - } - - code = taosCheckVersion(pConnectMsg->clientVersion, version, 3); - if (code != TSDB_CODE_SUCCESS) { - goto connect_over; - } - - SUserObj *pUser = pMsg->pUser; - SAcctObj *pAcct = pUser->pAcct; - - if (pConnectMsg->db[0]) { - char dbName[TSDB_TABLE_ID_LEN * 3] = {0}; - sprintf(dbName, "%x%s%s", pAcct->acctId, TS_PATH_DELIMITER, pConnectMsg->db); - SDbObj *pDb = mgmtGetDb(dbName); - if (pDb == NULL) { - code = TSDB_CODE_INVALID_DB; - goto connect_over; - } - mgmtDecDbRef(pDb); - } - - SCMConnectRsp *pConnectRsp = rpcMallocCont(sizeof(SCMConnectRsp)); - if (pConnectRsp == NULL) { - code = TSDB_CODE_SERV_OUT_OF_MEMORY; - goto connect_over; - } - - sprintf(pConnectRsp->acctId, "%x", pAcct->acctId); - strcpy(pConnectRsp->serverVersion, version); - pConnectRsp->writeAuth = pUser->writeAuth; - pConnectRsp->superAuth = pUser->superAuth; - - mgmtGetMnodeIpSet(&pConnectRsp->ipList); - -connect_over: - rpcRsp.code = code; - if (code != TSDB_CODE_SUCCESS) { - mLError("user:%s login from %s, result:%s", connInfo.user, taosIpStr(connInfo.clientIp), tstrerror(code)); - } else { - mLPrint("user:%s login from %s, result:%s", connInfo.user, taosIpStr(connInfo.clientIp), tstrerror(code)); - rpcRsp.pCont = pConnectRsp; - rpcRsp.contLen = sizeof(SCMConnectRsp); - } - rpcSendResponse(&rpcRsp); -} - -static void mgmtProcessUseMsg(SQueuedMsg *pMsg) { - SRpcMsg rpcRsp = {.handle = pMsg->thandle, .pCont = NULL, .contLen = 0, .code = 0, .msgType = 0}; - - SCMUseDbMsg *pUseDbMsg = pMsg->pCont; - - // todo check for priority of current user - int32_t code = TSDB_CODE_SUCCESS; - if (pMsg->pDb == NULL) pMsg->pDb = mgmtGetDb(pUseDbMsg->db); - if (pMsg->pDb == NULL) { - code = TSDB_CODE_INVALID_DB; - } - - rpcRsp.code = code; - rpcSendResponse(&rpcRsp); -} - -/** - * check if we need to add mgmtProcessTableMetaMsg into tranQueue, which will be executed one-by-one. - */ -static bool mgmtCheckTableMetaMsgReadOnly(SQueuedMsg *pMsg) { - SCMTableInfoMsg *pInfo = pMsg->pCont; - if (pMsg->pTable == NULL) pMsg->pTable = mgmtGetTable(pInfo->tableId); - if (pMsg->pTable != NULL) return true; - - // If table does not exists and autoCreate flag is set, we add the handler into task queue - int16_t autoCreate = htons(pInfo->createFlag); - if (autoCreate == 1) { - mTrace("table:%s auto created task added", pInfo->tableId); - return false; - } - - return true; -} - -static bool mgmtCheckMsgReadOnly(SQueuedMsg *pMsg) { - if (pMsg->msgType == TSDB_MSG_TYPE_CM_TABLE_META) { - return mgmtCheckTableMetaMsgReadOnly(pMsg); - } - - if (pMsg->msgType == TSDB_MSG_TYPE_CM_STABLE_VGROUP || pMsg->msgType == TSDB_MSG_TYPE_CM_RETRIEVE || - pMsg->msgType == TSDB_MSG_TYPE_CM_SHOW || pMsg->msgType == TSDB_MSG_TYPE_CM_TABLES_META || - pMsg->msgType == TSDB_MSG_TYPE_CM_CONNECT) { - return true; - } - - return false; -} - -static void mgmtProcessUnSupportMsg(SRpcMsg *rpcMsg) { - mError("%s is not processed in mnode shell", taosMsg[rpcMsg->msgType]); - SRpcMsg rpcRsp = { - .msgType = 0, - .pCont = 0, - .contLen = 0, - .code = TSDB_CODE_OPS_NOT_SUPPORT, - .handle = rpcMsg->handle - }; - rpcSendResponse(&rpcRsp); -} - -void mgmtSendSimpleResp(void *thandle, int32_t code) { - SRpcMsg rpcRsp = { - .msgType = 0, - .pCont = 0, - .contLen = 0, - .code = code, - .handle = thandle - }; - rpcSendResponse(&rpcRsp); -} - -bool mgmtCheckQhandle(uint64_t qhandle) { - void *pSaved = taosCacheAcquireByData(tsQhandleCache, (void *)qhandle); - if (pSaved == (void *)qhandle) { - mTrace("show:%p, is retrieved", qhandle); - return true; - } else { - mTrace("show:%p, is already released", qhandle); - return false; - } -} - -void* mgmtSaveQhandle(void *qhandle, int32_t size) { - if (tsQhandleCache != NULL) { - char key[24]; - sprintf(key, "show:%p", qhandle); - void *newQhandle = taosCachePut(tsQhandleCache, key, qhandle, size, 60); - free(qhandle); - - mTrace("show:%p, is saved", newQhandle); - return newQhandle; - } - - return NULL; -} - -static void mgmtFreeShowObj(void *data) { - SShowObj *pShow = data; - sdbFreeIter(pShow->pIter); - mTrace("show:%p, is destroyed", pShow); -} - -void mgmtFreeQhandle(void *qhandle, bool forceRemove) { - mTrace("show:%p, is released, force:%s", qhandle, forceRemove ? "true" : "false"); - taosCacheRelease(tsQhandleCache, &qhandle, forceRemove); -} - -void *mgmtMallocQueuedMsg(SRpcMsg *rpcMsg) { - SUserObj *pUser = mgmtGetUserFromConn(rpcMsg->handle); - if (pUser == NULL) { - return NULL; - } - - SQueuedMsg *pMsg = calloc(1, sizeof(SQueuedMsg)); - pMsg->thandle = rpcMsg->handle; - pMsg->msgType = rpcMsg->msgType; - pMsg->contLen = rpcMsg->contLen; - pMsg->pCont = rpcMsg->pCont; - pMsg->pUser = pUser; - - return pMsg; -} - -void mgmtFreeQueuedMsg(SQueuedMsg *pMsg) { - if (pMsg != NULL) { - rpcFreeCont(pMsg->pCont); - if (pMsg->pUser) mgmtDecUserRef(pMsg->pUser); - if (pMsg->pDb) mgmtDecDbRef(pMsg->pDb); - if (pMsg->pVgroup) mgmtDecVgroupRef(pMsg->pVgroup); - if (pMsg->pTable) mgmtDecTableRef(pMsg->pTable); - if (pMsg->pAcct) mgmtDecAcctRef(pMsg->pAcct); - if (pMsg->pDnode) mgmtDecDnodeRef(pMsg->pDnode); - free(pMsg); - } -} - -void* mgmtCloneQueuedMsg(SQueuedMsg *pSrcMsg) { - SQueuedMsg *pDestMsg = calloc(1, sizeof(SQueuedMsg)); - - pDestMsg->thandle = pSrcMsg->thandle; - pDestMsg->msgType = pSrcMsg->msgType; - pDestMsg->pCont = pSrcMsg->pCont; - pDestMsg->contLen = pSrcMsg->contLen; - pDestMsg->retry = pSrcMsg->retry; - pDestMsg->maxRetry= pSrcMsg->maxRetry; - pDestMsg->pUser = pSrcMsg->pUser; - - pSrcMsg->pCont = NULL; - pSrcMsg->pUser = NULL; - - return pDestMsg; -} diff --git a/src/mnode/src/mgmtAcct.c b/src/mnode/src/mnodeAcct.c similarity index 71% rename from src/mnode/src/mgmtAcct.c rename to src/mnode/src/mnodeAcct.c index a35591382c9a2d00dbedcdaa7f34f870ae74887c..3836bcea52cde122398d7f7ac6c7e585a7a5b8cb 100644 --- a/src/mnode/src/mgmtAcct.c +++ b/src/mnode/src/mnodeAcct.c @@ -17,59 +17,58 @@ #include "os.h" #include "taoserror.h" #include "ttime.h" -#include "tutil.h" #include "dnode.h" -#include "mgmtDef.h" -#include "mgmtInt.h" -#include "mgmtAcct.h" -#include "mgmtDb.h" -#include "mgmtSdb.h" -#include "mgmtUser.h" +#include "mnodeDef.h" +#include "mnodeInt.h" +#include "mnodeAcct.h" +#include "mnodeDb.h" +#include "mnodeSdb.h" +#include "mnodeUser.h" void * tsAcctSdb = NULL; static int32_t tsAcctUpdateSize; -static void mgmtCreateRootAcct(); +static void mnodeCreateRootAcct(); -static int32_t mgmtActionAcctDestroy(SSdbOper *pOper) { +static int32_t mnodeAcctActionDestroy(SSdbOper *pOper) { SAcctObj *pAcct = pOper->pObj; pthread_mutex_destroy(&pAcct->mutex); tfree(pOper->pObj); return TSDB_CODE_SUCCESS; } -static int32_t mgmtAcctActionInsert(SSdbOper *pOper) { +static int32_t mnodeAcctActionInsert(SSdbOper *pOper) { SAcctObj *pAcct = pOper->pObj; memset(&pAcct->acctInfo, 0, sizeof(SAcctInfo)); pthread_mutex_init(&pAcct->mutex, NULL); return TSDB_CODE_SUCCESS; } -static int32_t mgmtActionAcctDelete(SSdbOper *pOper) { +static int32_t mnodeAcctActionDelete(SSdbOper *pOper) { SAcctObj *pAcct = pOper->pObj; - mgmtDropAllUsers(pAcct); - mgmtDropAllDbs(pAcct); + mnodeDropAllUsers(pAcct); + mnodeDropAllDbs(pAcct); return TSDB_CODE_SUCCESS; } -static int32_t mgmtActionAcctUpdate(SSdbOper *pOper) { +static int32_t mnodeAcctActionUpdate(SSdbOper *pOper) { SAcctObj *pAcct = pOper->pObj; - SAcctObj *pSaved = mgmtGetAcct(pAcct->user); + SAcctObj *pSaved = mnodeGetAcct(pAcct->user); if (pAcct != pSaved) { memcpy(pSaved, pAcct, tsAcctUpdateSize); free(pAcct); } - mgmtDecAcctRef(pSaved); + mnodeDecAcctRef(pSaved); return TSDB_CODE_SUCCESS; } -static int32_t mgmtActionActionEncode(SSdbOper *pOper) { +static int32_t mnodeActionActionEncode(SSdbOper *pOper) { SAcctObj *pAcct = pOper->pObj; memcpy(pOper->rowData, pAcct, tsAcctUpdateSize); pOper->rowSize = tsAcctUpdateSize; return TSDB_CODE_SUCCESS; } -static int32_t mgmtActionAcctDecode(SSdbOper *pOper) { +static int32_t mnodeAcctActionDecode(SSdbOper *pOper) { SAcctObj *pAcct = (SAcctObj *) calloc(1, sizeof(SAcctObj)); if (pAcct == NULL) return TSDB_CODE_SERV_OUT_OF_MEMORY; @@ -78,16 +77,16 @@ static int32_t mgmtActionAcctDecode(SSdbOper *pOper) { return TSDB_CODE_SUCCESS; } -static int32_t mgmtActionAcctRestored() { +static int32_t mnodeAcctActionRestored() { if (dnodeIsFirstDeploy()) { - mgmtCreateRootAcct(); + mnodeCreateRootAcct(); } acctInit(); return TSDB_CODE_SUCCESS; } -int32_t mgmtInitAccts() { +int32_t mnodeInitAccts() { SAcctObj tObj; tsAcctUpdateSize = (int8_t *)tObj.updateEnd - (int8_t *)&tObj; @@ -98,13 +97,13 @@ int32_t mgmtInitAccts() { .maxRowSize = tsAcctUpdateSize, .refCountPos = (int8_t *)(&tObj.refCount) - (int8_t *)&tObj, .keyType = SDB_KEY_STRING, - .insertFp = mgmtAcctActionInsert, - .deleteFp = mgmtActionAcctDelete, - .updateFp = mgmtActionAcctUpdate, - .encodeFp = mgmtActionActionEncode, - .decodeFp = mgmtActionAcctDecode, - .destroyFp = mgmtActionAcctDestroy, - .restoredFp = mgmtActionAcctRestored + .insertFp = mnodeAcctActionInsert, + .deleteFp = mnodeAcctActionDelete, + .updateFp = mnodeAcctActionUpdate, + .encodeFp = mnodeActionActionEncode, + .decodeFp = mnodeAcctActionDecode, + .destroyFp = mnodeAcctActionDestroy, + .restoredFp = mnodeAcctActionRestored }; tsAcctSdb = sdbOpenTable(&tableDesc); @@ -117,52 +116,52 @@ int32_t mgmtInitAccts() { return TSDB_CODE_SUCCESS; } -void mgmtCleanUpAccts() { +void mnodeCleanupAccts() { sdbCloseTable(tsAcctSdb); acctCleanUp(); } -void *mgmtGetAcct(char *name) { +void *mnodeGetAcct(char *name) { return sdbGetRow(tsAcctSdb, name); } -void *mgmtGetNextAcct(void *pIter, SAcctObj **pAcct) { +void *mnodeGetNextAcct(void *pIter, SAcctObj **pAcct) { return sdbFetchRow(tsAcctSdb, pIter, (void **)pAcct); } -void mgmtIncAcctRef(SAcctObj *pAcct) { +void mnodeIncAcctRef(SAcctObj *pAcct) { sdbIncRef(tsAcctSdb, pAcct); } -void mgmtDecAcctRef(SAcctObj *pAcct) { +void mnodeDecAcctRef(SAcctObj *pAcct) { sdbDecRef(tsAcctSdb, pAcct); } -void mgmtAddDbToAcct(SAcctObj *pAcct, SDbObj *pDb) { +void mnodeAddDbToAcct(SAcctObj *pAcct, SDbObj *pDb) { atomic_add_fetch_32(&pAcct->acctInfo.numOfDbs, 1); pDb->pAcct = pAcct; - mgmtIncAcctRef(pAcct); + mnodeIncAcctRef(pAcct); } -void mgmtDropDbFromAcct(SAcctObj *pAcct, SDbObj *pDb) { +void mnodeDropDbFromAcct(SAcctObj *pAcct, SDbObj *pDb) { atomic_sub_fetch_32(&pAcct->acctInfo.numOfDbs, 1); pDb->pAcct = NULL; - mgmtDecAcctRef(pAcct); + mnodeDecAcctRef(pAcct); } -void mgmtAddUserToAcct(SAcctObj *pAcct, SUserObj *pUser) { +void mnodeAddUserToAcct(SAcctObj *pAcct, SUserObj *pUser) { atomic_add_fetch_32(&pAcct->acctInfo.numOfUsers, 1); pUser->pAcct = pAcct; - mgmtIncAcctRef(pAcct); + mnodeIncAcctRef(pAcct); } -void mgmtDropUserFromAcct(SAcctObj *pAcct, SUserObj *pUser) { +void mnodeDropUserFromAcct(SAcctObj *pAcct, SUserObj *pUser) { atomic_sub_fetch_32(&pAcct->acctInfo.numOfUsers, 1); pUser->pAcct = NULL; - mgmtDecAcctRef(pAcct); + mnodeDecAcctRef(pAcct); } -static void mgmtCreateRootAcct() { +static void mnodeCreateRootAcct() { int32_t numOfAccts = sdbGetNumOfRows(tsAcctSdb); if (numOfAccts != 0) return; diff --git a/src/mnode/src/mgmtBalance.c b/src/mnode/src/mnodeBalance.c similarity index 83% rename from src/mnode/src/mgmtBalance.c rename to src/mnode/src/mnodeBalance.c index f8410a207ef34db7d6ad8f12557d220aba630968..64d3c6d7c7153aacf12e638b755330841368c59d 100644 --- a/src/mnode/src/mgmtBalance.c +++ b/src/mnode/src/mnodeBalance.c @@ -15,22 +15,18 @@ #define _DEFAULT_SOURCE #include "os.h" -#include "trpc.h" -#include "tbalance.h" #include "tglobal.h" -#include "mgmtDef.h" -#include "mgmtInt.h" -#include "mgmtMnode.h" -#include "mgmtDnode.h" -#include "mgmtSdb.h" -#include "mgmtVgroup.h" +#include "mnodeDef.h" +#include "mnodeInt.h" +#include "mnodeDnode.h" +#include "mnodeSdb.h" #ifndef _SYNC int32_t balanceInit() { return TSDB_CODE_SUCCESS; } void balanceCleanUp() {} void balanceNotify() {} -void balanceUpdateMgmt() {} +void balanceUpdateMnode() {} void balanceReset() {} int32_t balanceAllocVnodes(SVgObj *pVgroup) { @@ -40,12 +36,12 @@ int32_t balanceAllocVnodes(SVgObj *pVgroup) { float vnodeUsage = 1000.0; while (1) { - pIter = mgmtGetNextDnode(pIter, &pDnode); + pIter = mnodeGetNextDnode(pIter, &pDnode); if (pDnode == NULL) break; if (pDnode->totalVnodes > 0 && pDnode->openVnodes < pDnode->totalVnodes) { float openVnodes = pDnode->openVnodes; - if (pDnode->isMgmt) openVnodes += tsMgmtEqualVnodeNum; + if (pDnode->isMgmt) openVnodes += tsMnodeEqualVnodeNum; float usage = openVnodes / pDnode->totalVnodes; if (usage <= vnodeUsage) { @@ -53,7 +49,7 @@ int32_t balanceAllocVnodes(SVgObj *pVgroup) { vnodeUsage = usage; } } - mgmtDecDnodeRef(pDnode); + mnodeDecDnodeRef(pDnode); } sdbFreeIter(pIter); diff --git a/src/mnode/src/mgmtDb.c b/src/mnode/src/mnodeDb.c similarity index 80% rename from src/mnode/src/mgmtDb.c rename to src/mnode/src/mnodeDb.c index df18fe8786f523cac624f5a4ffa0718ee5a906ee..0ad835279e44d76eeaf8aa7ee9f2cdc551bdda54 100644 --- a/src/mnode/src/mgmtDb.c +++ b/src/mnode/src/mnodeDb.c @@ -15,7 +15,6 @@ #define _DEFAULT_SOURCE #include "os.h" - #include "taoserror.h" #include "tutil.h" #include "tgrant.h" @@ -24,39 +23,41 @@ #include "tname.h" #include "tbalance.h" #include "tdataformat.h" -#include "mgmtDef.h" -#include "mgmtInt.h" -#include "mgmtAcct.h" -#include "mgmtDb.h" -#include "mgmtDnode.h" -#include "mgmtMnode.h" -#include "mgmtShell.h" -#include "mgmtProfile.h" -#include "mgmtSdb.h" -#include "mgmtTable.h" -#include "mgmtUser.h" -#include "mgmtVgroup.h" +#include "mnode.h" +#include "mnodeDef.h" +#include "mnodeInt.h" +#include "mnodeAcct.h" +#include "mnodeDb.h" +#include "mnodeDnode.h" +#include "mnodeMnode.h" +#include "mnodeProfile.h" +#include "mnodeWrite.h" +#include "mnodeSdb.h" +#include "mnodeShow.h" +#include "mnodeTable.h" +#include "mnodeUser.h" +#include "mnodeVgroup.h" static void * tsDbSdb = NULL; static int32_t tsDbUpdateSize; -static int32_t mgmtCreateDb(SAcctObj *pAcct, SCMCreateDbMsg *pCreate); -static void mgmtDropDb(SQueuedMsg *newMsg); -static int32_t mgmtSetDbDropping(SDbObj *pDb); -static int32_t mgmtGetDbMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn); -static int32_t mgmtRetrieveDbs(SShowObj *pShow, char *data, int32_t rows, void *pConn); -static void mgmtProcessCreateDbMsg(SQueuedMsg *pMsg); -static void mgmtProcessAlterDbMsg(SQueuedMsg *pMsg); -static void mgmtProcessDropDbMsg(SQueuedMsg *pMsg); +static int32_t mnodeCreateDb(SAcctObj *pAcct, SCMCreateDbMsg *pCreate); +static int32_t mnodeDropDb(SMnodeMsg *newMsg); +static int32_t mnodeSetDbDropping(SDbObj *pDb); +static int32_t mnodeGetDbMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn); +static int32_t mnodeRetrieveDbs(SShowObj *pShow, char *data, int32_t rows, void *pConn); +static int32_t mnodeProcessCreateDbMsg(SMnodeMsg *pMsg); +static int32_t mnodeProcessAlterDbMsg(SMnodeMsg *pMsg); +static int32_t mnodeProcessDropDbMsg(SMnodeMsg *pMsg); -static int32_t mgmtDbActionDestroy(SSdbOper *pOper) { +static int32_t mnodeDbActionDestroy(SSdbOper *pOper) { tfree(pOper->pObj); return TSDB_CODE_SUCCESS; } -static int32_t mgmtDbActionInsert(SSdbOper *pOper) { +static int32_t mnodeDbActionInsert(SSdbOper *pOper) { SDbObj *pDb = pOper->pObj; - SAcctObj *pAcct = mgmtGetAcct(pDb->acct); + SAcctObj *pAcct = mnodeGetAcct(pDb->acct); pDb->pHead = NULL; pDb->pTail = NULL; @@ -65,8 +66,8 @@ static int32_t mgmtDbActionInsert(SSdbOper *pOper) { pDb->numOfSuperTables = 0; if (pAcct != NULL) { - mgmtAddDbToAcct(pAcct, pDb); - mgmtDecAcctRef(pAcct); + mnodeAddDbToAcct(pAcct, pDb); + mnodeDecAcctRef(pAcct); } else { mError("db:%s, acct:%s info not exist in sdb", pDb->name, pDb->acct); @@ -76,39 +77,39 @@ static int32_t mgmtDbActionInsert(SSdbOper *pOper) { return TSDB_CODE_SUCCESS; } -static int32_t mgmtDbActionDelete(SSdbOper *pOper) { +static int32_t mnodeDbActionDelete(SSdbOper *pOper) { SDbObj *pDb = pOper->pObj; - SAcctObj *pAcct = mgmtGetAcct(pDb->acct); + SAcctObj *pAcct = mnodeGetAcct(pDb->acct); - mgmtDropDbFromAcct(pAcct, pDb); - mgmtDropAllChildTables(pDb); - mgmtDropAllSuperTables(pDb); - mgmtDropAllDbVgroups(pDb, false); - mgmtDecAcctRef(pAcct); + mnodeDropAllChildTables(pDb); + mnodeDropAllSuperTables(pDb); + mnodeDropAllDbVgroups(pDb); + mnodeDropDbFromAcct(pAcct, pDb); + mnodeDecAcctRef(pAcct); return TSDB_CODE_SUCCESS; } -static int32_t mgmtDbActionUpdate(SSdbOper *pOper) { +static int32_t mnodeDbActionUpdate(SSdbOper *pOper) { SDbObj *pDb = pOper->pObj; - SDbObj *pSaved = mgmtGetDb(pDb->name); + SDbObj *pSaved = mnodeGetDb(pDb->name); if (pDb != pSaved) { memcpy(pSaved, pDb, pOper->rowSize); free(pDb); } - mgmtUpdateAllDbVgroups(pSaved); - mgmtDecDbRef(pSaved); + mnodeUpdateAllDbVgroups(pSaved); + mnodeDecDbRef(pSaved); return TSDB_CODE_SUCCESS; } -static int32_t mgmtDbActionEncode(SSdbOper *pOper) { +static int32_t mnodeDbActionEncode(SSdbOper *pOper) { SDbObj *pDb = pOper->pObj; memcpy(pOper->rowData, pDb, tsDbUpdateSize); pOper->rowSize = tsDbUpdateSize; return TSDB_CODE_SUCCESS; } -static int32_t mgmtDbActionDecode(SSdbOper *pOper) { +static int32_t mnodeDbActionDecode(SSdbOper *pOper) { SDbObj *pDb = (SDbObj *) calloc(1, sizeof(SDbObj)); if (pDb == NULL) return TSDB_CODE_SERV_OUT_OF_MEMORY; @@ -117,11 +118,11 @@ static int32_t mgmtDbActionDecode(SSdbOper *pOper) { return TSDB_CODE_SUCCESS; } -static int32_t mgmtDbActionRestored() { +static int32_t mnodeDbActionRestored() { return 0; } -int32_t mgmtInitDbs() { +int32_t mnodeInitDbs() { SDbObj tObj; tsDbUpdateSize = (int8_t *)tObj.updateEnd - (int8_t *)&tObj; @@ -132,13 +133,13 @@ int32_t mgmtInitDbs() { .maxRowSize = tsDbUpdateSize, .refCountPos = (int8_t *)(&tObj.refCount) - (int8_t *)&tObj, .keyType = SDB_KEY_STRING, - .insertFp = mgmtDbActionInsert, - .deleteFp = mgmtDbActionDelete, - .updateFp = mgmtDbActionUpdate, - .encodeFp = mgmtDbActionEncode, - .decodeFp = mgmtDbActionDecode, - .destroyFp = mgmtDbActionDestroy, - .restoredFp = mgmtDbActionRestored + .insertFp = mnodeDbActionInsert, + .deleteFp = mnodeDbActionDelete, + .updateFp = mnodeDbActionUpdate, + .encodeFp = mnodeDbActionEncode, + .decodeFp = mnodeDbActionDecode, + .destroyFp = mnodeDbActionDestroy, + .restoredFp = mnodeDbActionRestored }; tsDbSdb = sdbOpenTable(&tableDesc); @@ -147,33 +148,33 @@ int32_t mgmtInitDbs() { return -1; } - mgmtAddShellMsgHandle(TSDB_MSG_TYPE_CM_CREATE_DB, mgmtProcessCreateDbMsg); - mgmtAddShellMsgHandle(TSDB_MSG_TYPE_CM_ALTER_DB, mgmtProcessAlterDbMsg); - mgmtAddShellMsgHandle(TSDB_MSG_TYPE_CM_DROP_DB, mgmtProcessDropDbMsg); - mgmtAddShellShowMetaHandle(TSDB_MGMT_TABLE_DB, mgmtGetDbMeta); - mgmtAddShellShowRetrieveHandle(TSDB_MGMT_TABLE_DB, mgmtRetrieveDbs); + mnodeAddWriteMsgHandle(TSDB_MSG_TYPE_CM_CREATE_DB, mnodeProcessCreateDbMsg); + mnodeAddWriteMsgHandle(TSDB_MSG_TYPE_CM_ALTER_DB, mnodeProcessAlterDbMsg); + mnodeAddWriteMsgHandle(TSDB_MSG_TYPE_CM_DROP_DB, mnodeProcessDropDbMsg); + mnodeAddShowMetaHandle(TSDB_MGMT_TABLE_DB, mnodeGetDbMeta); + mnodeAddShowRetrieveHandle(TSDB_MGMT_TABLE_DB, mnodeRetrieveDbs); mTrace("table:dbs table is created"); return 0; } -void *mgmtGetNextDb(void *pIter, SDbObj **pDb) { +void *mnodeGetNextDb(void *pIter, SDbObj **pDb) { return sdbFetchRow(tsDbSdb, pIter, (void **)pDb); } -SDbObj *mgmtGetDb(char *db) { +SDbObj *mnodeGetDb(char *db) { return (SDbObj *)sdbGetRow(tsDbSdb, db); } -void mgmtIncDbRef(SDbObj *pDb) { +void mnodeIncDbRef(SDbObj *pDb) { return sdbIncRef(tsDbSdb, pDb); } -void mgmtDecDbRef(SDbObj *pDb) { +void mnodeDecDbRef(SDbObj *pDb) { return sdbDecRef(tsDbSdb, pDb); } -SDbObj *mgmtGetDbByTableId(char *tableId) { +SDbObj *mnodeGetDbByTableId(char *tableId) { char db[TSDB_TABLE_ID_LEN], *pos; pos = strstr(tableId, TS_PATH_DELIMITER); @@ -181,10 +182,10 @@ SDbObj *mgmtGetDbByTableId(char *tableId) { memset(db, 0, sizeof(db)); strncpy(db, tableId, pos - tableId); - return mgmtGetDb(db); + return mnodeGetDb(db); } -static int32_t mgmtCheckDbCfg(SDbCfg *pCfg) { +static int32_t mnodeCheckDbCfg(SDbCfg *pCfg) { if (pCfg->cacheBlockSize < TSDB_MIN_CACHE_BLOCK_SIZE || pCfg->cacheBlockSize > TSDB_MAX_CACHE_BLOCK_SIZE) { mError("invalid db option cacheBlockSize:%d valid range: [%d, %d]", pCfg->cacheBlockSize, TSDB_MIN_CACHE_BLOCK_SIZE, TSDB_MAX_CACHE_BLOCK_SIZE); @@ -290,7 +291,7 @@ static int32_t mgmtCheckDbCfg(SDbCfg *pCfg) { return TSDB_CODE_SUCCESS; } -static void mgmtSetDefaultDbCfg(SDbCfg *pCfg) { +static void mnodeSetDefaultDbCfg(SDbCfg *pCfg) { if (pCfg->cacheBlockSize < 0) pCfg->cacheBlockSize = tsCacheBlockSize; if (pCfg->totalBlocks < 0) pCfg->totalBlocks = tsBlocksPerVnode; if (pCfg->maxTables < 0) pCfg->maxTables = tsMaxTablePerVnode; @@ -307,13 +308,13 @@ static void mgmtSetDefaultDbCfg(SDbCfg *pCfg) { if (pCfg->replications < 0) pCfg->replications = tsReplications; } -static int32_t mgmtCreateDb(SAcctObj *pAcct, SCMCreateDbMsg *pCreate) { +static int32_t mnodeCreateDb(SAcctObj *pAcct, SCMCreateDbMsg *pCreate) { int32_t code = acctCheck(pAcct, ACCT_GRANT_DB); if (code != 0) return code; - SDbObj *pDb = mgmtGetDb(pCreate->db); + SDbObj *pDb = mnodeGetDb(pCreate->db); if (pDb != NULL) { - mgmtDecDbRef(pDb); + mnodeDecDbRef(pDb); if (pCreate->ignoreExist) { mTrace("db:%s, already exist, ignore exist is set", pCreate->db); return TSDB_CODE_SUCCESS; @@ -347,9 +348,9 @@ static int32_t mgmtCreateDb(SAcctObj *pAcct, SCMCreateDbMsg *pCreate) { .replications = pCreate->replications }; - mgmtSetDefaultDbCfg(&pDb->cfg); + mnodeSetDefaultDbCfg(&pDb->cfg); - code = mgmtCheckDbCfg(&pDb->cfg); + code = mnodeCheckDbCfg(&pDb->cfg); if (code != TSDB_CODE_SUCCESS) { tfree(pDb); return code; @@ -371,7 +372,7 @@ static int32_t mgmtCreateDb(SAcctObj *pAcct, SCMCreateDbMsg *pCreate) { return code; } -bool mgmtCheckIsMonitorDB(char *db, char *monitordb) { +bool mnodeCheckIsMonitorDB(char *db, char *monitordb) { char dbName[TSDB_DB_NAME_LEN + 1] = {0}; extractDBName(db, dbName); @@ -380,7 +381,7 @@ bool mgmtCheckIsMonitorDB(char *db, char *monitordb) { } #if 0 -void mgmtPrintVgroups(SDbObj *pDb, char *oper) { +void mnodePrintVgroups(SDbObj *pDb, char *oper) { mPrint("db:%s, vgroup link from head, oper:%s", pDb->name, oper); SVgObj *pVgroup = pDb->pHead; while (pVgroup != NULL) { @@ -397,7 +398,7 @@ void mgmtPrintVgroups(SDbObj *pDb, char *oper) { } #endif -void mgmtAddVgroupIntoDb(SVgObj *pVgroup) { +void mnodeAddVgroupIntoDb(SVgObj *pVgroup) { SDbObj *pDb = pVgroup->pDb; pVgroup->next = pDb->pHead; @@ -410,7 +411,7 @@ void mgmtAddVgroupIntoDb(SVgObj *pVgroup) { pDb->numOfVgroups++; } -void mgmtAddVgroupIntoDbTail(SVgObj *pVgroup) { +void mnodeAddVgroupIntoDbTail(SVgObj *pVgroup) { SDbObj *pDb = pVgroup->pDb; pVgroup->next = NULL; pVgroup->prev = pDb->pTail; @@ -422,7 +423,7 @@ void mgmtAddVgroupIntoDbTail(SVgObj *pVgroup) { pDb->numOfVgroups++; } -void mgmtRemoveVgroupFromDb(SVgObj *pVgroup) { +void mnodeRemoveVgroupFromDb(SVgObj *pVgroup) { SDbObj *pDb = pVgroup->pDb; if (pVgroup->prev) pVgroup->prev->next = pVgroup->next; if (pVgroup->next) pVgroup->next->prev = pVgroup->prev; @@ -431,25 +432,25 @@ void mgmtRemoveVgroupFromDb(SVgObj *pVgroup) { pDb->numOfVgroups--; } -void mgmtMoveVgroupToTail(SVgObj *pVgroup) { - mgmtRemoveVgroupFromDb(pVgroup); - mgmtAddVgroupIntoDbTail(pVgroup); +void mnodeMoveVgroupToTail(SVgObj *pVgroup) { + mnodeRemoveVgroupFromDb(pVgroup); + mnodeAddVgroupIntoDbTail(pVgroup); } -void mgmtMoveVgroupToHead(SVgObj *pVgroup) { - mgmtRemoveVgroupFromDb(pVgroup); - mgmtAddVgroupIntoDb(pVgroup); +void mnodeMoveVgroupToHead(SVgObj *pVgroup) { + mnodeRemoveVgroupFromDb(pVgroup); + mnodeAddVgroupIntoDb(pVgroup); } -void mgmtCleanUpDbs() { +void mnodeCleanupDbs() { sdbCloseTable(tsDbSdb); } -static int32_t mgmtGetDbMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) { +static int32_t mnodeGetDbMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) { int32_t cols = 0; SSchema *pSchema = pMeta->schema; - SUserObj *pUser = mgmtGetUserFromConn(pConn); + SUserObj *pUser = mnodeGetUserFromConn(pConn); if (pUser == NULL) return 0; pShow->bytes[cols] = TSDB_DB_NAME_LEN + VARSTR_HEADER_SIZE; @@ -583,32 +584,32 @@ static int32_t mgmtGetDbMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) pShow->rowSize = pShow->offset[cols - 1] + pShow->bytes[cols - 1]; pShow->numOfRows = pUser->pAcct->acctInfo.numOfDbs; - mgmtDecUserRef(pUser); + mnodeDecUserRef(pUser); return 0; } -static char *mgmtGetDbStr(char *src) { +static char *mnodeGetDbStr(char *src) { char *pos = strstr(src, TS_PATH_DELIMITER); return ++pos; } -static int32_t mgmtRetrieveDbs(SShowObj *pShow, char *data, int32_t rows, void *pConn) { +static int32_t mnodeRetrieveDbs(SShowObj *pShow, char *data, int32_t rows, void *pConn) { int32_t numOfRows = 0; SDbObj *pDb = NULL; char * pWrite; int32_t cols = 0; - SUserObj *pUser = mgmtGetUserFromConn(pConn); + SUserObj *pUser = mnodeGetUserFromConn(pConn); if (pUser == NULL) return 0; while (numOfRows < rows) { - pShow->pIter = mgmtGetNextDb(pShow->pIter, &pDb); + pShow->pIter = mnodeGetNextDb(pShow->pIter, &pDb); if (pDb == NULL) break; cols = 0; pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows; - char* name = mgmtGetDbStr(pDb->name); + char* name = mnodeGetDbStr(pDb->name); STR_WITH_MAXSIZE_TO_VARSTR(pWrite, name, TSDB_DB_NAME_LEN); cols++; @@ -706,31 +707,31 @@ static int32_t mgmtRetrieveDbs(SShowObj *pShow, char *data, int32_t rows, void * cols++; numOfRows++; - mgmtDecDbRef(pDb); + mnodeDecDbRef(pDb); } pShow->numOfReads += numOfRows; - mgmtDecUserRef(pUser); + mnodeDecUserRef(pUser); return numOfRows; } -void mgmtAddSuperTableIntoDb(SDbObj *pDb) { +void mnodeAddSuperTableIntoDb(SDbObj *pDb) { atomic_add_fetch_32(&pDb->numOfSuperTables, 1); } -void mgmtRemoveSuperTableFromDb(SDbObj *pDb) { +void mnodeRemoveSuperTableFromDb(SDbObj *pDb) { atomic_add_fetch_32(&pDb->numOfSuperTables, -1); } -void mgmtAddTableIntoDb(SDbObj *pDb) { +void mnodeAddTableIntoDb(SDbObj *pDb) { atomic_add_fetch_32(&pDb->numOfTables, 1); } -void mgmtRemoveTableFromDb(SDbObj *pDb) { +void mnodeRemoveTableFromDb(SDbObj *pDb) { atomic_add_fetch_32(&pDb->numOfTables, -1); } -static int32_t mgmtSetDbDropping(SDbObj *pDb) { +static int32_t mnodeSetDbDropping(SDbObj *pDb) { if (pDb->status) return TSDB_CODE_SUCCESS; pDb->status = true; @@ -748,9 +749,8 @@ static int32_t mgmtSetDbDropping(SDbObj *pDb) { return code; } -static void mgmtProcessCreateDbMsg(SQueuedMsg *pMsg) { - SCMCreateDbMsg *pCreate = pMsg->pCont; - +static int32_t mnodeProcessCreateDbMsg(SMnodeMsg *pMsg) { + SCMCreateDbMsg *pCreate = pMsg->rpcMsg.pCont; pCreate->maxTables = htonl(pCreate->maxTables); pCreate->cacheBlockSize = htonl(pCreate->cacheBlockSize); pCreate->totalBlocks = htonl(pCreate->totalBlocks); @@ -768,7 +768,7 @@ static void mgmtProcessCreateDbMsg(SQueuedMsg *pMsg) { } else if (!pMsg->pUser->writeAuth) { code = TSDB_CODE_NO_RIGHTS; } else { - code = mgmtCreateDb(pMsg->pUser->pAcct, pCreate); + code = mnodeCreateDb(pMsg->pUser->pAcct, pCreate); if (code == TSDB_CODE_SUCCESS) { mLPrint("db:%s, is created by %s", pCreate->db, pMsg->pUser->user); } else { @@ -776,10 +776,10 @@ static void mgmtProcessCreateDbMsg(SQueuedMsg *pMsg) { } } - mgmtSendSimpleResp(pMsg->thandle, code); + return code; } -static SDbCfg mgmtGetAlterDbOption(SDbObj *pDb, SCMAlterDbMsg *pAlter) { +static SDbCfg mnodeGetAlterDbOption(SDbObj *pDb, SCMAlterDbMsg *pAlter) { SDbCfg newCfg = pDb->cfg; int32_t maxTables = htonl(pAlter->maxTables); int32_t cacheBlockSize = htonl(pAlter->cacheBlockSize); @@ -876,7 +876,7 @@ static SDbCfg mgmtGetAlterDbOption(SDbObj *pDb, SCMAlterDbMsg *pAlter) { terrno = TSDB_CODE_INVALID_OPTION; } - if (replications > mgmtGetDnodesNum()) { + if (replications > mnodeGetDnodesNum()) { mError("db:%s, no enough dnode to change replica:%d", pDb->name, replications); terrno = TSDB_CODE_NO_ENOUGH_DNODES; } @@ -890,13 +890,13 @@ static SDbCfg mgmtGetAlterDbOption(SDbObj *pDb, SCMAlterDbMsg *pAlter) { return newCfg; } -static int32_t mgmtAlterDb(SDbObj *pDb, SCMAlterDbMsg *pAlter) { - SDbCfg newCfg = mgmtGetAlterDbOption(pDb, pAlter); +static int32_t mnodeAlterDb(SDbObj *pDb, SCMAlterDbMsg *pAlter) { + SDbCfg newCfg = mnodeGetAlterDbOption(pDb, pAlter); if (terrno != TSDB_CODE_SUCCESS) { return terrno; } - int32_t code = mgmtCheckDbCfg(&newCfg); + int32_t code = mnodeCheckDbCfg(&newCfg); if (code != TSDB_CODE_SUCCESS) { return code; } @@ -921,10 +921,10 @@ static int32_t mgmtAlterDb(SDbObj *pDb, SCMAlterDbMsg *pAlter) { void *pIter = NULL; while (1) { SVgObj *pVgroup = NULL; - pIter = mgmtGetNextVgroup(pIter, &pVgroup); + pIter = mnodeGetNextVgroup(pIter, &pVgroup); if (pVgroup == NULL) break; - mgmtSendCreateVgroupMsg(pVgroup, NULL); - mgmtDecVgroupRef(pVgroup); + mnodeSendCreateVgroupMsg(pVgroup, NULL); + mnodeDecVgroupRef(pVgroup); } sdbFreeIter(pIter); @@ -935,35 +935,27 @@ static int32_t mgmtAlterDb(SDbObj *pDb, SCMAlterDbMsg *pAlter) { return TSDB_CODE_SUCCESS; } -static void mgmtProcessAlterDbMsg(SQueuedMsg *pMsg) { - SCMAlterDbMsg *pAlter = pMsg->pCont; - mTrace("db:%s, alter db msg is received from thandle:%p", pAlter->db, pMsg->thandle); - - if (grantCheck(TSDB_GRANT_TIME) != TSDB_CODE_SUCCESS) { - mError("db:%s, failed to alter, grant expired", pAlter->db); - mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_GRANT_EXPIRED); - return; - } +static int32_t mnodeProcessAlterDbMsg(SMnodeMsg *pMsg) { + SCMAlterDbMsg *pAlter = pMsg->rpcMsg.pCont; + mTrace("db:%s, alter db msg is received from thandle:%p", pAlter->db, pMsg->rpcMsg.handle); - if (pMsg->pDb == NULL) pMsg->pDb = mgmtGetDb(pAlter->db); + if (pMsg->pDb == NULL) pMsg->pDb = mnodeGetDb(pAlter->db); if (pMsg->pDb == NULL) { mError("db:%s, failed to alter, invalid db", pAlter->db); - mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_INVALID_DB); - return; + return TSDB_CODE_INVALID_DB; } - int32_t code = mgmtAlterDb(pMsg->pDb, pAlter); + int32_t code = mnodeAlterDb(pMsg->pDb, pAlter); if (code != TSDB_CODE_SUCCESS) { mError("db:%s, failed to alter, invalid db option", pAlter->db); - mgmtSendSimpleResp(pMsg->thandle, code); - return; + return code; } mTrace("db:%s, all vgroups is altered", pMsg->pDb->name); - mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_SUCCESS); + return TSDB_CODE_SUCCESS; } -static void mgmtDropDb(SQueuedMsg *pMsg) { +static int32_t mnodeDropDb(SMnodeMsg *pMsg) { SDbObj *pDb = pMsg->pDb; mPrint("db:%s, drop db from sdb", pDb->name); @@ -977,64 +969,42 @@ static void mgmtDropDb(SQueuedMsg *pMsg) { code = TSDB_CODE_SDB_ERROR; } - mgmtSendSimpleResp(pMsg->thandle, code); + return code; } -static void mgmtProcessDropDbMsg(SQueuedMsg *pMsg) { - SCMDropDbMsg *pDrop = pMsg->pCont; - mTrace("db:%s, drop db msg is received from thandle:%p", pDrop->db, pMsg->thandle); - - if (grantCheck(TSDB_GRANT_TIME) != TSDB_CODE_SUCCESS) { - mError("db:%s, failed to drop, grant expired", pDrop->db); - mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_GRANT_EXPIRED); - return; - } +static int32_t mnodeProcessDropDbMsg(SMnodeMsg *pMsg) { + SCMDropDbMsg *pDrop = pMsg->rpcMsg.pCont; + mTrace("db:%s, drop db msg is received from thandle:%p", pDrop->db, pMsg->rpcMsg.handle); - if (pMsg->pDb == NULL) pMsg->pDb = mgmtGetDb(pDrop->db); + if (pMsg->pDb == NULL) pMsg->pDb = mnodeGetDb(pDrop->db); if (pMsg->pDb == NULL) { if (pDrop->ignoreNotExists) { mTrace("db:%s, db is not exist, think drop success", pDrop->db); - mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_SUCCESS); - return; + return TSDB_CODE_SUCCESS; } else { mError("db:%s, failed to drop, invalid db", pDrop->db); - mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_INVALID_DB); - return; + return TSDB_CODE_INVALID_DB; } } - if (mgmtCheckIsMonitorDB(pMsg->pDb->name, tsMonitorDbName)) { + if (mnodeCheckIsMonitorDB(pMsg->pDb->name, tsMonitorDbName)) { mError("db:%s, can't drop monitor database", pDrop->db); - mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_MONITOR_DB_FORBIDDEN); - return; + return TSDB_CODE_MONITOR_DB_FORBIDDEN; } - int32_t code = mgmtSetDbDropping(pMsg->pDb); + int32_t code = mnodeSetDbDropping(pMsg->pDb); if (code != TSDB_CODE_SUCCESS) { mError("db:%s, failed to drop, reason:%s", pDrop->db, tstrerror(code)); - mgmtSendSimpleResp(pMsg->thandle, code); - return; + return code; } -#if 1 - mgmtDropAllDbVgroups(pMsg->pDb, true); -#else - SVgObj *pVgroup = pMsg->pDb->pHead; - if (pVgroup != NULL) { - mPrint("vgId:%d, will be dropped", pVgroup->vgId); - SQueuedMsg *newMsg = mgmtCloneQueuedMsg(pMsg); - newMsg->ahandle = pVgroup; - newMsg->expected = pVgroup->numOfVnodes; - mgmtDropVgroup(pVgroup, newMsg); - return; - } -#endif + mnodeSendDropAllDbVgroupsMsg(pMsg->pDb); mTrace("db:%s, all vgroups is dropped", pMsg->pDb->name); - mgmtDropDb(pMsg); + return mnodeDropDb(pMsg); } -void mgmtDropAllDbs(SAcctObj *pAcct) { +void mnodeDropAllDbs(SAcctObj *pAcct) { int32_t numOfDbs = 0; SDbObj *pDb = NULL; void * pIter = NULL; @@ -1042,7 +1012,7 @@ void mgmtDropAllDbs(SAcctObj *pAcct) { mPrint("acct:%s, all dbs will be dropped from sdb", pAcct->user); while (1) { - pIter = mgmtGetNextDb(pIter, &pDb); + pIter = mnodeGetNextDb(pIter, &pDb); if (pDb == NULL) break; if (pDb->pAcct == pAcct) { @@ -1056,7 +1026,7 @@ void mgmtDropAllDbs(SAcctObj *pAcct) { sdbDeleteRow(&oper); numOfDbs++; } - mgmtDecDbRef(pDb); + mnodeDecDbRef(pDb); } sdbFreeIter(pIter); diff --git a/src/mnode/src/mgmtDnode.c b/src/mnode/src/mnodeDnode.c similarity index 66% rename from src/mnode/src/mgmtDnode.c rename to src/mnode/src/mnodeDnode.c index 93c8276b14f3571f7743667f4536c5b8a375437b..506511ece9ed38785521571cde28b9a7d94f3913 100644 --- a/src/mnode/src/mgmtDnode.c +++ b/src/mnode/src/mnodeDnode.c @@ -25,15 +25,18 @@ #include "tbalance.h" #include "tsync.h" #include "tdataformat.h" +#include "mnode.h" #include "dnode.h" -#include "mgmtDef.h" -#include "mgmtInt.h" -#include "mgmtDnode.h" -#include "mgmtMnode.h" -#include "mgmtSdb.h" -#include "mgmtShell.h" -#include "mgmtUser.h" -#include "mgmtVgroup.h" +#include "mnodeDef.h" +#include "mnodeInt.h" +#include "mnodeDnode.h" +#include "mnodeMnode.h" +#include "mnodeSdb.h" +#include "mnodeShow.h" +#include "mnodeUser.h" +#include "mnodeVgroup.h" +#include "mnodeWrite.h" +#include "mnodePeer.h" int32_t tsAccessSquence = 0; static void *tsDnodeSdb = NULL; @@ -41,27 +44,28 @@ static int32_t tsDnodeUpdateSize = 0; extern void * tsMnodeSdb; extern void * tsVgroupSdb; -static int32_t mgmtCreateDnode(char *ep); -static void mgmtProcessCreateDnodeMsg(SQueuedMsg *pMsg); -static void mgmtProcessDropDnodeMsg(SQueuedMsg *pMsg); -static void mgmtProcessCfgDnodeMsg(SQueuedMsg *pMsg); -static void mgmtProcessCfgDnodeMsgRsp(SRpcMsg *rpcMsg) ; -static void mgmtProcessDnodeStatusMsg(SRpcMsg *rpcMsg); -static int32_t mgmtGetModuleMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn); -static int32_t mgmtRetrieveModules(SShowObj *pShow, char *data, int32_t rows, void *pConn); -static int32_t mgmtGetConfigMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn); -static int32_t mgmtRetrieveConfigs(SShowObj *pShow, char *data, int32_t rows, void *pConn); -static int32_t mgmtGetVnodeMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn); -static int32_t mgmtRetrieveVnodes(SShowObj *pShow, char *data, int32_t rows, void *pConn); -static int32_t mgmtGetDnodeMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn); -static int32_t mgmtRetrieveDnodes(SShowObj *pShow, char *data, int32_t rows, void *pConn); - -static int32_t mgmtDnodeActionDestroy(SSdbOper *pOper) { +static int32_t mnodeCreateDnode(char *ep); +static int32_t mnodeProcessCreateDnodeMsg(SMnodeMsg *pMsg); +static int32_t mnodeProcessDropDnodeMsg(SMnodeMsg *pMsg); +static int32_t mnodeProcessCfgDnodeMsg(SMnodeMsg *pMsg); +static void mnodeProcessCfgDnodeMsgRsp(SRpcMsg *rpcMsg) ; +static int32_t mnodeProcessDnodeStatusMsg(SMnodeMsg *rpcMsg); +static int32_t mnodeGetModuleMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn); +static int32_t mnodeRetrieveModules(SShowObj *pShow, char *data, int32_t rows, void *pConn); +static int32_t mnodeGetConfigMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn); +static int32_t mnodeRetrieveConfigs(SShowObj *pShow, char *data, int32_t rows, void *pConn); +static int32_t mnodeGetVnodeMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn); +static int32_t mnodeRetrieveVnodes(SShowObj *pShow, char *data, int32_t rows, void *pConn); +static int32_t mnodeGetDnodeMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn); +static int32_t mnodeRetrieveDnodes(SShowObj *pShow, char *data, int32_t rows, void *pConn); +static char* mnodeGetDnodeAlternativeRoleStr(int32_t alternativeRole); + +static int32_t mnodeDnodeActionDestroy(SSdbOper *pOper) { tfree(pOper->pObj); return TSDB_CODE_SUCCESS; } -static int32_t mgmtDnodeActionInsert(SSdbOper *pOper) { +static int32_t mnodeDnodeActionInsert(SSdbOper *pOper) { SDnodeObj *pDnode = pOper->pObj; if (pDnode->status != TAOS_DN_STATUS_DROPPING) { pDnode->status = TAOS_DN_STATUS_OFFLINE; @@ -70,38 +74,38 @@ static int32_t mgmtDnodeActionInsert(SSdbOper *pOper) { return TSDB_CODE_SUCCESS; } -static int32_t mgmtDnodeActionDelete(SSdbOper *pOper) { +static int32_t mnodeDnodeActionDelete(SSdbOper *pOper) { SDnodeObj *pDnode = pOper->pObj; #ifndef _SYNC - mgmtDropAllDnodeVgroups(pDnode); + mnodeDropAllDnodeVgroups(pDnode); #endif - mgmtDropMnodeLocal(pDnode->dnodeId); + mnodeDropMnodeLocal(pDnode->dnodeId); balanceNotify(); mTrace("dnode:%d, all vgroups is dropped from sdb", pDnode->dnodeId); return TSDB_CODE_SUCCESS; } -static int32_t mgmtDnodeActionUpdate(SSdbOper *pOper) { +static int32_t mnodeDnodeActionUpdate(SSdbOper *pOper) { SDnodeObj *pDnode = pOper->pObj; - SDnodeObj *pSaved = mgmtGetDnode(pDnode->dnodeId); + SDnodeObj *pSaved = mnodeGetDnode(pDnode->dnodeId); if (pDnode != pSaved) { memcpy(pSaved, pDnode, pOper->rowSize); free(pDnode); } - mgmtDecDnodeRef(pSaved); + mnodeDecDnodeRef(pSaved); return TSDB_CODE_SUCCESS; } -static int32_t mgmtDnodeActionEncode(SSdbOper *pOper) { +static int32_t mnodeDnodeActionEncode(SSdbOper *pOper) { SDnodeObj *pDnode = pOper->pObj; memcpy(pOper->rowData, pDnode, tsDnodeUpdateSize); pOper->rowSize = tsDnodeUpdateSize; return TSDB_CODE_SUCCESS; } -static int32_t mgmtDnodeActionDecode(SSdbOper *pOper) { +static int32_t mnodeDnodeActionDecode(SSdbOper *pOper) { SDnodeObj *pDnode = (SDnodeObj *) calloc(1, sizeof(SDnodeObj)); if (pDnode == NULL) return TSDB_CODE_SERV_OUT_OF_MEMORY; @@ -110,19 +114,19 @@ static int32_t mgmtDnodeActionDecode(SSdbOper *pOper) { return TSDB_CODE_SUCCESS; } -static int32_t mgmtDnodeActionRestored() { +static int32_t mnodeDnodeActionRestored() { int32_t numOfRows = sdbGetNumOfRows(tsDnodeSdb); if (numOfRows <= 0 && dnodeIsFirstDeploy()) { - mgmtCreateDnode(tsLocalEp); - SDnodeObj *pDnode = mgmtGetDnodeByEp(tsLocalEp); - mgmtAddMnode(pDnode->dnodeId); - mgmtDecDnodeRef(pDnode); + mnodeCreateDnode(tsLocalEp); + SDnodeObj *pDnode = mnodeGetDnodeByEp(tsLocalEp); + mnodeAddMnode(pDnode->dnodeId); + mnodeDecDnodeRef(pDnode); } return TSDB_CODE_SUCCESS; } -int32_t mgmtInitDnodes() { +int32_t mnodeInitDnodes() { SDnodeObj tObj; tsDnodeUpdateSize = (int8_t *)tObj.updateEnd - (int8_t *)&tObj; @@ -133,13 +137,13 @@ int32_t mgmtInitDnodes() { .maxRowSize = tsDnodeUpdateSize, .refCountPos = (int8_t *)(&tObj.refCount) - (int8_t *)&tObj, .keyType = SDB_KEY_AUTO, - .insertFp = mgmtDnodeActionInsert, - .deleteFp = mgmtDnodeActionDelete, - .updateFp = mgmtDnodeActionUpdate, - .encodeFp = mgmtDnodeActionEncode, - .decodeFp = mgmtDnodeActionDecode, - .destroyFp = mgmtDnodeActionDestroy, - .restoredFp = mgmtDnodeActionRestored + .insertFp = mnodeDnodeActionInsert, + .deleteFp = mnodeDnodeActionDelete, + .updateFp = mnodeDnodeActionUpdate, + .encodeFp = mnodeDnodeActionEncode, + .decodeFp = mnodeDnodeActionDecode, + .destroyFp = mnodeDnodeActionDestroy, + .restoredFp = mnodeDnodeActionRestored }; tsDnodeSdb = sdbOpenTable(&tableDesc); @@ -148,46 +152,46 @@ int32_t mgmtInitDnodes() { return -1; } - mgmtAddShellMsgHandle(TSDB_MSG_TYPE_CM_CREATE_DNODE, mgmtProcessCreateDnodeMsg); - mgmtAddShellMsgHandle(TSDB_MSG_TYPE_CM_DROP_DNODE, mgmtProcessDropDnodeMsg); - mgmtAddShellMsgHandle(TSDB_MSG_TYPE_CM_CONFIG_DNODE, mgmtProcessCfgDnodeMsg); - dnodeAddClientRspHandle(TSDB_MSG_TYPE_MD_CONFIG_DNODE_RSP, mgmtProcessCfgDnodeMsgRsp); - dnodeAddServerMsgHandle(TSDB_MSG_TYPE_DM_STATUS, mgmtProcessDnodeStatusMsg); - mgmtAddShellShowMetaHandle(TSDB_MGMT_TABLE_MODULE, mgmtGetModuleMeta); - mgmtAddShellShowRetrieveHandle(TSDB_MGMT_TABLE_MODULE, mgmtRetrieveModules); - mgmtAddShellShowMetaHandle(TSDB_MGMT_TABLE_CONFIGS, mgmtGetConfigMeta); - mgmtAddShellShowRetrieveHandle(TSDB_MGMT_TABLE_CONFIGS, mgmtRetrieveConfigs); - mgmtAddShellShowMetaHandle(TSDB_MGMT_TABLE_VNODES, mgmtGetVnodeMeta); - mgmtAddShellShowRetrieveHandle(TSDB_MGMT_TABLE_VNODES, mgmtRetrieveVnodes); - mgmtAddShellShowMetaHandle(TSDB_MGMT_TABLE_DNODE, mgmtGetDnodeMeta); - mgmtAddShellShowRetrieveHandle(TSDB_MGMT_TABLE_DNODE, mgmtRetrieveDnodes); + mnodeAddWriteMsgHandle(TSDB_MSG_TYPE_CM_CREATE_DNODE, mnodeProcessCreateDnodeMsg); + mnodeAddWriteMsgHandle(TSDB_MSG_TYPE_CM_DROP_DNODE, mnodeProcessDropDnodeMsg); + mnodeAddWriteMsgHandle(TSDB_MSG_TYPE_CM_CONFIG_DNODE, mnodeProcessCfgDnodeMsg); + mnodeAddPeerRspHandle(TSDB_MSG_TYPE_MD_CONFIG_DNODE_RSP, mnodeProcessCfgDnodeMsgRsp); + mnodeAddPeerMsgHandle(TSDB_MSG_TYPE_DM_STATUS, mnodeProcessDnodeStatusMsg); + mnodeAddShowMetaHandle(TSDB_MGMT_TABLE_MODULE, mnodeGetModuleMeta); + mnodeAddShowRetrieveHandle(TSDB_MGMT_TABLE_MODULE, mnodeRetrieveModules); + mnodeAddShowMetaHandle(TSDB_MGMT_TABLE_CONFIGS, mnodeGetConfigMeta); + mnodeAddShowRetrieveHandle(TSDB_MGMT_TABLE_CONFIGS, mnodeRetrieveConfigs); + mnodeAddShowMetaHandle(TSDB_MGMT_TABLE_VNODES, mnodeGetVnodeMeta); + mnodeAddShowRetrieveHandle(TSDB_MGMT_TABLE_VNODES, mnodeRetrieveVnodes); + mnodeAddShowMetaHandle(TSDB_MGMT_TABLE_DNODE, mnodeGetDnodeMeta); + mnodeAddShowRetrieveHandle(TSDB_MGMT_TABLE_DNODE, mnodeRetrieveDnodes); mTrace("table:dnodes table is created"); return 0; } -void mgmtCleanupDnodes() { +void mnodeCleanupDnodes() { sdbCloseTable(tsDnodeSdb); } -void *mgmtGetNextDnode(void *pIter, SDnodeObj **pDnode) { +void *mnodeGetNextDnode(void *pIter, SDnodeObj **pDnode) { return sdbFetchRow(tsDnodeSdb, pIter, (void **)pDnode); } -int32_t mgmtGetDnodesNum() { +int32_t mnodeGetDnodesNum() { return sdbGetNumOfRows(tsDnodeSdb); } -int32_t mgmtGetOnlinDnodesNum(char *ep) { +int32_t mnodeGetOnlinDnodesNum(char *ep) { SDnodeObj *pDnode = NULL; void * pIter = NULL; int32_t onlineDnodes = 0; while (1) { - pIter = mgmtGetNextDnode(pIter, &pDnode); + pIter = mnodeGetNextDnode(pIter, &pDnode); if (pDnode == NULL) break; if (pDnode->status != TAOS_DN_STATUS_OFFLINE) onlineDnodes++; - mgmtDecDnodeRef(pDnode); + mnodeDecDnodeRef(pDnode); } sdbFreeIter(pIter); @@ -195,22 +199,22 @@ int32_t mgmtGetOnlinDnodesNum(char *ep) { return onlineDnodes; } -void *mgmtGetDnode(int32_t dnodeId) { +void *mnodeGetDnode(int32_t dnodeId) { return sdbGetRow(tsDnodeSdb, &dnodeId); } -void *mgmtGetDnodeByEp(char *ep) { +void *mnodeGetDnodeByEp(char *ep) { SDnodeObj *pDnode = NULL; void * pIter = NULL; while (1) { - pIter = mgmtGetNextDnode(pIter, &pDnode); + pIter = mnodeGetNextDnode(pIter, &pDnode); if (pDnode == NULL) break; if (strcmp(ep, pDnode->dnodeEp) == 0) { sdbFreeIter(pIter); return pDnode; } - mgmtDecDnodeRef(pDnode); + mnodeDecDnodeRef(pDnode); } sdbFreeIter(pIter); @@ -218,15 +222,15 @@ void *mgmtGetDnodeByEp(char *ep) { return NULL; } -void mgmtIncDnodeRef(SDnodeObj *pDnode) { +void mnodeIncDnodeRef(SDnodeObj *pDnode) { sdbIncRef(tsDnodeSdb, pDnode); } -void mgmtDecDnodeRef(SDnodeObj *pDnode) { +void mnodeDecDnodeRef(SDnodeObj *pDnode) { sdbDecRef(tsDnodeSdb, pDnode); } -void mgmtUpdateDnode(SDnodeObj *pDnode) { +void mnodeUpdateDnode(SDnodeObj *pDnode) { SSdbOper oper = { .type = SDB_OPER_GLOBAL, .table = tsDnodeSdb, @@ -236,10 +240,8 @@ void mgmtUpdateDnode(SDnodeObj *pDnode) { sdbUpdateRow(&oper); } -void mgmtProcessCfgDnodeMsg(SQueuedMsg *pMsg) { - SRpcMsg rpcRsp = {.handle = pMsg->thandle, .pCont = NULL, .contLen = 0, .code = 0, .msgType = 0}; - - SCMCfgDnodeMsg *pCmCfgDnode = pMsg->pCont; +static int32_t mnodeProcessCfgDnodeMsg(SMnodeMsg *pMsg) { + SCMCfgDnodeMsg *pCmCfgDnode = pMsg->rpcMsg.pCont; if (pCmCfgDnode->ep[0] == 0) { strcpy(pCmCfgDnode->ep, tsLocalEp); } else { @@ -247,36 +249,34 @@ void mgmtProcessCfgDnodeMsg(SQueuedMsg *pMsg) { } if (strcmp(pMsg->pUser->user, "root") != 0) { - rpcRsp.code = TSDB_CODE_NO_RIGHTS; - } else { - SRpcIpSet ipSet = mgmtGetIpSetFromIp(pCmCfgDnode->ep); - SMDCfgDnodeMsg *pMdCfgDnode = rpcMallocCont(sizeof(SMDCfgDnodeMsg)); - strcpy(pMdCfgDnode->ep, pCmCfgDnode->ep); - strcpy(pMdCfgDnode->config, pCmCfgDnode->config); - SRpcMsg rpcMdCfgDnodeMsg = { - .handle = 0, - .code = 0, - .msgType = TSDB_MSG_TYPE_MD_CONFIG_DNODE, - .pCont = pMdCfgDnode, - .contLen = sizeof(SMDCfgDnodeMsg) - }; - dnodeSendMsgToDnode(&ipSet, &rpcMdCfgDnodeMsg); - rpcRsp.code = TSDB_CODE_SUCCESS; + return TSDB_CODE_NO_RIGHTS; } - if (rpcRsp.code == TSDB_CODE_SUCCESS) { - mPrint("dnode:%s, is configured by %s", pCmCfgDnode->ep, pMsg->pUser->user); - } + SRpcIpSet ipSet = mnodeGetIpSetFromIp(pCmCfgDnode->ep); + SMDCfgDnodeMsg *pMdCfgDnode = rpcMallocCont(sizeof(SMDCfgDnodeMsg)); + strcpy(pMdCfgDnode->ep, pCmCfgDnode->ep); + strcpy(pMdCfgDnode->config, pCmCfgDnode->config); - rpcSendResponse(&rpcRsp); + SRpcMsg rpcMdCfgDnodeMsg = { + .handle = 0, + .code = 0, + .msgType = TSDB_MSG_TYPE_MD_CONFIG_DNODE, + .pCont = pMdCfgDnode, + .contLen = sizeof(SMDCfgDnodeMsg) + }; + dnodeSendMsgToDnode(&ipSet, &rpcMdCfgDnodeMsg); + + mPrint("dnode:%s, is configured by %s", pCmCfgDnode->ep, pMsg->pUser->user); + + return TSDB_CODE_SUCCESS; } -static void mgmtProcessCfgDnodeMsgRsp(SRpcMsg *rpcMsg) { +static void mnodeProcessCfgDnodeMsgRsp(SRpcMsg *rpcMsg) { mPrint("cfg dnode rsp is received"); } -void mgmtProcessDnodeStatusMsg(SRpcMsg *rpcMsg) { - SDMStatusMsg *pStatus = rpcMsg->pCont; +static int32_t mnodeProcessDnodeStatusMsg(SMnodeMsg *pMsg) { + SDMStatusMsg *pStatus = pMsg->rpcMsg.pCont; pStatus->dnodeId = htonl(pStatus->dnodeId); pStatus->moduleStatus = htonl(pStatus->moduleStatus); pStatus->lastReboot = htonl(pStatus->lastReboot); @@ -286,24 +286,21 @@ void mgmtProcessDnodeStatusMsg(SRpcMsg *rpcMsg) { uint32_t version = htonl(pStatus->version); if (version != tsVersion) { mError("status msg version:%d not equal with mnode:%d", version, tsVersion); - mgmtSendSimpleResp(rpcMsg->handle, TSDB_CODE_INVALID_MSG_VERSION); - return ; + return TSDB_CODE_INVALID_MSG_VERSION; } SDnodeObj *pDnode = NULL; if (pStatus->dnodeId == 0) { - pDnode = mgmtGetDnodeByEp(pStatus->dnodeEp); + pDnode = mnodeGetDnodeByEp(pStatus->dnodeEp); if (pDnode == NULL) { mTrace("dnode %s not created", pStatus->dnodeEp); - mgmtSendSimpleResp(rpcMsg->handle, TSDB_CODE_DNODE_NOT_EXIST); - return; + return TSDB_CODE_DNODE_NOT_EXIST; } } else { - pDnode = mgmtGetDnode(pStatus->dnodeId); + pDnode = mnodeGetDnode(pStatus->dnodeId); if (pDnode == NULL) { mError("dnode id:%d, %s not exist", pStatus->dnodeId, pStatus->dnodeEp); - mgmtSendSimpleResp(rpcMsg->handle, TSDB_CODE_DNODE_NOT_EXIST); - return; + return TSDB_CODE_DNODE_NOT_EXIST; } } @@ -327,34 +324,33 @@ void mgmtProcessDnodeStatusMsg(SRpcMsg *rpcMsg) { pVload->vgId = htonl(pVload->vgId); pVload->cfgVersion = htonl(pVload->cfgVersion); - SVgObj *pVgroup = mgmtGetVgroup(pVload->vgId); + SVgObj *pVgroup = mnodeGetVgroup(pVload->vgId); if (pVgroup == NULL) { - SRpcIpSet ipSet = mgmtGetIpSetFromIp(pDnode->dnodeEp); + SRpcIpSet ipSet = mnodeGetIpSetFromIp(pDnode->dnodeEp); mPrint("dnode:%d, vgId:%d not exist in mnode, drop it", pDnode->dnodeId, pVload->vgId); - mgmtSendDropVnodeMsg(pVload->vgId, &ipSet, NULL); + mnodeSendDropVnodeMsg(pVload->vgId, &ipSet, NULL); } else { - mgmtUpdateVgroupStatus(pVgroup, pDnode, pVload); - mgmtDecVgroupRef(pVgroup); + mnodeUpdateVgroupStatus(pVgroup, pDnode, pVload); + mnodeDecVgroupRef(pVgroup); } } if (pDnode->status == TAOS_DN_STATUS_OFFLINE) { mTrace("dnode:%d, from offline to online", pDnode->dnodeId); pDnode->status = TAOS_DN_STATUS_READY; - balanceUpdateMgmt(); + balanceUpdateMnode(); balanceNotify(); } - mgmtDecDnodeRef(pDnode); + mnodeDecDnodeRef(pDnode); int32_t contLen = sizeof(SDMStatusRsp) + TSDB_MAX_VNODES * sizeof(SDMVgroupAccess); SDMStatusRsp *pRsp = rpcMallocCont(contLen); if (pRsp == NULL) { - mgmtSendSimpleResp(rpcMsg->handle, TSDB_CODE_SERV_OUT_OF_MEMORY); - return; + return TSDB_CODE_SERV_OUT_OF_MEMORY; } - mgmtGetMnodeInfos(&pRsp->mnodes); + mnodeGetMnodeInfos(&pRsp->mnodes); pRsp->dnodeCfg.dnodeId = htonl(pDnode->dnodeId); pRsp->dnodeCfg.moduleStatus = htonl((int32_t)pDnode->isMgmt); @@ -364,25 +360,21 @@ void mgmtProcessDnodeStatusMsg(SRpcMsg *rpcMsg) { //TODO: set vnode access - SRpcMsg rpcRsp = { - .handle = rpcMsg->handle, - .code = TSDB_CODE_SUCCESS, - .pCont = pRsp, - .contLen = contLen - }; + pMsg->rpcRsp.len = contLen; + pMsg->rpcRsp.rsp = pRsp; - rpcSendResponse(&rpcRsp); + return TSDB_CODE_SUCCESS; } -static int32_t mgmtCreateDnode(char *ep) { +static int32_t mnodeCreateDnode(char *ep) { int32_t grantCode = grantCheck(TSDB_GRANT_DNODE); if (grantCode != TSDB_CODE_SUCCESS) { return grantCode; } - SDnodeObj *pDnode = mgmtGetDnodeByEp(ep); + SDnodeObj *pDnode = mnodeGetDnodeByEp(ep); if (pDnode != NULL) { - mgmtDecDnodeRef(pDnode); + mnodeDecDnodeRef(pDnode); mError("dnode:%d is alredy exist, %s:%d", pDnode->dnodeId, pDnode->dnodeFqdn, pDnode->dnodePort); return TSDB_CODE_DNODE_ALREADY_EXIST; } @@ -413,7 +405,7 @@ static int32_t mgmtCreateDnode(char *ep) { return code; } -int32_t mgmtDropDnode(SDnodeObj *pDnode) { +int32_t mnodeDropDnode(SDnodeObj *pDnode) { SSdbOper oper = { .type = SDB_OPER_GLOBAL, .table = tsDnodeSdb, @@ -429,15 +421,14 @@ int32_t mgmtDropDnode(SDnodeObj *pDnode) { return code; } -static int32_t mgmtDropDnodeByEp(char *ep) { - - SDnodeObj *pDnode = mgmtGetDnodeByEp(ep); +static int32_t mnodeDropDnodeByEp(char *ep) { + SDnodeObj *pDnode = mnodeGetDnodeByEp(ep); if (pDnode == NULL) { mError("dnode:%s, is not exist", ep); return TSDB_CODE_DNODE_NOT_EXIST; } - mgmtDecDnodeRef(pDnode); + mnodeDecDnodeRef(pDnode); if (strcmp(pDnode->dnodeEp, dnodeGetMnodeMasterEp()) == 0) { mError("dnode:%d, can't drop dnode:%s which is master", pDnode->dnodeId, ep); return TSDB_CODE_NO_REMOVE_MASTER; @@ -445,58 +436,56 @@ static int32_t mgmtDropDnodeByEp(char *ep) { mPrint("dnode:%d, start to drop it", pDnode->dnodeId); #ifndef _SYNC - return mgmtDropDnode(pDnode); + return mnodeDropDnode(pDnode); #else return balanceDropDnode(pDnode); #endif } -static void mgmtProcessCreateDnodeMsg(SQueuedMsg *pMsg) { - SRpcMsg rpcRsp = {.handle = pMsg->thandle, .pCont = NULL, .contLen = 0, .code = 0, .msgType = 0}; - - SCMCreateDnodeMsg *pCreate = pMsg->pCont; +static int32_t mnodeProcessCreateDnodeMsg(SMnodeMsg *pMsg) { + SCMCreateDnodeMsg *pCreate = pMsg->rpcMsg.pCont; if (strcmp(pMsg->pUser->user, "root") != 0) { - rpcRsp.code = TSDB_CODE_NO_RIGHTS; + return TSDB_CODE_NO_RIGHTS; } else { - rpcRsp.code = mgmtCreateDnode(pCreate->ep); - if (rpcRsp.code == TSDB_CODE_SUCCESS) { - SDnodeObj *pDnode = mgmtGetDnodeByEp(pCreate->ep); + int32_t code = mnodeCreateDnode(pCreate->ep); + + if (code == TSDB_CODE_SUCCESS) { + SDnodeObj *pDnode = mnodeGetDnodeByEp(pCreate->ep); mLPrint("dnode:%d, %s is created by %s", pDnode->dnodeId, pCreate->ep, pMsg->pUser->user); - mgmtDecDnodeRef(pDnode); + mnodeDecDnodeRef(pDnode); } else { - mError("failed to create dnode:%s, reason:%s", pCreate->ep, tstrerror(rpcRsp.code)); + mError("failed to create dnode:%s, reason:%s", pCreate->ep, tstrerror(code)); } + + return code; } - rpcSendResponse(&rpcRsp); } - -static void mgmtProcessDropDnodeMsg(SQueuedMsg *pMsg) { - SRpcMsg rpcRsp = {.handle = pMsg->thandle, .pCont = NULL, .contLen = 0, .code = 0, .msgType = 0}; - - SCMDropDnodeMsg *pDrop = pMsg->pCont; +static int32_t mnodeProcessDropDnodeMsg(SMnodeMsg *pMsg) { + SCMDropDnodeMsg *pDrop = pMsg->rpcMsg.pCont; if (strcmp(pMsg->pUser->user, "root") != 0) { - rpcRsp.code = TSDB_CODE_NO_RIGHTS; + return TSDB_CODE_NO_RIGHTS; } else { - rpcRsp.code = mgmtDropDnodeByEp(pDrop->ep); - if (rpcRsp.code == TSDB_CODE_SUCCESS) { + int32_t code = mnodeDropDnodeByEp(pDrop->ep); + + if (code == TSDB_CODE_SUCCESS) { mLPrint("dnode:%s is dropped by %s", pDrop->ep, pMsg->pUser->user); } else { - mError("failed to drop dnode:%s, reason:%s", pDrop->ep, tstrerror(rpcRsp.code)); + mError("failed to drop dnode:%s, reason:%s", pDrop->ep, tstrerror(code)); } - } - rpcSendResponse(&rpcRsp); + return code; + } } -static int32_t mgmtGetDnodeMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) { - SUserObj *pUser = mgmtGetUserFromConn(pConn); +static int32_t mnodeGetDnodeMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) { + SUserObj *pUser = mnodeGetUserFromConn(pConn); if (pUser == NULL) return 0; if (strcmp(pUser->pAcct->user, "root") != 0) { - mgmtDecUserRef(pUser); + mnodeDecUserRef(pUser); return TSDB_CODE_NO_RIGHTS; } @@ -533,6 +522,12 @@ static int32_t mgmtGetDnodeMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pCo pSchema[cols].bytes = htons(pShow->bytes[cols]); cols++; + pShow->bytes[cols] = 6 + VARSTR_HEADER_SIZE; + pSchema[cols].type = TSDB_DATA_TYPE_BINARY; + strcpy(pSchema[cols].name, "alternativeRole"); + pSchema[cols].bytes = htons(pShow->bytes[cols]); + cols++; + pShow->bytes[cols] = 8; pSchema[cols].type = TSDB_DATA_TYPE_TIMESTAMP; strcpy(pSchema[cols].name, "create_time"); @@ -547,23 +542,23 @@ static int32_t mgmtGetDnodeMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pCo pShow->offset[i] = pShow->offset[i - 1] + pShow->bytes[i - 1]; } - pShow->numOfRows = mgmtGetDnodesNum(); + pShow->numOfRows = mnodeGetDnodesNum(); pShow->rowSize = pShow->offset[cols - 1] + pShow->bytes[cols - 1]; pShow->pIter = NULL; - mgmtDecUserRef(pUser); + mnodeDecUserRef(pUser); return 0; } -static int32_t mgmtRetrieveDnodes(SShowObj *pShow, char *data, int32_t rows, void *pConn) { +static int32_t mnodeRetrieveDnodes(SShowObj *pShow, char *data, int32_t rows, void *pConn) { int32_t numOfRows = 0; int32_t cols = 0; SDnodeObj *pDnode = NULL; char *pWrite; while (numOfRows < rows) { - pShow->pIter = mgmtGetNextDnode(pShow->pIter, &pDnode); + pShow->pIter = mnodeGetNextDnode(pShow->pIter, &pDnode); if (pDnode == NULL) break; cols = 0; @@ -584,38 +579,42 @@ static int32_t mgmtRetrieveDnodes(SShowObj *pShow, char *data, int32_t rows, voi *(int16_t *)pWrite = pDnode->totalVnodes; cols++; - pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows; - - char* status = mgmtGetDnodeStatusStr(pDnode->status); + pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows; + char* status = mnodeGetDnodeStatusStr(pDnode->status); STR_TO_VARSTR(pWrite, status); cols++; + pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows; + char* role = mnodeGetDnodeAlternativeRoleStr(pDnode->alternativeRole); + STR_TO_VARSTR(pWrite, role); + cols++; + pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows; *(int64_t *)pWrite = pDnode->createdTime; cols++; numOfRows++; - mgmtDecDnodeRef(pDnode); + mnodeDecDnodeRef(pDnode); } pShow->numOfReads += numOfRows; return numOfRows; } -static bool mgmtCheckModuleInDnode(SDnodeObj *pDnode, int32_t moduleType) { +static bool mnodeCheckModuleInDnode(SDnodeObj *pDnode, int32_t moduleType) { uint32_t status = pDnode->moduleStatus & (1 << moduleType); return status > 0; } -static int32_t mgmtGetModuleMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) { +static int32_t mnodeGetModuleMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) { int32_t cols = 0; - SUserObj *pUser = mgmtGetUserFromConn(pConn); + SUserObj *pUser = mnodeGetUserFromConn(pConn); if (pUser == NULL) return 0; if (strcmp(pUser->user, "root") != 0) { - mgmtDecUserRef(pUser); + mnodeDecUserRef(pUser); return TSDB_CODE_NO_RIGHTS; } @@ -653,21 +652,21 @@ static int32_t mgmtGetModuleMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pC pShow->offset[i] = pShow->offset[i - 1] + pShow->bytes[i - 1]; } - pShow->numOfRows = mgmtGetDnodesNum() * TSDB_MOD_MAX; + pShow->numOfRows = mnodeGetDnodesNum() * TSDB_MOD_MAX; pShow->rowSize = pShow->offset[cols - 1] + pShow->bytes[cols - 1]; pShow->pIter = NULL; - mgmtDecUserRef(pUser); + mnodeDecUserRef(pUser); return 0; } -int32_t mgmtRetrieveModules(SShowObj *pShow, char *data, int32_t rows, void *pConn) { +int32_t mnodeRetrieveModules(SShowObj *pShow, char *data, int32_t rows, void *pConn) { int32_t numOfRows = 0; char * pWrite; while (numOfRows < rows) { SDnodeObj *pDnode = NULL; - pShow->pIter = mgmtGetNextDnode(pShow->pIter, (SDnodeObj **)&pDnode); + pShow->pIter = mnodeGetNextDnode(pShow->pIter, (SDnodeObj **)&pDnode); if (pDnode == NULL) break; for (int32_t moduleType = 0; moduleType < TSDB_MOD_MAX; ++moduleType) { @@ -683,8 +682,8 @@ int32_t mgmtRetrieveModules(SShowObj *pShow, char *data, int32_t rows, void *pCo pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows; switch (moduleType) { - case TSDB_MOD_MGMT: - strcpy(pWrite, "mgmt"); + case TSDB_MOD_MNODE: + strcpy(pWrite, "mnode"); break; case TSDB_MOD_HTTP: strcpy(pWrite, "http"); @@ -698,34 +697,34 @@ int32_t mgmtRetrieveModules(SShowObj *pShow, char *data, int32_t rows, void *pCo cols++; pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows; - bool enable = mgmtCheckModuleInDnode(pDnode, moduleType); + bool enable = mnodeCheckModuleInDnode(pDnode, moduleType); strcpy(pWrite, enable ? "enable" : "disable"); cols++; numOfRows++; } - mgmtDecDnodeRef(pDnode); + mnodeDecDnodeRef(pDnode); } pShow->numOfReads += numOfRows; return numOfRows; } -static bool mgmtCheckConfigShow(SGlobalCfg *cfg) { +static bool mnodeCheckConfigShow(SGlobalCfg *cfg) { if (!(cfg->cfgType & TSDB_CFG_CTYPE_B_SHOW)) return false; return true; } -static int32_t mgmtGetConfigMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) { +static int32_t mnodeGetConfigMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) { int32_t cols = 0; - SUserObj *pUser = mgmtGetUserFromConn(pConn); + SUserObj *pUser = mnodeGetUserFromConn(pConn); if (pUser == NULL) return 0; if (strcmp(pUser->user, "root") != 0) { - mgmtDecUserRef(pUser); + mnodeDecUserRef(pUser); return TSDB_CODE_NO_RIGHTS; } @@ -752,23 +751,23 @@ static int32_t mgmtGetConfigMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pC pShow->numOfRows = 0; for (int32_t i = tsGlobalConfigNum - 1; i >= 0; --i) { SGlobalCfg *cfg = tsGlobalConfig + i; - if (!mgmtCheckConfigShow(cfg)) continue; + if (!mnodeCheckConfigShow(cfg)) continue; pShow->numOfRows++; } pShow->rowSize = pShow->offset[cols - 1] + pShow->bytes[cols - 1]; pShow->pIter = NULL; - mgmtDecUserRef(pUser); + mnodeDecUserRef(pUser); return 0; } -static int32_t mgmtRetrieveConfigs(SShowObj *pShow, char *data, int32_t rows, void *pConn) { +static int32_t mnodeRetrieveConfigs(SShowObj *pShow, char *data, int32_t rows, void *pConn) { int32_t numOfRows = 0; for (int32_t i = tsGlobalConfigNum - 1; i >= 0 && numOfRows < rows; --i) { SGlobalCfg *cfg = tsGlobalConfig + i; - if (!mgmtCheckConfigShow(cfg)) continue; + if (!mnodeCheckConfigShow(cfg)) continue; char *pWrite; int32_t cols = 0; @@ -806,13 +805,13 @@ static int32_t mgmtRetrieveConfigs(SShowObj *pShow, char *data, int32_t rows, vo return numOfRows; } -static int32_t mgmtGetVnodeMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) { +static int32_t mnodeGetVnodeMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) { int32_t cols = 0; - SUserObj *pUser = mgmtGetUserFromConn(pConn); + SUserObj *pUser = mnodeGetUserFromConn(pConn); if (pUser == NULL) return 0; if (strcmp(pUser->user, "root") != 0) { - mgmtDecUserRef(pUser); + mnodeDecUserRef(pUser); return TSDB_CODE_NO_RIGHTS; } @@ -838,25 +837,25 @@ static int32_t mgmtGetVnodeMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pCo SDnodeObj *pDnode = NULL; if (pShow->payloadLen > 0 ) { - pDnode = mgmtGetDnodeByEp(pShow->payload); + pDnode = mnodeGetDnodeByEp(pShow->payload); } else { - void *pIter = mgmtGetNextDnode(NULL, (SDnodeObj **)&pDnode); + void *pIter = mnodeGetNextDnode(NULL, (SDnodeObj **)&pDnode); sdbFreeIter(pIter); } if (pDnode != NULL) { pShow->numOfRows += pDnode->openVnodes; - mgmtDecDnodeRef(pDnode); + mnodeDecDnodeRef(pDnode); } pShow->rowSize = pShow->offset[cols - 1] + pShow->bytes[cols - 1]; pShow->pIter = pDnode; - mgmtDecUserRef(pUser); + mnodeDecUserRef(pUser); return 0; } -static int32_t mgmtRetrieveVnodes(SShowObj *pShow, char *data, int32_t rows, void *pConn) { +static int32_t mnodeRetrieveVnodes(SShowObj *pShow, char *data, int32_t rows, void *pConn) { int32_t numOfRows = 0; SDnodeObj *pDnode = NULL; char * pWrite; @@ -869,7 +868,7 @@ static int32_t mgmtRetrieveVnodes(SShowObj *pShow, char *data, int32_t rows, voi void *pIter = NULL; SVgObj *pVgroup; while (1) { - pIter = mgmtGetNextVgroup(pIter, &pVgroup); + pIter = mnodeGetNextVgroup(pIter, &pVgroup); if (pVgroup == NULL) break; for (int32_t i = 0; i < pVgroup->numOfVnodes; ++i) { @@ -882,12 +881,12 @@ static int32_t mgmtRetrieveVnodes(SShowObj *pShow, char *data, int32_t rows, voi cols++; pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows; - strcpy(pWrite, mgmtGetMnodeRoleStr(pVgid->role)); + strcpy(pWrite, mnodeGetMnodeRoleStr(pVgid->role)); cols++; } } - mgmtDecVgroupRef(pVgroup); + mnodeDecVgroupRef(pVgroup); } sdbFreeIter(pIter); } else { @@ -898,7 +897,7 @@ static int32_t mgmtRetrieveVnodes(SShowObj *pShow, char *data, int32_t rows, voi return numOfRows; } -char* mgmtGetDnodeStatusStr(int32_t dnodeStatus) { +char* mnodeGetDnodeStatusStr(int32_t dnodeStatus) { switch (dnodeStatus) { case TAOS_DN_STATUS_OFFLINE: return "offline"; case TAOS_DN_STATUS_DROPPING: return "dropping"; @@ -907,3 +906,13 @@ char* mgmtGetDnodeStatusStr(int32_t dnodeStatus) { default: return "undefined"; } } + +static char* mnodeGetDnodeAlternativeRoleStr(int32_t alternativeRole) { + switch (alternativeRole) { + case TAOS_DN_ALTERNATIVE_ROLE_ANY: return "any"; + case TAOS_DN_ALTERNATIVE_ROLE_MNODE: return "mnode"; + case TAOS_DN_ALTERNATIVE_ROLE_VNODE: return "vnode"; + default:return "any"; + } +} + diff --git a/src/mnode/src/mgmtGrant.c b/src/mnode/src/mnodeGrant.c similarity index 97% rename from src/mnode/src/mgmtGrant.c rename to src/mnode/src/mnodeGrant.c index f44b47e20c9b516ebb81b84e488664d74bdca8f5..7329629ef92f61d29794807bd868cda354270548 100644 --- a/src/mnode/src/mgmtGrant.c +++ b/src/mnode/src/mnodeGrant.c @@ -18,7 +18,7 @@ #include "os.h" #include "taoserror.h" #include "tgrant.h" -#include "mgmtInt.h" +#include "mnodeInt.h" int32_t grantInit() { return TSDB_CODE_SUCCESS; } void grantCleanUp() {} diff --git a/src/mnode/src/mnodeInt.c b/src/mnode/src/mnodeInt.c new file mode 100644 index 0000000000000000000000000000000000000000..1cb421bef72d46513deb62ec80f2493b528dfd98 --- /dev/null +++ b/src/mnode/src/mnodeInt.c @@ -0,0 +1,60 @@ +/* + * Copyright (c) 2019 TAOS Data, Inc. + * + * This program is free software: you can use, redistribute, and/or modify + * it under the terms of the GNU Affero General Public License, version 3 + * or later ("AGPL"), as published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ + +#define _DEFAULT_SOURCE +#include "os.h" +#include "taosmsg.h" +#include "taoserror.h" +#include "trpc.h" +#include "tcache.h" +#include "mnode.h" +#include "dnode.h" +#include "mnodeDef.h" +#include "mnodeInt.h" +#include "mnodeAcct.h" +#include "mnodeDb.h" +#include "mnodeDnode.h" +#include "mnodeMnode.h" +#include "mnodeProfile.h" +#include "mnodeSdb.h" +#include "mnodeShow.h" +#include "mnodeTable.h" +#include "mnodeUser.h" +#include "mnodeVgroup.h" + +void mnodeCreateMsg(SMnodeMsg *pMsg, SRpcMsg *rpcMsg) { + pMsg->rpcMsg = *rpcMsg; +} + +int32_t mnodeInitMsg(SMnodeMsg *pMsg) { + pMsg->pUser = mnodeGetUserFromConn(pMsg->rpcMsg.handle); + if (pMsg->pUser == NULL) { + return TSDB_CODE_INVALID_USER; + } + + return TSDB_CODE_SUCCESS; +} + +void mnodeCleanupMsg(SMnodeMsg *pMsg) { + if (pMsg != NULL) { + if (pMsg->rpcMsg.pCont) rpcFreeCont(pMsg->rpcMsg.pCont); + if (pMsg->pUser) mnodeDecUserRef(pMsg->pUser); + if (pMsg->pDb) mnodeDecDbRef(pMsg->pDb); + if (pMsg->pVgroup) mnodeDecVgroupRef(pMsg->pVgroup); + if (pMsg->pTable) mnodeDecTableRef(pMsg->pTable); + if (pMsg->pAcct) mnodeDecAcctRef(pMsg->pAcct); + if (pMsg->pDnode) mnodeDecDnodeRef(pMsg->pDnode); + } +} diff --git a/src/mnode/src/mgmtMain.c b/src/mnode/src/mnodeMain.c similarity index 50% rename from src/mnode/src/mgmtMain.c rename to src/mnode/src/mnodeMain.c index 2a8e139eecc47934df09ca49e5ccab64dcf66985..baecfac0775983ca34706a838a1667544c1bf179 100644 --- a/src/mnode/src/mgmtMain.c +++ b/src/mnode/src/mnodeMain.c @@ -22,65 +22,72 @@ #include "ttimer.h" #include "tglobal.h" #include "dnode.h" -#include "mgmtDef.h" -#include "mgmtInt.h" -#include "mgmtServer.h" -#include "mgmtAcct.h" -#include "mgmtDnode.h" -#include "mgmtMnode.h" -#include "mgmtDb.h" -#include "mgmtSdb.h" -#include "mgmtVgroup.h" -#include "mgmtUser.h" -#include "mgmtTable.h" -#include "mgmtShell.h" - -extern void *tsMgmtTmr; +#include "mnodeDef.h" +#include "mnodeInt.h" +#include "mnodeAcct.h" +#include "mnodeDnode.h" +#include "mnodeMnode.h" +#include "mnodeDb.h" +#include "mnodeSdb.h" +#include "mnodeVgroup.h" +#include "mnodeUser.h" +#include "mnodeTable.h" +#include "mnodeShow.h" + +void *tsMnodeTmr; static bool tsMgmtIsRunning = false; -int32_t mgmtStartSystem() { +static void mnodeInitTimer(); +static void mnodeCleanupTimer(); +static bool mnodeNeedStart() ; + +int32_t mnodeStartSystem() { if (tsMgmtIsRunning) { - mPrint("TDengine mgmt module already started..."); + mPrint("mnode module already started..."); return 0; } - mPrint("starting to initialize TDengine mgmt ..."); + mPrint("starting to initialize mnode ..."); struct stat dirstat; if (stat(tsMnodeDir, &dirstat) < 0) { mkdir(tsMnodeDir, 0755); } - if (mgmtInitAccts() < 0) { + dnodeAllocateMnodeWqueue(); + dnodeAllocateMnodeRqueue(); + dnodeAllocateMnodePqueue(); + + if (mnodeInitAccts() < 0) { mError("failed to init accts"); return -1; } - if (mgmtInitUsers() < 0) { + if (mnodeInitUsers() < 0) { mError("failed to init users"); return -1; } - if (mgmtInitDnodes() < 0) { + if (mnodeInitDnodes() < 0) { mError("failed to init dnodes"); return -1; } - if (mgmtInitDbs() < 0) { + if (mnodeInitDbs() < 0) { mError("failed to init dbs"); return -1; } - if (mgmtInitVgroups() < 0) { + if (mnodeInitVgroups() < 0) { mError("failed to init vgroups"); return -1; } - if (mgmtInitTables() < 0) { + if (mnodeInitTables() < 0) { mError("failed to init tables"); return -1; } - if (mgmtInitMnodes() < 0) { + if (mnodeInitMnodes() < 0) { mError("failed to init mnodes"); return -1; } @@ -99,63 +106,83 @@ int32_t mgmtStartSystem() { return -1; } - if (mgmtInitServer() < 0) { + if (mnodeInitShow() < 0) { + mError("failed to init show"); return -1; } grantReset(TSDB_GRANT_ALL, 0); tsMgmtIsRunning = true; - mPrint("TDengine mgmt is initialized successfully"); + mPrint("mnode is initialized successfully"); return 0; } -int32_t mgmtInitSystem() { - if (mgmtInitShell() != 0) { - mError("failed to init shell"); - return -1; - } - - struct stat dirstat; - bool fileExist = (stat(tsMnodeDir, &dirstat) == 0); - bool asMaster = (strcmp(tsFirst, tsLocalEp) == 0); - - if (asMaster || fileExist) { - if (mgmtStartSystem() != 0) { - return -1; - } - } - - return 0; +int32_t mnodeInitSystem() { + mnodeInitTimer(); + if (!mnodeNeedStart()) return 0; + return mnodeStartSystem(); } -void mgmtCleanUpSystem() { - mPrint("starting to clean up mgmt"); +void mnodeCleanupSystem() { + mPrint("starting to clean up mnode"); tsMgmtIsRunning = false; - mgmtCleanUpShell(); - mgmtCleanupServer(); + + dnodeFreeMnodeWqueue(); + dnodeFreeMnodeRqueue(); + dnodeFreeMnodePqueue(); + mnodeCleanupTimer(); + mnodeCleanUpShow(); grantCleanUp(); balanceCleanUp(); sdbCleanUp(); - mgmtCleanupMnodes(); - mgmtCleanUpTables(); - mgmtCleanUpVgroups(); - mgmtCleanUpDbs(); - mgmtCleanupDnodes(); - mgmtCleanUpUsers(); - mgmtCleanUpAccts(); - mPrint("mgmt is cleaned up"); + mnodeCleanupMnodes(); + mnodeCleanupTables(); + mnodeCleanupVgroups(); + mnodeCleanupDbs(); + mnodeCleanupDnodes(); + mnodeCleanupUsers(); + mnodeCleanupAccts(); + mPrint("mnode is cleaned up"); } -void mgmtStopSystem() { +void mnodeStopSystem() { if (sdbIsMaster()) { - mTrace("it is a master mgmt node, it could not be stopped"); + mTrace("it is a master mnode, it could not be stopped"); return; } + + mnodeCleanupSystem(); + mPrint("mnode file is removed"); + remove(tsMnodeDir); +} - mgmtCleanUpSystem(); +static void mnodeInitTimer() { + if (tsMnodeTmr == NULL) { + tsMnodeTmr = taosTmrInit(tsMaxShellConns, 200, 3600000, "MND"); + } +} - mPrint("mgmt file is removed"); - remove(tsMnodeDir); +static void mnodeCleanupTimer() { + if (tsMnodeTmr != NULL) { + taosTmrCleanUp(tsMnodeTmr); + tsMnodeTmr = NULL; + } +} + +static bool mnodeNeedStart() { + struct stat dirstat; + bool fileExist = (stat(tsMnodeDir, &dirstat) == 0); + bool asMaster = (strcmp(tsFirst, tsLocalEp) == 0); + + if (asMaster || fileExist) { + return true; + } + + return false; +} + +bool mnodeIsRunning() { + return tsMgmtIsRunning; } diff --git a/src/mnode/src/mgmtMnode.c b/src/mnode/src/mnodeMnode.c similarity index 53% rename from src/mnode/src/mgmtMnode.c rename to src/mnode/src/mnodeMnode.c index 6471b7f182b41c9e077230671f722f8e37b1985d..de1826a174ca7922033d0d3c546b5f00646141fd 100644 --- a/src/mnode/src/mgmtMnode.c +++ b/src/mnode/src/mnodeMnode.c @@ -23,85 +23,85 @@ #include "ttime.h" #include "tsocket.h" #include "tdataformat.h" -#include "mgmtDef.h" -#include "mgmtInt.h" -#include "mgmtMnode.h" -#include "mgmtDnode.h" -#include "mgmtSdb.h" -#include "mgmtShell.h" -#include "mgmtUser.h" - -static void * tsMnodeSdb = NULL; -static int32_t tsMnodeUpdateSize = 0; -static int32_t mgmtGetMnodeMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn); -static int32_t mgmtRetrieveMnodes(SShowObj *pShow, char *data, int32_t rows, void *pConn); - -static SRpcIpSet tsMnodeRpcIpSet; +#include "mnodeDef.h" +#include "mnodeInt.h" +#include "mnodeMnode.h" +#include "mnodeDnode.h" +#include "mnodeSdb.h" +#include "mnodeShow.h" +#include "mnodeUser.h" + +static void * tsMnodeSdb = NULL; +static int32_t tsMnodeUpdateSize = 0; +static SRpcIpSet tsMnodeIpSetForShell; +static SRpcIpSet tsMnodeIpSetForPeer; static SDMMnodeInfos tsMnodeInfos; +static int32_t mnodeGetMnodeMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn); +static int32_t mnodeRetrieveMnodes(SShowObj *pShow, char *data, int32_t rows, void *pConn); #if defined(LINUX) - static pthread_rwlock_t tsMnodeLock; - #define mgmtMnodeWrLock() pthread_rwlock_wrlock(&tsMnodeLock) - #define mgmtMnodeRdLock() pthread_rwlock_rdlock(&tsMnodeLock) - #define mgmtMnodeUnLock() pthread_rwlock_unlock(&tsMnodeLock) - #define mgmtMnodeInitLock() pthread_rwlock_init(&tsMnodeLock, NULL) - #define mgmtMnodeDestroyLock() pthread_rwlock_destroy(&tsMnodeLock) + static pthread_rwlock_t tsMnodeLock; + #define mnodeMnodeWrLock() pthread_rwlock_wrlock(&tsMnodeLock) + #define mnodeMnodeRdLock() pthread_rwlock_rdlock(&tsMnodeLock) + #define mnodeMnodeUnLock() pthread_rwlock_unlock(&tsMnodeLock) + #define mnodeMnodeInitLock() pthread_rwlock_init(&tsMnodeLock, NULL) + #define mnodeMnodeDestroyLock() pthread_rwlock_destroy(&tsMnodeLock) #else - static pthread_mutex_t tsMnodeLock; - #define mgmtMnodeWrLock() pthread_mutex_lock(&tsMnodeLock) - #define mgmtMnodeRdLock() pthread_mutex_lock(&tsMnodeLock) - #define mgmtMnodeUnLock() pthread_mutex_unlock(&tsMnodeLock) - #define mgmtMnodeInitLock() pthread_mutex_init(&tsMnodeLock, NULL) - #define mgmtMnodeDestroyLock() pthread_mutex_destroy(&tsMnodeLock) + static pthread_mutex_t tsMnodeLock; + #define mnodeMnodeWrLock() pthread_mutex_lock(&tsMnodeLock) + #define mnodeMnodeRdLock() pthread_mutex_lock(&tsMnodeLock) + #define mnodeMnodeUnLock() pthread_mutex_unlock(&tsMnodeLock) + #define mnodeMnodeInitLock() pthread_mutex_init(&tsMnodeLock, NULL) + #define mnodeMnodeDestroyLock() pthread_mutex_destroy(&tsMnodeLock) #endif -static int32_t mgmtMnodeActionDestroy(SSdbOper *pOper) { +static int32_t mnodeMnodeActionDestroy(SSdbOper *pOper) { tfree(pOper->pObj); return TSDB_CODE_SUCCESS; } -static int32_t mgmtMnodeActionInsert(SSdbOper *pOper) { +static int32_t mnodeMnodeActionInsert(SSdbOper *pOper) { SMnodeObj *pMnode = pOper->pObj; - SDnodeObj *pDnode = mgmtGetDnode(pMnode->mnodeId); + SDnodeObj *pDnode = mnodeGetDnode(pMnode->mnodeId); if (pDnode == NULL) return TSDB_CODE_DNODE_NOT_EXIST; pDnode->isMgmt = true; - mgmtDecDnodeRef(pDnode); + mnodeDecDnodeRef(pDnode); return TSDB_CODE_SUCCESS; } -static int32_t mgmtMnodeActionDelete(SSdbOper *pOper) { +static int32_t mnodeMnodeActionDelete(SSdbOper *pOper) { SMnodeObj *pMnode = pOper->pObj; - SDnodeObj *pDnode = mgmtGetDnode(pMnode->mnodeId); + SDnodeObj *pDnode = mnodeGetDnode(pMnode->mnodeId); if (pDnode == NULL) return TSDB_CODE_DNODE_NOT_EXIST; pDnode->isMgmt = false; - mgmtDecDnodeRef(pDnode); + mnodeDecDnodeRef(pDnode); mTrace("mnode:%d, is dropped from sdb", pMnode->mnodeId); return TSDB_CODE_SUCCESS; } -static int32_t mgmtMnodeActionUpdate(SSdbOper *pOper) { +static int32_t mnodeMnodeActionUpdate(SSdbOper *pOper) { SMnodeObj *pMnode = pOper->pObj; - SMnodeObj *pSaved = mgmtGetMnode(pMnode->mnodeId); + SMnodeObj *pSaved = mnodeGetMnode(pMnode->mnodeId); if (pMnode != pSaved) { memcpy(pSaved, pMnode, pOper->rowSize); free(pMnode); } - mgmtDecMnodeRef(pSaved); + mnodeDecMnodeRef(pSaved); return TSDB_CODE_SUCCESS; } -static int32_t mgmtMnodeActionEncode(SSdbOper *pOper) { +static int32_t mnodeMnodeActionEncode(SSdbOper *pOper) { SMnodeObj *pMnode = pOper->pObj; memcpy(pOper->rowData, pMnode, tsMnodeUpdateSize); pOper->rowSize = tsMnodeUpdateSize; return TSDB_CODE_SUCCESS; } -static int32_t mgmtMnodeActionDecode(SSdbOper *pOper) { +static int32_t mnodeMnodeActionDecode(SSdbOper *pOper) { SMnodeObj *pMnode = calloc(1, sizeof(SMnodeObj)); if (pMnode == NULL) return TSDB_CODE_SERV_OUT_OF_MEMORY; @@ -110,24 +110,24 @@ static int32_t mgmtMnodeActionDecode(SSdbOper *pOper) { return TSDB_CODE_SUCCESS; } -static int32_t mgmtMnodeActionRestored() { - if (mgmtGetMnodesNum() == 1) { +static int32_t mnodeMnodeActionRestored() { + if (mnodeGetMnodesNum() == 1) { SMnodeObj *pMnode = NULL; - void *pIter = mgmtGetNextMnode(NULL, &pMnode); + void *pIter = mnodeGetNextMnode(NULL, &pMnode); if (pMnode != NULL) { pMnode->role = TAOS_SYNC_ROLE_MASTER; - mgmtDecMnodeRef(pMnode); + mnodeDecMnodeRef(pMnode); } sdbFreeIter(pIter); } - mgmtUpdateMnodeIpSet(); + mnodeUpdateMnodeIpSet(); return TSDB_CODE_SUCCESS; } -int32_t mgmtInitMnodes() { - mgmtMnodeInitLock(); +int32_t mnodeInitMnodes() { + mnodeMnodeInitLock(); SMnodeObj tObj; tsMnodeUpdateSize = (int8_t *)tObj.updateEnd - (int8_t *)&tObj; @@ -139,13 +139,13 @@ int32_t mgmtInitMnodes() { .maxRowSize = tsMnodeUpdateSize, .refCountPos = (int8_t *)(&tObj.refCount) - (int8_t *)&tObj, .keyType = SDB_KEY_INT, - .insertFp = mgmtMnodeActionInsert, - .deleteFp = mgmtMnodeActionDelete, - .updateFp = mgmtMnodeActionUpdate, - .encodeFp = mgmtMnodeActionEncode, - .decodeFp = mgmtMnodeActionDecode, - .destroyFp = mgmtMnodeActionDestroy, - .restoredFp = mgmtMnodeActionRestored + .insertFp = mnodeMnodeActionInsert, + .deleteFp = mnodeMnodeActionDelete, + .updateFp = mnodeMnodeActionUpdate, + .encodeFp = mnodeMnodeActionEncode, + .decodeFp = mnodeMnodeActionDecode, + .destroyFp = mnodeMnodeActionDestroy, + .restoredFp = mnodeMnodeActionRestored }; tsMnodeSdb = sdbOpenTable(&tableDesc); @@ -154,39 +154,39 @@ int32_t mgmtInitMnodes() { return -1; } - mgmtAddShellShowMetaHandle(TSDB_MGMT_TABLE_MNODE, mgmtGetMnodeMeta); - mgmtAddShellShowRetrieveHandle(TSDB_MGMT_TABLE_MNODE, mgmtRetrieveMnodes); + mnodeAddShowMetaHandle(TSDB_MGMT_TABLE_MNODE, mnodeGetMnodeMeta); + mnodeAddShowRetrieveHandle(TSDB_MGMT_TABLE_MNODE, mnodeRetrieveMnodes); mTrace("table:mnodes table is created"); return TSDB_CODE_SUCCESS; } -void mgmtCleanupMnodes() { +void mnodeCleanupMnodes() { sdbCloseTable(tsMnodeSdb); - mgmtMnodeDestroyLock(); + mnodeMnodeDestroyLock(); } -int32_t mgmtGetMnodesNum() { +int32_t mnodeGetMnodesNum() { return sdbGetNumOfRows(tsMnodeSdb); } -void *mgmtGetMnode(int32_t mnodeId) { +void *mnodeGetMnode(int32_t mnodeId) { return sdbGetRow(tsMnodeSdb, &mnodeId); } -void mgmtIncMnodeRef(SMnodeObj *pMnode) { +void mnodeIncMnodeRef(SMnodeObj *pMnode) { sdbIncRef(tsMnodeSdb, pMnode); } -void mgmtDecMnodeRef(SMnodeObj *pMnode) { +void mnodeDecMnodeRef(SMnodeObj *pMnode) { sdbDecRef(tsMnodeSdb, pMnode); } -void *mgmtGetNextMnode(void *pIter, SMnodeObj **pMnode) { +void *mnodeGetNextMnode(void *pIter, SMnodeObj **pMnode) { return sdbFetchRow(tsMnodeSdb, pIter, (void **)pMnode); } -char *mgmtGetMnodeRoleStr(int32_t role) { +char *mnodeGetMnodeRoleStr(int32_t role) { switch (role) { case TAOS_SYNC_ROLE_OFFLINE: return "offline"; @@ -201,68 +201,77 @@ char *mgmtGetMnodeRoleStr(int32_t role) { } } -void mgmtUpdateMnodeIpSet() { - SRpcIpSet *ipSet = &tsMnodeRpcIpSet; - SDMMnodeInfos *mnodes = &tsMnodeInfos; +void mnodeUpdateMnodeIpSet() { + mPrint("update mnodes ipset, numOfIps:%d ", mnodeGetMnodesNum()); - mPrint("update mnodes ipset, numOfIps:%d ", mgmtGetMnodesNum()); + mnodeMnodeWrLock(); - mgmtMnodeWrLock(); - - memset(ipSet, 0, sizeof(tsMnodeRpcIpSet)); - memset(mnodes, 0, sizeof(SDMMnodeInfos)); + memset(&tsMnodeIpSetForShell, 0, sizeof(SRpcIpSet)); + memset(&tsMnodeIpSetForPeer, 0, sizeof(SRpcIpSet)); + memset(&tsMnodeInfos, 0, sizeof(SDMMnodeInfos)); int32_t index = 0; void * pIter = NULL; while (1) { SMnodeObj *pMnode = NULL; - pIter = mgmtGetNextMnode(pIter, &pMnode); + pIter = mnodeGetNextMnode(pIter, &pMnode); if (pMnode == NULL) break; - SDnodeObj *pDnode = mgmtGetDnode(pMnode->mnodeId); + SDnodeObj *pDnode = mnodeGetDnode(pMnode->mnodeId); if (pDnode != NULL) { - strcpy(ipSet->fqdn[ipSet->numOfIps], pDnode->dnodeFqdn); - ipSet->port[ipSet->numOfIps] = htons(pDnode->dnodePort); + strcpy(tsMnodeIpSetForShell.fqdn[index], pDnode->dnodeFqdn); + tsMnodeIpSetForShell.port[index] = htons(pDnode->dnodePort); + mTrace("mnode:%d, for shell fqdn:%s %d", pDnode->dnodeId, tsMnodeIpSetForShell.fqdn[index], htons(tsMnodeIpSetForShell.port[index])); + + strcpy(tsMnodeIpSetForPeer.fqdn[index], pDnode->dnodeFqdn); + tsMnodeIpSetForPeer.port[index] = htons(pDnode->dnodePort + TSDB_PORT_DNODEDNODE); + mTrace("mnode:%d, for peer fqdn:%s %d", pDnode->dnodeId, tsMnodeIpSetForPeer.fqdn[index], htons(tsMnodeIpSetForPeer.port[index])); - mnodes->nodeInfos[index].nodeId = htonl(pMnode->mnodeId); - strcpy(mnodes->nodeInfos[index].nodeEp, pDnode->dnodeEp); + tsMnodeInfos.nodeInfos[index].nodeId = htonl(pMnode->mnodeId); + strcpy(tsMnodeInfos.nodeInfos[index].nodeEp, pDnode->dnodeEp); if (pMnode->role == TAOS_SYNC_ROLE_MASTER) { - ipSet->inUse = ipSet->numOfIps; - mnodes->inUse = index; + tsMnodeIpSetForShell.inUse = index; + tsMnodeIpSetForPeer.inUse = index; + tsMnodeInfos.inUse = index; } - mPrint("mnode:%d, ep:%s %s", index, pDnode->dnodeEp, - pMnode->role == TAOS_SYNC_ROLE_MASTER ? "master" : ""); - - ipSet->numOfIps++; + mPrint("mnode:%d, ep:%s %s", pDnode->dnodeId, pDnode->dnodeEp, pMnode->role == TAOS_SYNC_ROLE_MASTER ? "master" : ""); index++; } - mgmtDecDnodeRef(pDnode); - mgmtDecMnodeRef(pMnode); + mnodeDecDnodeRef(pDnode); + mnodeDecMnodeRef(pMnode); } - mnodes->nodeNum = index; + tsMnodeInfos.nodeNum = index; + tsMnodeIpSetForShell.numOfIps = index; + tsMnodeIpSetForPeer.numOfIps = index; sdbFreeIter(pIter); - mgmtMnodeUnLock(); + mnodeMnodeUnLock(); +} + +void mnodeGetMnodeIpSetForPeer(SRpcIpSet *ipSet) { + mnodeMnodeRdLock(); + *ipSet = tsMnodeIpSetForPeer; + mnodeMnodeUnLock(); } -void mgmtGetMnodeIpSet(SRpcIpSet *ipSet) { - mgmtMnodeRdLock(); - *ipSet = tsMnodeRpcIpSet; - mgmtMnodeUnLock(); +void mnodeGetMnodeIpSetForShell(SRpcIpSet *ipSet) { + mnodeMnodeRdLock(); + *ipSet = tsMnodeIpSetForShell; + mnodeMnodeUnLock(); } -void mgmtGetMnodeInfos(void *mnodeInfos) { - mgmtMnodeRdLock(); +void mnodeGetMnodeInfos(void *mnodeInfos) { + mnodeMnodeRdLock(); *(SDMMnodeInfos *)mnodeInfos = tsMnodeInfos; - mgmtMnodeUnLock(); + mnodeMnodeUnLock(); } -int32_t mgmtAddMnode(int32_t dnodeId) { +int32_t mnodeAddMnode(int32_t dnodeId) { SMnodeObj *pMnode = calloc(1, sizeof(SMnodeObj)); pMnode->mnodeId = dnodeId; pMnode->createdTime = taosGetTimestampMs(); @@ -279,24 +288,24 @@ int32_t mgmtAddMnode(int32_t dnodeId) { code = TSDB_CODE_SDB_ERROR; } - mgmtUpdateMnodeIpSet(); + mnodeUpdateMnodeIpSet(); return code; } -void mgmtDropMnodeLocal(int32_t dnodeId) { - SMnodeObj *pMnode = mgmtGetMnode(dnodeId); +void mnodeDropMnodeLocal(int32_t dnodeId) { + SMnodeObj *pMnode = mnodeGetMnode(dnodeId); if (pMnode != NULL) { SSdbOper oper = {.type = SDB_OPER_LOCAL, .table = tsMnodeSdb, .pObj = pMnode}; sdbDeleteRow(&oper); - mgmtDecMnodeRef(pMnode); + mnodeDecMnodeRef(pMnode); } - mgmtUpdateMnodeIpSet(); + mnodeUpdateMnodeIpSet(); } -int32_t mgmtDropMnode(int32_t dnodeId) { - SMnodeObj *pMnode = mgmtGetMnode(dnodeId); +int32_t mnodeDropMnode(int32_t dnodeId) { + SMnodeObj *pMnode = mnodeGetMnode(dnodeId); if (pMnode == NULL) { return TSDB_CODE_DNODE_NOT_EXIST; } @@ -314,18 +323,18 @@ int32_t mgmtDropMnode(int32_t dnodeId) { sdbDecRef(tsMnodeSdb, pMnode); - mgmtUpdateMnodeIpSet(); + mnodeUpdateMnodeIpSet(); return code; } -static int32_t mgmtGetMnodeMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) { +static int32_t mnodeGetMnodeMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) { sdbUpdateMnodeRoles(); - SUserObj *pUser = mgmtGetUserFromConn(pConn); + SUserObj *pUser = mnodeGetUserFromConn(pConn); if (pUser == NULL) return 0; if (strcmp(pUser->pAcct->user, "root") != 0) { - mgmtDecUserRef(pUser); + mnodeDecUserRef(pUser); return TSDB_CODE_NO_RIGHTS; } @@ -364,22 +373,22 @@ static int32_t mgmtGetMnodeMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pCo pShow->offset[i] = pShow->offset[i - 1] + pShow->bytes[i - 1]; } - pShow->numOfRows = mgmtGetMnodesNum(); + pShow->numOfRows = mnodeGetMnodesNum(); pShow->rowSize = pShow->offset[cols - 1] + pShow->bytes[cols - 1]; pShow->pIter = NULL; - mgmtDecUserRef(pUser); + mnodeDecUserRef(pUser); return 0; } -static int32_t mgmtRetrieveMnodes(SShowObj *pShow, char *data, int32_t rows, void *pConn) { +static int32_t mnodeRetrieveMnodes(SShowObj *pShow, char *data, int32_t rows, void *pConn) { int32_t numOfRows = 0; int32_t cols = 0; SMnodeObj *pMnode = NULL; char *pWrite; while (numOfRows < rows) { - pShow->pIter = mgmtGetNextMnode(pShow->pIter, &pMnode); + pShow->pIter = mnodeGetNextMnode(pShow->pIter, &pMnode); if (pMnode == NULL) break; cols = 0; @@ -390,18 +399,18 @@ static int32_t mgmtRetrieveMnodes(SShowObj *pShow, char *data, int32_t rows, voi pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows; - SDnodeObj *pDnode = mgmtGetDnode(pMnode->mnodeId); + SDnodeObj *pDnode = mnodeGetDnode(pMnode->mnodeId); if (pDnode != NULL) { STR_WITH_MAXSIZE_TO_VARSTR(pWrite, pDnode->dnodeEp, pShow->bytes[cols] - VARSTR_HEADER_SIZE); } else { STR_WITH_MAXSIZE_TO_VARSTR(pWrite, "invalid ep", pShow->bytes[cols] - VARSTR_HEADER_SIZE); } - mgmtDecDnodeRef(pDnode); + mnodeDecDnodeRef(pDnode); cols++; pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows; - char* roles = mgmtGetMnodeRoleStr(pMnode->role); + char* roles = mnodeGetMnodeRoleStr(pMnode->role); STR_TO_VARSTR(pWrite, roles); cols++; @@ -411,7 +420,7 @@ static int32_t mgmtRetrieveMnodes(SShowObj *pShow, char *data, int32_t rows, voi numOfRows++; - mgmtDecMnodeRef(pMnode); + mnodeDecMnodeRef(pMnode); } pShow->numOfReads += numOfRows; diff --git a/src/mnode/src/mnodePeer.c b/src/mnode/src/mnodePeer.c new file mode 100644 index 0000000000000000000000000000000000000000..d3699948f21de905d3e907835710f2d82a2172f1 --- /dev/null +++ b/src/mnode/src/mnodePeer.c @@ -0,0 +1,83 @@ +/* + * Copyright (c) 2019 TAOS Data, Inc. + * + * This program is free software: you can use, redistribute, and/or modify + * it under the terms of the GNU Affero General Public License, version 3 + * or later ("AGPL"), as published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ + +#define _DEFAULT_SOURCE +#include "os.h" +#include "taoserror.h" +#include "tsched.h" +#include "tsystem.h" +#include "tutil.h" +#include "tgrant.h" +#include "tbalance.h" +#include "tglobal.h" +#include "mnode.h" +#include "dnode.h" +#include "mnodeDef.h" +#include "mnodeInt.h" +#include "mnodeDb.h" +#include "mnodeMnode.h" +#include "mnodeProfile.h" +#include "mnodeShow.h" +#include "mnodeSdb.h" +#include "mnodeTable.h" +#include "mnodeVgroup.h" + +static int32_t (*tsMnodeProcessPeerMsgFp[TSDB_MSG_TYPE_MAX])(SMnodeMsg *); +static void (*tsMnodeProcessPeerRspFp[TSDB_MSG_TYPE_MAX])(SRpcMsg *); + +void mnodeAddPeerMsgHandle(uint8_t msgType, int32_t (*fp)(SMnodeMsg *mnodeMsg)) { + tsMnodeProcessPeerMsgFp[msgType] = fp; +} + +void mnodeAddPeerRspHandle(uint8_t msgType, void (*fp)(SRpcMsg *rpcMsg)) { + tsMnodeProcessPeerRspFp[msgType] = fp; +} + +int32_t mnodeProcessPeerReq(SMnodeMsg *pMsg) { + if (pMsg->rpcMsg.pCont == NULL) { + mError("%p, msg:%s in mpeer queue, content is null", pMsg->rpcMsg.ahandle, taosMsg[pMsg->rpcMsg.msgType]); + return TSDB_CODE_INVALID_MSG_LEN; + } + + if (!sdbIsMaster()) { + SMnodeRsp *rpcRsp = &pMsg->rpcRsp; + SRpcIpSet *ipSet = rpcMallocCont(sizeof(SRpcIpSet)); + mnodeGetMnodeIpSetForPeer(ipSet); + rpcRsp->rsp = ipSet; + rpcRsp->len = sizeof(SRpcIpSet); + + mTrace("%p, msg:%s in mpeer queue, will be redireced inUse:%d", pMsg->rpcMsg.ahandle, taosMsg[pMsg->rpcMsg.msgType], ipSet->inUse); + for (int32_t i = 0; i < ipSet->numOfIps; ++i) { + mTrace("mnode index:%d ip:%s:%d", i, ipSet->fqdn[i], htons(ipSet->port[i])); + } + + return TSDB_CODE_REDIRECT; + } + + if (tsMnodeProcessPeerMsgFp[pMsg->rpcMsg.msgType] == NULL) { + mError("%p, msg:%s in mpeer queue, not processed", pMsg->rpcMsg.ahandle, taosMsg[pMsg->rpcMsg.msgType]); + return TSDB_CODE_MSG_NOT_PROCESSED; + } + + return (*tsMnodeProcessPeerMsgFp[pMsg->rpcMsg.msgType])(pMsg); +} + +void mnodeProcessPeerRsp(SRpcMsg *pMsg) { + if (tsMnodeProcessPeerRspFp[pMsg->msgType]) { + (*tsMnodeProcessPeerRspFp[pMsg->msgType])(pMsg); + } else { + mError("%p, msg:%s is not processed", pMsg->ahandle, taosMsg[pMsg->msgType]); + } +} diff --git a/src/mnode/src/mgmtProfile.c b/src/mnode/src/mnodeProfile.c similarity index 66% rename from src/mnode/src/mgmtProfile.c rename to src/mnode/src/mnodeProfile.c index 6667bff0524e9cc1ce340ff2337d49c2a3613104..a37f5436c6ee70722dd131373f5ffbe3b0298564 100644 --- a/src/mnode/src/mgmtProfile.c +++ b/src/mnode/src/mnodeProfile.c @@ -18,23 +18,25 @@ #include "taosmsg.h" #include "taoserror.h" #include "tutil.h" -#include "mgmtDef.h" -#include "mgmtInt.h" -#include "mgmtAcct.h" -#include "mgmtDnode.h" -#include "mgmtDb.h" -#include "mgmtMnode.h" -#include "mgmtProfile.h" -#include "mgmtShell.h" -#include "mgmtTable.h" -#include "mgmtUser.h" -#include "mgmtVgroup.h" - -int32_t mgmtSaveQueryStreamList(SCMHeartBeatMsg *pHBMsg); - -int32_t mgmtKillQuery(char *qidstr, void *pConn); -int32_t mgmtKillStream(char *qidstr, void *pConn); -int32_t mgmtKillConnection(char *qidstr, void *pConn); +#include "mnode.h" +#include "mnodeDef.h" +#include "mnodeInt.h" +#include "mnodeAcct.h" +#include "mnodeDnode.h" +#include "mnodeDb.h" +#include "mnodeMnode.h" +#include "mnodeProfile.h" +#include "mnodeShow.h" +#include "mnodeTable.h" +#include "mnodeUser.h" +#include "mnodeVgroup.h" +#include "mnodeWrite.h" + +int32_t mnodeSaveQueryStreamList(SCMHeartBeatMsg *pHBMsg); + +int32_t mnodeKillQuery(char *qidstr, void *pConn); +int32_t mnodeKillStream(char *qidstr, void *pConn); +int32_t mnodeKillConnection(char *qidstr, void *pConn); typedef struct { char user[TSDB_TABLE_ID_LEN + 1]; @@ -98,7 +100,7 @@ int32_t mgmtSaveQueryStreamList(SCMHeartBeatMsg *pHBMsg) { return TSDB_CODE_SUCCESS; } -int32_t mgmtGetQueries(SShowObj *pShow, void *pConn) { +int32_t mnodeGetQueries(SShowObj *pShow, void *pConn) { // SAcctObj * pAcct = pConn->pAcct; // SQueryShow *pQueryShow; // @@ -145,7 +147,7 @@ int32_t mgmtGetQueries(SShowObj *pShow, void *pConn) { return 0; } -int32_t mgmtGetQueryMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) { +int32_t mnodeGetQueryMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) { int32_t cols = 0; SSchema *pSchema = pMeta->schema; @@ -190,11 +192,11 @@ int32_t mgmtGetQueryMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) { pShow->pIter = NULL; pShow->rowSize = pShow->offset[cols - 1] + pShow->bytes[cols - 1]; - mgmtGetQueries(pShow, pConn); + mnodeGetQueries(pShow, pConn); return 0; } -int32_t mgmtKillQuery(char *qidstr, void *pConn) { +int32_t mnodeKillQuery(char *qidstr, void *pConn) { // char *temp, *chr, idstr[64]; // strcpy(idstr, qidstr); // @@ -247,7 +249,7 @@ int32_t mgmtKillQuery(char *qidstr, void *pConn) { return TSDB_CODE_INVALID_QUERY_ID; } -int32_t mgmtRetrieveQueries(SShowObj *pShow, char *data, int32_t rows, void *pConn) { +int32_t mnodeRetrieveQueries(SShowObj *pShow, char *data, int32_t rows, void *pConn) { int32_t numOfRows = 0; char *pWrite; int32_t cols = 0; @@ -297,7 +299,7 @@ int32_t mgmtRetrieveQueries(SShowObj *pShow, char *data, int32_t rows, void *pCo return numOfRows; } -int32_t mgmtGetStreams(SShowObj *pShow, void *pConn) { +int32_t mnodeGetStreams(SShowObj *pShow, void *pConn) { // SAcctObj * pAcct = pConn->pAcct; // SStreamShow *pStreamShow; // @@ -344,125 +346,8 @@ int32_t mgmtGetStreams(SShowObj *pShow, void *pConn) { return 0; } -int32_t mgmtGetStreamMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) { - int32_t cols = 0; - SSchema *pSchema = pMeta->schema; - - pShow->bytes[cols] = TSDB_USER_LEN; - pSchema[cols].type = TSDB_DATA_TYPE_BINARY; - strcpy(pSchema[cols].name, "user"); - pSchema[cols].bytes = htons(pShow->bytes[cols]); - cols++; - - pShow->bytes[cols] = TSDB_IPv4ADDR_LEN + 14; - pSchema[cols].type = TSDB_DATA_TYPE_BINARY; - strcpy(pSchema[cols].name, "ip:port:id"); - pSchema[cols].bytes = htons(pShow->bytes[cols]); - cols++; - - pShow->bytes[cols] = 8; - pSchema[cols].type = TSDB_DATA_TYPE_TIMESTAMP; - strcpy(pSchema[cols].name, "created time"); - pSchema[cols].bytes = htons(pShow->bytes[cols]); - cols++; - - pShow->bytes[cols] = 8; - pSchema[cols].type = TSDB_DATA_TYPE_TIMESTAMP; - strcpy(pSchema[cols].name, "exec time"); - pSchema[cols].bytes = htons(pShow->bytes[cols]); - cols++; - - pShow->bytes[cols] = 8; - pSchema[cols].type = TSDB_DATA_TYPE_INT; - strcpy(pSchema[cols].name, "time(us)"); - pSchema[cols].bytes = htons(pShow->bytes[cols]); - cols++; - - pShow->bytes[cols] = TSDB_SHOW_SQL_LEN; - pSchema[cols].type = TSDB_DATA_TYPE_BINARY; - strcpy(pSchema[cols].name, "sql"); - pSchema[cols].bytes = htons(pShow->bytes[cols]); - cols++; - - pShow->bytes[cols] = 4; - pSchema[cols].type = TSDB_DATA_TYPE_INT; - strcpy(pSchema[cols].name, "cycles"); - pSchema[cols].bytes = htons(pShow->bytes[cols]); - cols++; - - pMeta->numOfColumns = htons(cols); - pShow->numOfColumns = cols; - - pShow->offset[0] = 0; - for (int32_t i = 1; i < cols; ++i) pShow->offset[i] = pShow->offset[i - 1] + pShow->bytes[i - 1]; - - pShow->numOfRows = 1000000; - pShow->pIter = NULL; - pShow->rowSize = pShow->offset[cols - 1] + pShow->bytes[cols - 1]; - - mgmtGetStreams(pShow, pConn); - return 0; -} - -int32_t mgmtRetrieveStreams(SShowObj *pShow, char *data, int32_t rows, void *pConn) { - int32_t numOfRows = 0; - char *pWrite; - int32_t cols = 0; - - SStreamShow *pStreamShow = (SStreamShow *)pShow->pIter; - - if (rows > pStreamShow->numOfStreams - pStreamShow->index) rows = pStreamShow->numOfStreams - pStreamShow->index; - - while (numOfRows < rows) { - SStreamDesc *pNode = pStreamShow->sdesc + pStreamShow->index; - SCDesc *pCDesc = pStreamShow->cdesc[pStreamShow->index]; - cols = 0; - - pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows; - strcpy(pWrite, pCDesc->user); - cols++; - - pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows; - uint32_t ip = pCDesc->ip; - sprintf(pWrite, "%d.%d.%d.%d:%hu:%d", ip & 0xFF, (ip >> 8) & 0xFF, (ip >> 16) & 0xFF, ip >> 24, htons(pCDesc->port), - pNode->streamId); - cols++; - - pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows; - *(int64_t *)pWrite = pNode->ctime; - cols++; - - pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows; - *(int64_t *)pWrite = pNode->stime; - cols++; - - pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows; - *(int64_t *)pWrite = pNode->useconds; - cols++; - - pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows; - strcpy(pWrite, pNode->sql); - cols++; - - pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows; - *(int32_t *)pWrite = pNode->num; - cols++; - - numOfRows++; - pStreamShow->index++; - } - - if (numOfRows == 0) { - tfree(pStreamShow->cdesc); - tfree(pStreamShow->connInfo); - tfree(pStreamShow); - } - pShow->numOfReads += numOfRows; - return numOfRows; -} - -int32_t mgmtKillStream(char *qidstr, void *pConn) { +int32_t mnodeKillStream(char *qidstr, void *pConn) { // char *temp, *chr, idstr[64]; // strcpy(idstr, qidstr); // @@ -515,7 +400,7 @@ int32_t mgmtKillStream(char *qidstr, void *pConn) { return TSDB_CODE_INVALID_STREAM_ID; } -int32_t mgmtKillConnection(char *qidstr, void *pConn) { +int32_t mnodeKillConnection(char *qidstr, void *pConn) { // void *pConn1 = NULL; // char * temp, *chr, idstr[64]; // strcpy(idstr, qidstr); @@ -562,7 +447,7 @@ int32_t mgmtKillConnection(char *qidstr, void *pConn) { } -int mgmtGetConns(SShowObj *pShow, void *pConn) { +int mnodeGetConns(SShowObj *pShow, void *pConn) { // SAcctObj * pAcct = pConn->pAcct; // SConnShow *pConnShow; // @@ -597,7 +482,7 @@ int mgmtGetConns(SShowObj *pShow, void *pConn) { return 0; } -int32_t mgmtGetConnsMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) { +int32_t mnodeGetConnsMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) { int32_t cols = 0; pShow->bytes[cols] = TSDB_TABLE_NAME_LEN; @@ -630,11 +515,11 @@ int32_t mgmtGetConnsMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) { pShow->pIter = NULL; pShow->rowSize = pShow->offset[cols - 1] + pShow->bytes[cols - 1]; - mgmtGetConns(pShow, pConn); + mnodeGetConns(pShow, pConn); return 0; } -int32_t mgmtRetrieveConns(SShowObj *pShow, char *data, int32_t rows, void *pConn) { +int32_t mnodeRetrieveConns(SShowObj *pShow, char *data, int32_t rows, void *pConn) { int32_t numOfRows = 0; char *pWrite; int32_t cols = 0; @@ -672,91 +557,92 @@ int32_t mgmtRetrieveConns(SShowObj *pShow, char *data, int32_t rows, void *pConn return numOfRows; } -void mgmtProcessKillQueryMsg(SQueuedMsg *pMsg) { - SRpcMsg rpcRsp = {.handle = pMsg->thandle, .pCont = NULL, .contLen = 0, .code = 0, .msgType = 0}; +int32_t mnodeProcessKillQueryMsg(SMnodeMsg *pMsg) { + // SRpcMsg rpcRsp = {.handle = pMsg->thandle, .pCont = NULL, .contLen = 0, .code = 0, .msgType = 0}; - SUserObj *pUser = mgmtGetUserFromConn(pMsg->thandle); - if (pUser == NULL) { - rpcRsp.code = TSDB_CODE_INVALID_USER; - rpcSendResponse(&rpcRsp); - return; - } - - SCMKillQueryMsg *pKill = pMsg->pCont; - int32_t code; - - if (!pUser->writeAuth) { - code = TSDB_CODE_NO_RIGHTS; - } else { - code = mgmtKillQuery(pKill->queryId, pMsg->thandle); - } - - rpcRsp.code = code; - rpcSendResponse(&rpcRsp); - mgmtDecUserRef(pUser); + // SUserObj *pUser = mnodeGetUserFromConn(pMsg->thandle); + // if (pUser == NULL) { + // rpcRsp.code = TSDB_CODE_INVALID_USER; + // rpcSendResponse(&rpcRsp); + // return; + // } + + // SCMKillQueryMsg *pKill = pMsg->pCont; + // int32_t code; + + // if (!pUser->writeAuth) { + // code = TSDB_CODE_NO_RIGHTS; + // } else { + // code = mgmtKillQuery(pKill->queryId, pMsg->thandle); + // } + + // rpcRsp.code = code; + // rpcSendResponse(&rpcRsp); + // mnodeDecUserRef(pUser); + return TSDB_CODE_SUCCESS; } -void mgmtProcessKillStreamMsg(SQueuedMsg *pMsg) { - SRpcMsg rpcRsp = {.handle = pMsg->thandle, .pCont = NULL, .contLen = 0, .code = 0, .msgType = 0}; +int32_t mnodeProcessKillStreamMsg(SMnodeMsg *pMsg) { + // SRpcMsg rpcRsp = {.handle = pMsg->thandle, .pCont = NULL, .contLen = 0, .code = 0, .msgType = 0}; - SUserObj *pUser = mgmtGetUserFromConn(pMsg->thandle); - if (pUser == NULL) { - rpcRsp.code = TSDB_CODE_INVALID_USER; - rpcSendResponse(&rpcRsp); - return; - } - - SCMKillStreamMsg *pKill = pMsg->pCont; - int32_t code; - - if (!pUser->writeAuth) { - code = TSDB_CODE_NO_RIGHTS; - } else { - code = mgmtKillStream(pKill->queryId, pMsg->thandle); - } - - rpcRsp.code = code; - rpcSendResponse(&rpcRsp); - mgmtDecUserRef(pUser); + // SUserObj *pUser = mnodeGetUserFromConn(pMsg->thandle); + // if (pUser == NULL) { + // rpcRsp.code = TSDB_CODE_INVALID_USER; + // rpcSendResponse(&rpcRsp); + // return; + // } + + // SCMKillStreamMsg *pKill = pMsg->pCont; + // int32_t code; + + // if (!pUser->writeAuth) { + // code = TSDB_CODE_NO_RIGHTS; + // } else { + // code = mgmtKillStream(pKill->queryId, pMsg->thandle); + // } + + // rpcRsp.code = code; + // rpcSendResponse(&rpcRsp); + // mnodeDecUserRef(pUser); + return TSDB_CODE_SUCCESS; } -void mgmtProcessKillConnectionMsg(SQueuedMsg *pMsg) { - SRpcMsg rpcRsp = {.handle = pMsg->thandle, .pCont = NULL, .contLen = 0, .code = 0, .msgType = 0}; +int32_t mnodeProcessKillConnectionMsg(SMnodeMsg *pMsg) { + // SRpcMsg rpcRsp = {.handle = pMsg->thandle, .pCont = NULL, .contLen = 0, .code = 0, .msgType = 0}; - SUserObj *pUser = mgmtGetUserFromConn(pMsg->thandle); - if (pUser == NULL) { - rpcRsp.code = TSDB_CODE_INVALID_USER; - rpcSendResponse(&rpcRsp); - return; - } - - SCMKillConnMsg *pKill = pMsg->pCont; - int32_t code; - - if (!pUser->writeAuth) { - code = TSDB_CODE_NO_RIGHTS; - } else { - code = mgmtKillConnection(pKill->queryId, pMsg->thandle); - } - - rpcRsp.code = code; - rpcSendResponse(&rpcRsp); - mgmtDecUserRef(pUser); + // SUserObj *pUser = mnodeGetUserFromConn(pMsg->thandle); + // if (pUser == NULL) { + // rpcRsp.code = TSDB_CODE_INVALID_USER; + // rpcSendResponse(&rpcRsp); + // return; + // } + + // SCMKillConnMsg *pKill = pMsg->pCont; + // int32_t code; + + // if (!pUser->writeAuth) { + // code = TSDB_CODE_NO_RIGHTS; + // } else { + // code = mgmtKillConnection(pKill->queryId, pMsg->thandle); + // } + + // rpcRsp.code = code; + // rpcSendResponse(&rpcRsp); + // mnodeDecUserRef(pUser); + return TSDB_CODE_SUCCESS; } -int32_t mgmtInitProfile() { - mgmtAddShellShowMetaHandle(TSDB_MGMT_TABLE_QUERIES, mgmtGetQueryMeta); - mgmtAddShellShowRetrieveHandle(TSDB_MGMT_TABLE_QUERIES, mgmtRetrieveQueries); - mgmtAddShellShowMetaHandle(TSDB_MGMT_TABLE_CONNS, mgmtGetConnsMeta); - mgmtAddShellShowRetrieveHandle(TSDB_MGMT_TABLE_CONNS, mgmtRetrieveConns); - mgmtAddShellShowMetaHandle(TSDB_MGMT_TABLE_STREAMS, mgmtGetStreamMeta); - mgmtAddShellShowRetrieveHandle(TSDB_MGMT_TABLE_STREAMS, mgmtRetrieveStreams); - mgmtAddShellMsgHandle(TSDB_MSG_TYPE_CM_KILL_QUERY, mgmtProcessKillQueryMsg); - mgmtAddShellMsgHandle(TSDB_MSG_TYPE_CM_KILL_STREAM, mgmtProcessKillStreamMsg); - mgmtAddShellMsgHandle(TSDB_MSG_TYPE_CM_KILL_CONN, mgmtProcessKillConnectionMsg); +int32_t mnodeInitProfile() { + // mnodeAddShowMetaHandle(TSDB_MGMT_TABLE_QUERIES, mnodeGetQueryMeta); + // mnodeAddShowRetrieveHandle(TSDB_MGMT_TABLE_QUERIES, mnodeRetrieveQueries); + // mnodeAddShowMetaHandle(TSDB_MGMT_TABLE_CONNS, mnodeGetConnsMeta); + // mnodeAddShowRetrieveHandle(TSDB_MGMT_TABLE_CONNS, mnodeRetrieveConns); + + // mnodeAddWriteMsgHandle(TSDB_MSG_TYPE_CM_KILL_QUERY, mnodeProcessKillQueryMsg); + // mnodeAddWriteMsgHandle(TSDB_MSG_TYPE_CM_KILL_STREAM, mnodeProcessKillStreamMsg); + // mnodeAddWriteMsgHandle(TSDB_MSG_TYPE_CM_KILL_CONN, mnodeProcessKillConnectionMsg); return 0; } -void mgmtCleanUpProfile() { -} +void mnodeCleanupProfile() {} diff --git a/src/mnode/src/mnodeRead.c b/src/mnode/src/mnodeRead.c new file mode 100644 index 0000000000000000000000000000000000000000..0ee917f2d1322a00f97ec5dc2866266c86d14d6c --- /dev/null +++ b/src/mnode/src/mnodeRead.c @@ -0,0 +1,77 @@ +/* + * Copyright (c) 2019 TAOS Data, Inc. + * + * This program is free software: you can use, redistribute, and/or modify + * it under the terms of the GNU Affero General Public License, version 3 + * or later ("AGPL"), as published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ + +#define _DEFAULT_SOURCE +#include "os.h" +#include "taosdef.h" +#include "tsched.h" +#include "tbalance.h" +#include "tgrant.h" +#include "ttimer.h" +#include "tglobal.h" +#include "mnode.h" +#include "dnode.h" +#include "mnodeDef.h" +#include "mnodeInt.h" +#include "mnodeAcct.h" +#include "mnodeDnode.h" +#include "mnodeMnode.h" +#include "mnodeDb.h" +#include "mnodeSdb.h" +#include "mnodeVgroup.h" +#include "mnodeUser.h" +#include "mnodeTable.h" +#include "mnodeShow.h" + +static int32_t (*tsMnodeProcessReadMsgFp[TSDB_MSG_TYPE_MAX])(SMnodeMsg *); + +void mnodeAddReadMsgHandle(uint8_t msgType, int32_t (*fp)(SMnodeMsg *pMsg)) { + tsMnodeProcessReadMsgFp[msgType] = fp; +} + +int32_t mnodeProcessRead(SMnodeMsg *pMsg) { + if (pMsg->rpcMsg.pCont == NULL) { + mError("%p, msg:%s in mread queue, content is null", pMsg->rpcMsg.ahandle, taosMsg[pMsg->rpcMsg.msgType]); + return TSDB_CODE_INVALID_MSG_LEN; + } + + if (!sdbIsMaster()) { + SMnodeRsp *rpcRsp = &pMsg->rpcRsp; + SRpcIpSet *ipSet = rpcMallocCont(sizeof(SRpcIpSet)); + mnodeGetMnodeIpSetForShell(ipSet); + rpcRsp->rsp = ipSet; + rpcRsp->len = sizeof(SRpcIpSet); + + mTrace("%p, msg:%s in mread queue, will be redireced, inUse:%d", pMsg->rpcMsg.ahandle, taosMsg[pMsg->rpcMsg.msgType], ipSet->inUse); + for (int32_t i = 0; i < ipSet->numOfIps; ++i) { + mTrace("mnode index:%d ip:%s:%d", i, ipSet->fqdn[i], htons(ipSet->port[i])); + } + + return TSDB_CODE_REDIRECT; + } + + if (tsMnodeProcessReadMsgFp[pMsg->rpcMsg.msgType] == NULL) { + mError("%p, msg:%s in mread queue, not processed", pMsg->rpcMsg.ahandle, taosMsg[pMsg->rpcMsg.msgType]); + return TSDB_CODE_MSG_NOT_PROCESSED; + } + + int32_t code = mnodeInitMsg(pMsg); + if (code != TSDB_CODE_SUCCESS) { + mError("%p, msg:%s in mread queue, not processed reason:%s", pMsg->rpcMsg.ahandle, taosMsg[pMsg->rpcMsg.msgType], tstrerror(code)); + return code; + } + + return (*tsMnodeProcessReadMsgFp[pMsg->rpcMsg.msgType])(pMsg); +} diff --git a/src/mnode/src/mgmtSdb.c b/src/mnode/src/mnodeSdb.c similarity index 94% rename from src/mnode/src/mgmtSdb.c rename to src/mnode/src/mnodeSdb.c index 237d2ca499296d00ffd50007430a57649767fa9f..6638fba53b2200e5c4688c57183512a0442bc289 100644 --- a/src/mnode/src/mgmtSdb.c +++ b/src/mnode/src/mnodeSdb.c @@ -17,7 +17,6 @@ #include "os.h" #include "taoserror.h" #include "hash.h" -#include "trpc.h" #include "tutil.h" #include "tbalance.h" #include "tqueue.h" @@ -25,11 +24,11 @@ #include "tsync.h" #include "tglobal.h" #include "dnode.h" -#include "mgmtDef.h" -#include "mgmtInt.h" -#include "mgmtMnode.h" -#include "mgmtDnode.h" -#include "mgmtSdb.h" +#include "mnodeDef.h" +#include "mnodeInt.h" +#include "mnodeMnode.h" +#include "mnodeDnode.h" +#include "mnodeSdb.h" typedef enum { SDB_ACTION_INSERT, @@ -187,18 +186,18 @@ void sdbUpdateMnodeRoles() { SNodesRole roles = {0}; syncGetNodesRole(tsSdbObj.sync, &roles); - sdbPrint("update mnodes:%d sync roles", tsSdbObj.cfg.replica); + sdbPrint("update mnodes sync roles, total:%d", tsSdbObj.cfg.replica); for (int32_t i = 0; i < tsSdbObj.cfg.replica; ++i) { - SMnodeObj *pMnode = mgmtGetMnode(roles.nodeId[i]); + SMnodeObj *pMnode = mnodeGetMnode(roles.nodeId[i]); if (pMnode != NULL) { pMnode->role = roles.role[i]; - sdbPrint("mnode:%d, role:%s", pMnode->mnodeId, mgmtGetMnodeRoleStr(pMnode->role)); + sdbPrint("mnode:%d, role:%s", pMnode->mnodeId, mnodeGetMnodeRoleStr(pMnode->role)); if (pMnode->mnodeId == dnodeGetDnodeId()) tsSdbObj.role = pMnode->role; - mgmtDecMnodeRef(pMnode); + mnodeDecMnodeRef(pMnode); } } - mgmtUpdateMnodeIpSet(); + mnodeUpdateMnodeIpSet(); } static uint32_t sdbGetFileInfo(void *ahandle, char *name, uint32_t *index, int32_t *size, uint64_t *fversion) { @@ -211,7 +210,7 @@ static int sdbGetWalInfo(void *ahandle, char *name, uint32_t *index) { } static void sdbNotifyRole(void *ahandle, int8_t role) { - sdbPrint("mnode role changed from %s to %s", mgmtGetMnodeRoleStr(tsSdbObj.role), mgmtGetMnodeRoleStr(role)); + sdbPrint("mnode role changed from %s to %s", mnodeGetMnodeRoleStr(tsSdbObj.role), mnodeGetMnodeRoleStr(role)); if (role == TAOS_SYNC_ROLE_MASTER && tsSdbObj.role != TAOS_SYNC_ROLE_MASTER) { balanceReset(); @@ -256,20 +255,20 @@ void sdbUpdateSync() { void *pIter = NULL; while (1) { SMnodeObj *pMnode = NULL; - pIter = mgmtGetNextMnode(pIter, &pMnode); + pIter = mnodeGetNextMnode(pIter, &pMnode); if (pMnode == NULL) break; syncCfg.nodeInfo[index].nodeId = pMnode->mnodeId; - SDnodeObj *pDnode = mgmtGetDnode(pMnode->mnodeId); + SDnodeObj *pDnode = mnodeGetDnode(pMnode->mnodeId); if (pDnode != NULL) { syncCfg.nodeInfo[index].nodePort = pDnode->dnodePort + TSDB_PORT_SYNC; strcpy(syncCfg.nodeInfo[index].nodeFqdn, pDnode->dnodeEp); index++; } - mgmtDecDnodeRef(pDnode); - mgmtDecMnodeRef(pMnode); + mnodeDecDnodeRef(pDnode); + mnodeDecMnodeRef(pMnode); } sdbFreeIter(pIter); } @@ -324,7 +323,7 @@ int32_t sdbInit() { sdbRestoreTables(); - if (mgmtGetMnodesNum() == 1) { + if (mnodeGetMnodesNum() == 1) { tsSdbObj.role = TAOS_SYNC_ROLE_MASTER; } @@ -359,8 +358,8 @@ void sdbIncRef(void *handle, void *pObj) { SSdbTable *pTable = handle; int32_t * pRefCount = (int32_t *)(pObj + pTable->refCountPos); atomic_add_fetch_32(pRefCount, 1); - if (0 && (pTable->tableId == SDB_TABLE_MNODE || pTable->tableId == SDB_TABLE_DNODE)) { - sdbTrace("add ref to table:%s record:%s:%d", pTable->tableName, sdbGetKeyStrFromObj(pTable, pObj), *pRefCount); + if (0 && (pTable->tableId == SDB_TABLE_CTABLE || pTable->tableId == SDB_TABLE_DB)) { + sdbTrace("add ref to table:%s record:%p:%s:%d", pTable->tableName, pObj, sdbGetKeyStrFromObj(pTable, pObj), *pRefCount); } } @@ -370,13 +369,13 @@ void sdbDecRef(void *handle, void *pObj) { SSdbTable *pTable = handle; int32_t * pRefCount = (int32_t *)(pObj + pTable->refCountPos); int32_t refCount = atomic_sub_fetch_32(pRefCount, 1); - if (0 && (pTable->tableId == SDB_TABLE_MNODE || pTable->tableId == SDB_TABLE_DNODE)) { - sdbTrace("def ref of table:%s record:%s:%d", pTable->tableName, sdbGetKeyStrFromObj(pTable, pObj), *pRefCount); + if (0 && (pTable->tableId == SDB_TABLE_CTABLE || pTable->tableId == SDB_TABLE_DB)) { + sdbTrace("def ref of table:%s record:%p:%s:%d", pTable->tableName, pObj, sdbGetKeyStrFromObj(pTable, pObj), *pRefCount); } int8_t *updateEnd = pObj + pTable->refCountPos - 1; if (refCount <= 0 && *updateEnd) { - sdbTrace("table:%s, record:%s:%d is destroyed", pTable->tableName, sdbGetKeyStrFromObj(pTable, pObj), *pRefCount); + sdbTrace("table:%s, record:%p:%s:%d is destroyed", pTable->tableName, pObj, sdbGetKeyStrFromObj(pTable, pObj), *pRefCount); SSdbOper oper = {.pObj = pObj}; (*pTable->destroyFp)(&oper); } diff --git a/src/mnode/src/mnodeShow.c b/src/mnode/src/mnodeShow.c new file mode 100644 index 0000000000000000000000000000000000000000..20616bfbcdae827f6bb2ca31273d349650f704f2 --- /dev/null +++ b/src/mnode/src/mnodeShow.c @@ -0,0 +1,360 @@ +/* + * Copyright (c) 2019 TAOS Data, Inc. + * + * This program is free software: you can use, redistribute, and/or modify + * it under the terms of the GNU Affero General Public License, version 3 + * or later ("AGPL"), as published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ + +#define _DEFAULT_SOURCE +#include "os.h" +#include "taosmsg.h" +#include "taoserror.h" +#include "tsched.h" +#include "tutil.h" +#include "ttimer.h" +#include "tgrant.h" +#include "tglobal.h" +#include "tcache.h" +#include "mnode.h" +#include "dnode.h" +#include "mnodeDef.h" +#include "mnodeInt.h" +#include "mnodeAcct.h" +#include "mnodeDb.h" +#include "mnodeDnode.h" +#include "mnodeMnode.h" +#include "mnodeProfile.h" +#include "mnodeSdb.h" +#include "mnodeShow.h" +#include "mnodeTable.h" +#include "mnodeUser.h" +#include "mnodeVgroup.h" +#include "mnodeWrite.h" +#include "mnodeRead.h" + +static int32_t mnodeProcessShowMsg(SMnodeMsg *mnodeMsg); +static int32_t mnodeProcessRetrieveMsg(SMnodeMsg *mnodeMsg); +static int32_t mnodeProcessHeartBeatMsg(SMnodeMsg *mnodeMsg); +static int32_t mnodeProcessConnectMsg(SMnodeMsg *mnodeMsg); +static int32_t mnodeProcessUseMsg(SMnodeMsg *mnodeMsg); + +static void mnodeFreeShowObj(void *data); +static bool mnodeAccquireShowObj(SShowObj *pShow); +static bool mnodeCheckShowFinished(SShowObj *pShow); +static void *mnodePutShowObj(SShowObj *pShow, int32_t size); +static void mnodeReleaseShowObj(void *pShow, bool forceRemove); + +extern void *tsMnodeTmr; +static void *tsMnodeShowCache = NULL; +static int32_t tsShowObjIndex = 0; +static SShowMetaFp tsMnodeShowMetaFp[TSDB_MGMT_TABLE_MAX] = {0}; +static SShowRetrieveFp tsMnodeShowRetrieveFp[TSDB_MGMT_TABLE_MAX] = {0}; + +int32_t mnodeInitShow() { + mnodeAddReadMsgHandle(TSDB_MSG_TYPE_CM_SHOW, mnodeProcessShowMsg); + mnodeAddReadMsgHandle(TSDB_MSG_TYPE_CM_RETRIEVE, mnodeProcessRetrieveMsg); + mnodeAddReadMsgHandle(TSDB_MSG_TYPE_CM_HEARTBEAT, mnodeProcessHeartBeatMsg); + mnodeAddReadMsgHandle(TSDB_MSG_TYPE_CM_CONNECT, mnodeProcessConnectMsg); + mnodeAddReadMsgHandle(TSDB_MSG_TYPE_CM_USE_DB, mnodeProcessUseMsg); + + tsMnodeShowCache = taosCacheInitWithCb(tsMnodeTmr, 10, mnodeFreeShowObj); + return 0; +} + +void mnodeCleanUpShow() { + if (tsMnodeShowCache != NULL) { + mPrint("show cache is cleanup"); + taosCacheCleanup(tsMnodeShowCache); + tsMnodeShowCache = NULL; + } +} + +void mnodeAddShowMetaHandle(uint8_t showType, SShowMetaFp fp) { + tsMnodeShowMetaFp[showType] = fp; +} + +void mnodeAddShowRetrieveHandle(uint8_t msgType, SShowRetrieveFp fp) { + tsMnodeShowRetrieveFp[msgType] = fp; +} + +static char *mnodeGetShowType(int32_t showType) { + switch (showType) { + case TSDB_MGMT_TABLE_ACCT: return "show accounts"; + case TSDB_MGMT_TABLE_USER: return "show users"; + case TSDB_MGMT_TABLE_DB: return "show databases"; + case TSDB_MGMT_TABLE_TABLE: return "show tables"; + case TSDB_MGMT_TABLE_DNODE: return "show dnodes"; + case TSDB_MGMT_TABLE_MNODE: return "show mnodes"; + case TSDB_MGMT_TABLE_VGROUP: return "show vgroups"; + case TSDB_MGMT_TABLE_METRIC: return "show stables"; + case TSDB_MGMT_TABLE_MODULE: return "show modules"; + case TSDB_MGMT_TABLE_QUERIES: return "show queries"; + case TSDB_MGMT_TABLE_STREAMS: return "show streams"; + case TSDB_MGMT_TABLE_CONFIGS: return "show configs"; + case TSDB_MGMT_TABLE_CONNS: return "show connections"; + case TSDB_MGMT_TABLE_SCORES: return "show scores"; + case TSDB_MGMT_TABLE_GRANTS: return "show grants"; + case TSDB_MGMT_TABLE_VNODES: return "show vnodes"; + default: return "undefined"; + } +} + +static int32_t mnodeProcessShowMsg(SMnodeMsg *pMsg) { + SCMShowMsg *pShowMsg = pMsg->rpcMsg.pCont; + if (pShowMsg->type >= TSDB_MGMT_TABLE_MAX) { + return TSDB_CODE_INVALID_MSG_TYPE; + } + + if (!tsMnodeShowMetaFp[pShowMsg->type] || !tsMnodeShowRetrieveFp[pShowMsg->type]) { + mError("show type:%s is not support", mnodeGetShowType(pShowMsg->type)); + return TSDB_CODE_OPS_NOT_SUPPORT; + } + + int32_t showObjSize = sizeof(SShowObj) + htons(pShowMsg->payloadLen); + SShowObj *pShow = (SShowObj *) calloc(1, showObjSize); + pShow->type = pShowMsg->type; + pShow->payloadLen = htons(pShowMsg->payloadLen); + strcpy(pShow->db, pShowMsg->db); + memcpy(pShow->payload, pShowMsg->payload, pShow->payloadLen); + + pShow = mnodePutShowObj(pShow, showObjSize); + if (pShow == NULL) { + return TSDB_CODE_SERV_OUT_OF_MEMORY; + } + + int32_t size = sizeof(SCMShowRsp) + sizeof(SSchema) * TSDB_MAX_COLUMNS + TSDB_EXTRA_PAYLOAD_SIZE; + SCMShowRsp *pShowRsp = rpcMallocCont(size); + if (pShowRsp == NULL) { + mnodeReleaseShowObj(pShow, true); + return TSDB_CODE_SERV_OUT_OF_MEMORY; + } + pShowRsp->qhandle = htobe64((uint64_t) pShow); + + mTrace("%p, show type:%s, start to get meta", pShow, mnodeGetShowType(pShowMsg->type)); + int32_t code = (*tsMnodeShowMetaFp[pShowMsg->type])(&pShowRsp->tableMeta, pShow, pMsg->rpcMsg.handle); + if (code == 0) { + pMsg->rpcRsp.rsp = pShowRsp; + pMsg->rpcRsp.len = sizeof(SCMShowRsp) + sizeof(SSchema) * pShow->numOfColumns; + mnodeReleaseShowObj(pShow, false); + return TSDB_CODE_SUCCESS; + } else { + mError("%p, show type:%s, failed to get meta, reason:%s", pShow, mnodeGetShowType(pShowMsg->type), tstrerror(code)); + rpcFreeCont(pShowRsp); + mnodeReleaseShowObj(pShow, true); + return code; + } +} + +static int32_t mnodeProcessRetrieveMsg(SMnodeMsg *pMsg) { + int32_t rowsToRead = 0; + int32_t size = 0; + int32_t rowsRead = 0; + SRetrieveTableMsg *pRetrieve = pMsg->rpcMsg.pCont; + pRetrieve->qhandle = htobe64(pRetrieve->qhandle); + + SShowObj *pShow = (SShowObj *)pRetrieve->qhandle; + mTrace("%p, show type:%s, retrieve data", pShow, mnodeGetShowType(pShow->type)); + + /* + * in case of server restart, apps may hold qhandle created by server before + * restart, which is actually invalid, therefore, signature check is required. + */ + if (!mnodeAccquireShowObj(pShow)) { + mError("%p, show is invalid", pShow); + return TSDB_CODE_INVALID_QHANDLE; + } + + if (mnodeCheckShowFinished(pShow)) { + mTrace("%p, show is already read finished, numOfReads:%d numOfRows:%d", pShow, pShow->numOfReads, pShow->numOfRows); + pShow->numOfReads = pShow->numOfRows; + } + + if ((pRetrieve->free & TSDB_QUERY_TYPE_FREE_RESOURCE) != TSDB_QUERY_TYPE_FREE_RESOURCE) { + rowsToRead = pShow->numOfRows - pShow->numOfReads; + } + + /* return no more than 100 meters in one round trip */ + if (rowsToRead > 100) rowsToRead = 100; + + /* + * the actual number of table may be larger than the value of pShow->numOfRows, if a query is + * issued during a continuous create table operation. Therefore, rowToRead may be less than 0. + */ + if (rowsToRead < 0) rowsToRead = 0; + size = pShow->rowSize * rowsToRead; + + size += 100; + SRetrieveTableRsp *pRsp = rpcMallocCont(size); + + // if free flag is set, client wants to clean the resources + if ((pRetrieve->free & TSDB_QUERY_TYPE_FREE_RESOURCE) != TSDB_QUERY_TYPE_FREE_RESOURCE) + rowsRead = (*tsMnodeShowRetrieveFp[pShow->type])(pShow, pRsp->data, rowsToRead, pMsg->rpcMsg.handle); + + if (rowsRead < 0) { + rpcFreeCont(pRsp); + mnodeReleaseShowObj(pShow, false); + assert(false); + return TSDB_CODE_ACTION_IN_PROGRESS; + } + + pRsp->numOfRows = htonl(rowsRead); + pRsp->precision = htonl(TSDB_TIME_PRECISION_MILLI); // millisecond time precision + + pMsg->rpcRsp.rsp = pRsp; + pMsg->rpcRsp.len = size; + + if (rowsToRead == 0 || rowsRead == rowsToRead) { + pRsp->completed = 1; + mnodeReleaseShowObj(pShow, true); + } else { + mnodeReleaseShowObj(pShow, false); + } + + return TSDB_CODE_SUCCESS; +} + +static int32_t mnodeProcessHeartBeatMsg(SMnodeMsg *pMsg) { + SCMHeartBeatRsp *pHBRsp = (SCMHeartBeatRsp *) rpcMallocCont(sizeof(SCMHeartBeatRsp)); + if (pHBRsp == NULL) { + return TSDB_CODE_SERV_OUT_OF_MEMORY; + } + + pHBRsp->onlineDnodes = htonl(mnodeGetOnlinDnodesNum()); + pHBRsp->totalDnodes = htonl(mnodeGetDnodesNum()); + mnodeGetMnodeIpSetForShell(&pHBRsp->ipList); + + /* + * TODO + * Dispose kill stream or kill query message + */ + pHBRsp->queryId = 0; + pHBRsp->streamId = 0; + pHBRsp->killConnection = 0; + + pMsg->rpcRsp.rsp = pHBRsp; + pMsg->rpcRsp.len = sizeof(SCMHeartBeatRsp); + + return TSDB_CODE_SUCCESS; +} + +static int32_t mnodeProcessConnectMsg(SMnodeMsg *pMsg) { + SCMConnectMsg *pConnectMsg = pMsg->rpcMsg.pCont; + int32_t code = TSDB_CODE_SUCCESS; + + SRpcConnInfo connInfo; + if (rpcGetConnInfo(pMsg->rpcMsg.handle, &connInfo) != 0) { + mError("thandle:%p is already released while process connect msg", pMsg->rpcMsg.handle); + code = TSDB_CODE_INVALID_MSG_CONTENT; + goto connect_over; + } + + code = taosCheckVersion(pConnectMsg->clientVersion, version, 3); + if (code != TSDB_CODE_SUCCESS) { + goto connect_over; + } + + SUserObj *pUser = pMsg->pUser; + SAcctObj *pAcct = pUser->pAcct; + + if (pConnectMsg->db[0]) { + char dbName[TSDB_TABLE_ID_LEN * 3] = {0}; + sprintf(dbName, "%x%s%s", pAcct->acctId, TS_PATH_DELIMITER, pConnectMsg->db); + SDbObj *pDb = mnodeGetDb(dbName); + if (pDb == NULL) { + code = TSDB_CODE_INVALID_DB; + goto connect_over; + } + mnodeDecDbRef(pDb); + } + + SCMConnectRsp *pConnectRsp = rpcMallocCont(sizeof(SCMConnectRsp)); + if (pConnectRsp == NULL) { + code = TSDB_CODE_SERV_OUT_OF_MEMORY; + goto connect_over; + } + + sprintf(pConnectRsp->acctId, "%x", pAcct->acctId); + strcpy(pConnectRsp->serverVersion, version); + pConnectRsp->writeAuth = pUser->writeAuth; + pConnectRsp->superAuth = pUser->superAuth; + + mnodeGetMnodeIpSetForShell(&pConnectRsp->ipList); + +connect_over: + if (code != TSDB_CODE_SUCCESS) { + mLError("user:%s login from %s, result:%s", connInfo.user, taosIpStr(connInfo.clientIp), tstrerror(code)); + } else { + mLPrint("user:%s login from %s, result:%s", connInfo.user, taosIpStr(connInfo.clientIp), tstrerror(code)); + pMsg->rpcRsp.rsp = pConnectRsp; + pMsg->rpcRsp.len = sizeof(SCMConnectRsp); + } + + return code; +} + +static int32_t mnodeProcessUseMsg(SMnodeMsg *pMsg) { + SCMUseDbMsg *pUseDbMsg = pMsg->rpcMsg.pCont; + + int32_t code = TSDB_CODE_SUCCESS; + if (pMsg->pDb == NULL) pMsg->pDb = mnodeGetDb(pUseDbMsg->db); + if (pMsg->pDb == NULL) { + code = TSDB_CODE_INVALID_DB; + } + + return code; +} + +static bool mnodeCheckShowFinished(SShowObj *pShow) { + if (pShow->pIter == NULL && pShow->numOfReads != 0) { + return true; + } + return false; +} + +static bool mnodeAccquireShowObj(SShowObj *pShow) { + char key[10]; + sprintf(key, "%d", pShow->index); + + SShowObj *pSaved = taosCacheAcquireByName(tsMnodeShowCache, key); + if (pSaved == pShow) { + mTrace("%p, show is accquired from cache", pShow); + return true; + } else { + return false; + } +} + +static void *mnodePutShowObj(SShowObj *pShow, int32_t size) { + if (tsMnodeShowCache != NULL) { + char key[10]; + pShow->index = atomic_add_fetch_32(&tsShowObjIndex, 1); + sprintf(key, "%d", pShow->index); + + SShowObj *newQhandle = taosCachePut(tsMnodeShowCache, key, pShow, size, 60); + free(pShow); + + mTrace("%p, show is put into cache", newQhandle); + return newQhandle; + } + + return NULL; +} + +static void mnodeFreeShowObj(void *data) { + SShowObj *pShow = data; + sdbFreeIter(pShow->pIter); + mTrace("%p, show is destroyed", pShow); +} + +static void mnodeReleaseShowObj(void *pShow, bool forceRemove) { + mTrace("%p, show is released, force:%s", pShow, forceRemove ? "true" : "false"); + taosCacheRelease(tsMnodeShowCache, &pShow, forceRemove); +} diff --git a/src/mnode/src/mgmtTable.c b/src/mnode/src/mnodeTable.c similarity index 64% rename from src/mnode/src/mgmtTable.c rename to src/mnode/src/mnodeTable.c index 53fbd64f87e9c50fcb87deb15137c206412f1e35..84a1f659d58d6994940de5b568932403385f004c 100644 --- a/src/mnode/src/mgmtTable.c +++ b/src/mnode/src/mnodeTable.c @@ -24,105 +24,111 @@ #include "tname.h" #include "tidpool.h" #include "tglobal.h" -#include "hash.h" -#include "dnode.h" -#include "mgmtDef.h" -#include "mgmtInt.h" -#include "mgmtAcct.h" -#include "mgmtDb.h" -#include "mgmtDnode.h" -#include "tgrant.h" -#include "mgmtMnode.h" -#include "mgmtProfile.h" -#include "mgmtSdb.h" -#include "mgmtShell.h" -#include "mgmtTable.h" -#include "mgmtUser.h" -#include "mgmtVgroup.h" #include "tcompare.h" #include "tdataformat.h" +#include "tgrant.h" +#include "hash.h" +#include "mnode.h" +#include "dnode.h" +#include "mnodeDef.h" +#include "mnodeInt.h" +#include "mnodeAcct.h" +#include "mnodeDb.h" +#include "mnodeDnode.h" +#include "mnodeMnode.h" +#include "mnodeProfile.h" +#include "mnodeSdb.h" +#include "mnodeShow.h" +#include "mnodeTable.h" +#include "mnodeUser.h" +#include "mnodeVgroup.h" +#include "mnodeWrite.h" +#include "mnodeRead.h" +#include "mnodePeer.h" static void * tsChildTableSdb; static void * tsSuperTableSdb; static int32_t tsChildTableUpdateSize; static int32_t tsSuperTableUpdateSize; -static void * mgmtGetChildTable(char *tableId); -static void * mgmtGetSuperTable(char *tableId); -static void * mgmtGetSuperTableByUid(uint64_t uid); -static void mgmtDropAllChildTablesInStable(SSuperTableObj *pStable); -static void mgmtAddTableIntoStable(SSuperTableObj *pStable, SChildTableObj *pCtable); -static void mgmtRemoveTableFromStable(SSuperTableObj *pStable, SChildTableObj *pCtable); - -static int32_t mgmtGetShowTableMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn); -static int32_t mgmtRetrieveShowTables(SShowObj *pShow, char *data, int32_t rows, void *pConn); -static int32_t mgmtRetrieveShowSuperTables(SShowObj *pShow, char *data, int32_t rows, void *pConn); -static int32_t mgmtGetShowSuperTableMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn); - -static void mgmtProcessCreateTableMsg(SQueuedMsg *queueMsg); -static void mgmtProcessCreateSuperTableMsg(SQueuedMsg *pMsg); -static void mgmtProcessCreateChildTableMsg(SQueuedMsg *pMsg); -static void mgmtProcessCreateChildTableRsp(SRpcMsg *rpcMsg); - -static void mgmtProcessDropTableMsg(SQueuedMsg *queueMsg); -static void mgmtProcessDropSuperTableMsg(SQueuedMsg *pMsg); -static void mgmtProcessDropSuperTableRsp(SRpcMsg *rpcMsg); -static void mgmtProcessDropChildTableMsg(SQueuedMsg *pMsg); -static void mgmtProcessDropChildTableRsp(SRpcMsg *rpcMsg); - -static void mgmtProcessSuperTableVgroupMsg(SQueuedMsg *queueMsg); -static void mgmtProcessMultiTableMetaMsg(SQueuedMsg *queueMsg); -static void mgmtProcessTableCfgMsg(SRpcMsg *rpcMsg); - -static void mgmtProcessTableMetaMsg(SQueuedMsg *queueMsg); -static void mgmtGetSuperTableMeta(SQueuedMsg *pMsg); -static void mgmtGetChildTableMeta(SQueuedMsg *pMsg); -static void mgmtAutoCreateChildTable(SQueuedMsg *pMsg); - -static void mgmtProcessAlterTableMsg(SQueuedMsg *queueMsg); -static void mgmtProcessAlterTableRsp(SRpcMsg *rpcMsg); - -static int32_t mgmtFindSuperTableColumnIndex(SSuperTableObj *pStable, char *colName); - -static void mgmtDestroyChildTable(SChildTableObj *pTable) { +static void * mnodeGetChildTable(char *tableId); +static void * mnodeGetSuperTable(char *tableId); +static void * mnodeGetSuperTableByUid(uint64_t uid); +static void mnodeDropAllChildTablesInStable(SSuperTableObj *pStable); +static void mnodeAddTableIntoStable(SSuperTableObj *pStable, SChildTableObj *pCtable); +static void mnodeRemoveTableFromStable(SSuperTableObj *pStable, SChildTableObj *pCtable); + +static int32_t mnodeGetShowTableMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn); +static int32_t mnodeRetrieveShowTables(SShowObj *pShow, char *data, int32_t rows, void *pConn); +static int32_t mnodeGetShowSuperTableMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn); +static int32_t mnodeRetrieveShowSuperTables(SShowObj *pShow, char *data, int32_t rows, void *pConn); +static int32_t mnodeGetStreamMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn); +static int32_t mnodeRetrieveStreams(SShowObj *pShow, char *data, int32_t rows, void *pConn); + +static int32_t mnodeProcessCreateTableMsg(SMnodeMsg *mnodeMsg); +static int32_t mnodeProcessCreateSuperTableMsg(SMnodeMsg *pMsg); +static int32_t mnodeProcessCreateChildTableMsg(SMnodeMsg *pMsg); +static void mnodeProcessCreateChildTableRsp(SRpcMsg *rpcMsg); + +static int32_t mnodeProcessDropTableMsg(SMnodeMsg *mnodeMsg); +static int32_t mnodeProcessDropSuperTableMsg(SMnodeMsg *pMsg); +static void mnodeProcessDropSuperTableRsp(SRpcMsg *rpcMsg); +static int32_t mnodeProcessDropChildTableMsg(SMnodeMsg *pMsg); +static void mnodeProcessDropChildTableRsp(SRpcMsg *rpcMsg); + +static int32_t mnodeProcessSuperTableVgroupMsg(SMnodeMsg *mnodeMsg); +static int32_t mnodeProcessMultiTableMetaMsg(SMnodeMsg *mnodeMsg); +static int32_t mnodeProcessTableCfgMsg(SMnodeMsg *mnodeMsg); + +static int32_t mnodeProcessTableMetaMsg(SMnodeMsg *mnodeMsg); +static int32_t mnodeGetSuperTableMeta(SMnodeMsg *pMsg); +static int32_t mnodeGetChildTableMeta(SMnodeMsg *pMsg); +static int32_t mnodeAutoCreateChildTable(SMnodeMsg *pMsg); + +static int32_t mnodeProcessAlterTableMsg(SMnodeMsg *mnodeMsg); +static void mnodeProcessAlterTableRsp(SRpcMsg *rpcMsg); + +static int32_t mnodeFindSuperTableColumnIndex(SSuperTableObj *pStable, char *colName); + +static void mnodeDestroyChildTable(SChildTableObj *pTable) { tfree(pTable->info.tableId); tfree(pTable->schema); tfree(pTable->sql); tfree(pTable); } -static int32_t mgmtChildTableActionDestroy(SSdbOper *pOper) { - mgmtDestroyChildTable(pOper->pObj); +static int32_t mnodeChildTableActionDestroy(SSdbOper *pOper) { + mnodeDestroyChildTable(pOper->pObj); return TSDB_CODE_SUCCESS; } -static int32_t mgmtChildTableActionInsert(SSdbOper *pOper) { +static int32_t mnodeChildTableActionInsert(SSdbOper *pOper) { SChildTableObj *pTable = pOper->pObj; - SVgObj *pVgroup = mgmtGetVgroup(pTable->vgId); + SVgObj *pVgroup = mnodeGetVgroup(pTable->vgId); if (pVgroup == NULL) { mError("ctable:%s, not in vgId:%d", pTable->info.tableId, pTable->vgId); return TSDB_CODE_INVALID_VGROUP_ID; } - mgmtDecVgroupRef(pVgroup); + mnodeDecVgroupRef(pVgroup); - SDbObj *pDb = mgmtGetDb(pVgroup->dbName); + SDbObj *pDb = mnodeGetDb(pVgroup->dbName); if (pDb == NULL) { mError("ctable:%s, vgId:%d not in db:%s", pTable->info.tableId, pVgroup->vgId, pVgroup->dbName); return TSDB_CODE_INVALID_DB; } - mgmtDecDbRef(pDb); + mnodeDecDbRef(pDb); - SAcctObj *pAcct = mgmtGetAcct(pDb->acct); + SAcctObj *pAcct = mnodeGetAcct(pDb->acct); if (pAcct == NULL) { mError("ctable:%s, acct:%s not exists", pTable->info.tableId, pDb->acct); return TSDB_CODE_INVALID_ACCT; } - mgmtDecAcctRef(pAcct); + mnodeDecAcctRef(pAcct); if (pTable->info.type == TSDB_CHILD_TABLE) { // add ref - pTable->superTable = mgmtGetSuperTableByUid(pTable->suid); - mgmtAddTableIntoStable(pTable->superTable, pTable); + pTable->superTable = mnodeGetSuperTableByUid(pTable->suid); + mnodeAddTableIntoStable(pTable->superTable, pTable); grantAdd(TSDB_GRANT_TIMESERIES, pTable->superTable->numOfColumns - 1); pAcct->acctInfo.numOfTimeSeries += (pTable->superTable->numOfColumns - 1); } else { @@ -130,56 +136,49 @@ static int32_t mgmtChildTableActionInsert(SSdbOper *pOper) { pAcct->acctInfo.numOfTimeSeries += (pTable->numOfColumns - 1); } - mgmtAddTableIntoDb(pDb); - mgmtAddTableIntoVgroup(pVgroup, pTable); + mnodeAddTableIntoDb(pDb); + mnodeAddTableIntoVgroup(pVgroup, pTable); return TSDB_CODE_SUCCESS; } -static int32_t mgmtChildTableActionDelete(SSdbOper *pOper) { +static int32_t mnodeChildTableActionDelete(SSdbOper *pOper) { SChildTableObj *pTable = pOper->pObj; if (pTable->vgId == 0) { return TSDB_CODE_INVALID_VGROUP_ID; } - SVgObj *pVgroup = mgmtGetVgroup(pTable->vgId); - if (pVgroup == NULL) { - return TSDB_CODE_INVALID_VGROUP_ID; - } - mgmtDecVgroupRef(pVgroup); - - SDbObj *pDb = mgmtGetDb(pVgroup->dbName); - if (pDb == NULL) { - mError("ctable:%s, vgId:%d not in DB:%s", pTable->info.tableId, pVgroup->vgId, pVgroup->dbName); - return TSDB_CODE_INVALID_DB; - } - mgmtDecDbRef(pDb); - - SAcctObj *pAcct = mgmtGetAcct(pDb->acct); - if (pAcct == NULL) { - mError("ctable:%s, acct:%s not exists", pTable->info.tableId, pDb->acct); - return TSDB_CODE_INVALID_ACCT; - } - mgmtDecAcctRef(pAcct); + SVgObj *pVgroup = NULL; + SDbObj *pDb = NULL; + SAcctObj *pAcct = NULL; + + pVgroup = mnodeGetVgroup(pTable->vgId); + if (pVgroup != NULL) pDb = mnodeGetDb(pVgroup->dbName); + if (pDb != NULL) pAcct = mnodeGetAcct(pDb->acct); if (pTable->info.type == TSDB_CHILD_TABLE) { grantRestore(TSDB_GRANT_TIMESERIES, pTable->superTable->numOfColumns - 1); - pAcct->acctInfo.numOfTimeSeries -= (pTable->superTable->numOfColumns - 1); - mgmtRemoveTableFromStable(pTable->superTable, pTable); - mgmtDecTableRef(pTable->superTable); + if (pAcct != NULL) pAcct->acctInfo.numOfTimeSeries -= (pTable->superTable->numOfColumns - 1); + mnodeRemoveTableFromStable(pTable->superTable, pTable); + mnodeDecTableRef(pTable->superTable); } else { grantRestore(TSDB_GRANT_TIMESERIES, pTable->numOfColumns - 1); - pAcct->acctInfo.numOfTimeSeries -= (pTable->numOfColumns - 1); + if (pAcct != NULL) pAcct->acctInfo.numOfTimeSeries -= (pTable->numOfColumns - 1); } - mgmtRemoveTableFromDb(pDb); - mgmtRemoveTableFromVgroup(pVgroup, pTable); + + if (pDb != NULL) mnodeRemoveTableFromDb(pDb); + if (pVgroup != NULL) mnodeRemoveTableFromVgroup(pVgroup, pTable); + + mnodeDecVgroupRef(pVgroup); + mnodeDecDbRef(pDb); + mnodeDecAcctRef(pAcct); return TSDB_CODE_SUCCESS; } -static int32_t mgmtChildTableActionUpdate(SSdbOper *pOper) { +static int32_t mnodeChildTableActionUpdate(SSdbOper *pOper) { SChildTableObj *pNew = pOper->pObj; - SChildTableObj *pTable = mgmtGetChildTable(pNew->info.tableId); + SChildTableObj *pTable = mnodeGetChildTable(pNew->info.tableId); if (pTable != pNew) { void *oldTableId = pTable->info.tableId; void *oldSql = pTable->sql; @@ -192,12 +191,12 @@ static int32_t mgmtChildTableActionUpdate(SSdbOper *pOper) { free(oldSchema); free(oldTableId); } - mgmtDecTableRef(pTable); + mnodeDecTableRef(pTable); return TSDB_CODE_SUCCESS; } -static int32_t mgmtChildTableActionEncode(SSdbOper *pOper) { +static int32_t mnodeChildTableActionEncode(SSdbOper *pOper) { SChildTableObj *pTable = pOper->pObj; assert(pTable != NULL && pOper->rowData != NULL); @@ -227,13 +226,16 @@ static int32_t mgmtChildTableActionEncode(SSdbOper *pOper) { return TSDB_CODE_SUCCESS; } -static int32_t mgmtChildTableActionDecode(SSdbOper *pOper) { +static int32_t mnodeChildTableActionDecode(SSdbOper *pOper) { assert(pOper->rowData != NULL); SChildTableObj *pTable = calloc(1, sizeof(SChildTableObj)); if (pTable == NULL) return TSDB_CODE_SERV_OUT_OF_MEMORY; int32_t len = strlen(pOper->rowData); - if (len > TSDB_TABLE_ID_LEN) return TSDB_CODE_INVALID_TABLE_ID; + if (len > TSDB_TABLE_ID_LEN) { + free(pTable); + return TSDB_CODE_INVALID_TABLE_ID; + } pTable->info.tableId = strdup(pOper->rowData); len++; @@ -244,7 +246,7 @@ static int32_t mgmtChildTableActionDecode(SSdbOper *pOper) { int32_t schemaSize = pTable->numOfColumns * sizeof(SSchema); pTable->schema = (SSchema *)malloc(schemaSize); if (pTable->schema == NULL) { - mgmtDestroyChildTable(pTable); + mnodeDestroyChildTable(pTable); return TSDB_CODE_INVALID_TABLE_TYPE; } memcpy(pTable->schema, pOper->rowData + len, schemaSize); @@ -253,7 +255,7 @@ static int32_t mgmtChildTableActionDecode(SSdbOper *pOper) { if (pTable->sqlLen != 0) { pTable->sql = malloc(pTable->sqlLen); if (pTable->sql == NULL) { - mgmtDestroyChildTable(pTable); + mnodeDestroyChildTable(pTable); return TSDB_CODE_SERV_OUT_OF_MEMORY; } memcpy(pTable->sql, pOper->rowData + len, pTable->sqlLen); @@ -264,34 +266,34 @@ static int32_t mgmtChildTableActionDecode(SSdbOper *pOper) { return TSDB_CODE_SUCCESS; } -static int32_t mgmtChildTableActionRestored() { +static int32_t mnodeChildTableActionRestored() { void *pIter = NULL; SChildTableObj *pTable = NULL; while (1) { - pIter = mgmtGetNextChildTable(pIter, &pTable); + pIter = mnodeGetNextChildTable(pIter, &pTable); if (pTable == NULL) break; - SDbObj *pDb = mgmtGetDbByTableId(pTable->info.tableId); + SDbObj *pDb = mnodeGetDbByTableId(pTable->info.tableId); if (pDb == NULL) { mError("ctable:%s, failed to get db, discard it", pTable->info.tableId); SSdbOper desc = {.type = SDB_OPER_LOCAL, .pObj = pTable, .table = tsChildTableSdb}; sdbDeleteRow(&desc); - mgmtDecTableRef(pTable); + mnodeDecTableRef(pTable); continue; } - mgmtDecDbRef(pDb); + mnodeDecDbRef(pDb); - SVgObj *pVgroup = mgmtGetVgroup(pTable->vgId); + SVgObj *pVgroup = mnodeGetVgroup(pTable->vgId); if (pVgroup == NULL) { mError("ctable:%s, failed to get vgId:%d sid:%d, discard it", pTable->info.tableId, pTable->vgId, pTable->sid); pTable->vgId = 0; SSdbOper desc = {.type = SDB_OPER_LOCAL, .pObj = pTable, .table = tsChildTableSdb}; sdbDeleteRow(&desc); - mgmtDecTableRef(pTable); + mnodeDecTableRef(pTable); continue; } - mgmtDecVgroupRef(pVgroup); + mnodeDecVgroupRef(pVgroup); if (strcmp(pVgroup->dbName, pDb->name) != 0) { mError("ctable:%s, db:%s not match with vgId:%d db:%s sid:%d, discard it", @@ -299,7 +301,7 @@ static int32_t mgmtChildTableActionRestored() { pTable->vgId = 0; SSdbOper desc = {.type = SDB_OPER_LOCAL, .pObj = pTable, .table = tsChildTableSdb}; sdbDeleteRow(&desc); - mgmtDecTableRef(pTable); + mnodeDecTableRef(pTable); continue; } @@ -308,24 +310,24 @@ static int32_t mgmtChildTableActionRestored() { pTable->vgId = 0; SSdbOper desc = {.type = SDB_OPER_LOCAL, .pObj = pTable, .table = tsChildTableSdb}; sdbDeleteRow(&desc); - mgmtDecTableRef(pTable); + mnodeDecTableRef(pTable); continue; } if (pTable->info.type == TSDB_CHILD_TABLE) { - SSuperTableObj *pSuperTable = mgmtGetSuperTableByUid(pTable->suid); + SSuperTableObj *pSuperTable = mnodeGetSuperTableByUid(pTable->suid); if (pSuperTable == NULL) { mError("ctable:%s, stable:%" PRIu64 " not exist", pTable->info.tableId, pTable->suid); pTable->vgId = 0; SSdbOper desc = {.type = SDB_OPER_LOCAL, .pObj = pTable, .table = tsChildTableSdb}; sdbDeleteRow(&desc); - mgmtDecTableRef(pTable); + mnodeDecTableRef(pTable); continue; } - mgmtDecTableRef(pSuperTable); + mnodeDecTableRef(pSuperTable); } - mgmtDecTableRef(pTable); + mnodeDecTableRef(pTable); } sdbFreeIter(pIter); @@ -333,7 +335,7 @@ static int32_t mgmtChildTableActionRestored() { return 0; } -static int32_t mgmtInitChildTables() { +static int32_t mnodeInitChildTables() { SChildTableObj tObj; tsChildTableUpdateSize = (int8_t *)tObj.updateEnd - (int8_t *)&tObj.info.type; @@ -344,13 +346,13 @@ static int32_t mgmtInitChildTables() { .maxRowSize = sizeof(SChildTableObj) + sizeof(SSchema) * (TSDB_MAX_TAGS + TSDB_MAX_COLUMNS + 16) + TSDB_TABLE_ID_LEN + TSDB_CQ_SQL_SIZE, .refCountPos = (int8_t *)(&tObj.refCount) - (int8_t *)&tObj, .keyType = SDB_KEY_VAR_STRING, - .insertFp = mgmtChildTableActionInsert, - .deleteFp = mgmtChildTableActionDelete, - .updateFp = mgmtChildTableActionUpdate, - .encodeFp = mgmtChildTableActionEncode, - .decodeFp = mgmtChildTableActionDecode, - .destroyFp = mgmtChildTableActionDestroy, - .restoredFp = mgmtChildTableActionRestored + .insertFp = mnodeChildTableActionInsert, + .deleteFp = mnodeChildTableActionDelete, + .updateFp = mnodeChildTableActionUpdate, + .encodeFp = mnodeChildTableActionEncode, + .decodeFp = mnodeChildTableActionDecode, + .destroyFp = mnodeChildTableActionDestroy, + .restoredFp = mnodeChildTableActionRestored }; tsChildTableSdb = sdbOpenTable(&tableDesc); @@ -363,11 +365,11 @@ static int32_t mgmtInitChildTables() { return 0; } -static void mgmtCleanUpChildTables() { +static void mnodeCleanupChildTables() { sdbCloseTable(tsChildTableSdb); } -static void mgmtAddTableIntoStable(SSuperTableObj *pStable, SChildTableObj *pCtable) { +static void mnodeAddTableIntoStable(SSuperTableObj *pStable, SChildTableObj *pCtable) { pStable->numOfTables++; if (pStable->vgHash == NULL) { @@ -379,19 +381,19 @@ static void mgmtAddTableIntoStable(SSuperTableObj *pStable, SChildTableObj *pCta } } -static void mgmtRemoveTableFromStable(SSuperTableObj *pStable, SChildTableObj *pCtable) { +static void mnodeRemoveTableFromStable(SSuperTableObj *pStable, SChildTableObj *pCtable) { pStable->numOfTables--; if (pStable->vgHash == NULL) return; - SVgObj *pVgroup = mgmtGetVgroup(pCtable->vgId); + SVgObj *pVgroup = mnodeGetVgroup(pCtable->vgId); if (pVgroup == NULL) { taosHashRemove(pStable->vgHash, (char *)&pCtable->vgId, sizeof(pCtable->vgId)); } - mgmtDecVgroupRef(pVgroup); + mnodeDecVgroupRef(pVgroup); } -static void mgmtDestroySuperTable(SSuperTableObj *pStable) { +static void mnodeDestroySuperTable(SSuperTableObj *pStable) { if (pStable->vgHash != NULL) { taosHashCleanup(pStable->vgHash); pStable->vgHash = NULL; @@ -401,37 +403,37 @@ static void mgmtDestroySuperTable(SSuperTableObj *pStable) { tfree(pStable); } -static int32_t mgmtSuperTableActionDestroy(SSdbOper *pOper) { - mgmtDestroySuperTable(pOper->pObj); +static int32_t mnodeSuperTableActionDestroy(SSdbOper *pOper) { + mnodeDestroySuperTable(pOper->pObj); return TSDB_CODE_SUCCESS; } -static int32_t mgmtSuperTableActionInsert(SSdbOper *pOper) { +static int32_t mnodeSuperTableActionInsert(SSdbOper *pOper) { SSuperTableObj *pStable = pOper->pObj; - SDbObj *pDb = mgmtGetDbByTableId(pStable->info.tableId); + SDbObj *pDb = mnodeGetDbByTableId(pStable->info.tableId); if (pDb != NULL) { - mgmtAddSuperTableIntoDb(pDb); + mnodeAddSuperTableIntoDb(pDb); } - mgmtDecDbRef(pDb); + mnodeDecDbRef(pDb); return TSDB_CODE_SUCCESS; } -static int32_t mgmtSuperTableActionDelete(SSdbOper *pOper) { +static int32_t mnodeSuperTableActionDelete(SSdbOper *pOper) { SSuperTableObj *pStable = pOper->pObj; - SDbObj *pDb = mgmtGetDbByTableId(pStable->info.tableId); + SDbObj *pDb = mnodeGetDbByTableId(pStable->info.tableId); if (pDb != NULL) { - mgmtRemoveSuperTableFromDb(pDb); - mgmtDropAllChildTablesInStable((SSuperTableObj *)pStable); + mnodeRemoveSuperTableFromDb(pDb); + mnodeDropAllChildTablesInStable((SSuperTableObj *)pStable); } - mgmtDecDbRef(pDb); + mnodeDecDbRef(pDb); return TSDB_CODE_SUCCESS; } -static int32_t mgmtSuperTableActionUpdate(SSdbOper *pOper) { +static int32_t mnodeSuperTableActionUpdate(SSdbOper *pOper) { SSuperTableObj *pNew = pOper->pObj; - SSuperTableObj *pTable = mgmtGetSuperTable(pNew->info.tableId); + SSuperTableObj *pTable = mnodeGetSuperTable(pNew->info.tableId); if (pTable != pNew) { void *oldTableId = pTable->info.tableId; void *oldSchema = pTable->schema; @@ -442,11 +444,11 @@ static int32_t mgmtSuperTableActionUpdate(SSdbOper *pOper) { free(oldTableId); free(oldSchema); } - mgmtDecTableRef(pTable); + mnodeDecTableRef(pTable); return TSDB_CODE_SUCCESS; } -static int32_t mgmtSuperTableActionEncode(SSdbOper *pOper) { +static int32_t mnodeSuperTableActionEncode(SSdbOper *pOper) { SSuperTableObj *pStable = pOper->pObj; assert(pOper->pObj != NULL && pOper->rowData != NULL); @@ -469,13 +471,16 @@ static int32_t mgmtSuperTableActionEncode(SSdbOper *pOper) { return TSDB_CODE_SUCCESS; } -static int32_t mgmtSuperTableActionDecode(SSdbOper *pOper) { +static int32_t mnodeSuperTableActionDecode(SSdbOper *pOper) { assert(pOper->rowData != NULL); SSuperTableObj *pStable = (SSuperTableObj *) calloc(1, sizeof(SSuperTableObj)); if (pStable == NULL) return TSDB_CODE_SERV_OUT_OF_MEMORY; int32_t len = strlen(pOper->rowData); - if (len > TSDB_TABLE_ID_LEN) return TSDB_CODE_INVALID_TABLE_ID; + if (len > TSDB_TABLE_ID_LEN){ + free(pStable); + return TSDB_CODE_INVALID_TABLE_ID; + } pStable->info.tableId = strdup(pOper->rowData); len++; @@ -485,7 +490,7 @@ static int32_t mgmtSuperTableActionDecode(SSdbOper *pOper) { int32_t schemaSize = sizeof(SSchema) * (pStable->numOfColumns + pStable->numOfTags); pStable->schema = malloc(schemaSize); if (pStable->schema == NULL) { - mgmtDestroySuperTable(pStable); + mnodeDestroySuperTable(pStable); return TSDB_CODE_NOT_SUPER_TABLE; } @@ -496,11 +501,11 @@ static int32_t mgmtSuperTableActionDecode(SSdbOper *pOper) { return TSDB_CODE_SUCCESS; } -static int32_t mgmtSuperTableActionRestored() { +static int32_t mnodeSuperTableActionRestored() { return 0; } -static int32_t mgmtInitSuperTables() { +static int32_t mnodeInitSuperTables() { SSuperTableObj tObj; tsSuperTableUpdateSize = (int8_t *)tObj.updateEnd - (int8_t *)&tObj.info.type; @@ -511,13 +516,13 @@ static int32_t mgmtInitSuperTables() { .maxRowSize = sizeof(SSuperTableObj) + sizeof(SSchema) * (TSDB_MAX_TAGS + TSDB_MAX_COLUMNS + 16) + TSDB_TABLE_ID_LEN, .refCountPos = (int8_t *)(&tObj.refCount) - (int8_t *)&tObj, .keyType = SDB_KEY_VAR_STRING, - .insertFp = mgmtSuperTableActionInsert, - .deleteFp = mgmtSuperTableActionDelete, - .updateFp = mgmtSuperTableActionUpdate, - .encodeFp = mgmtSuperTableActionEncode, - .decodeFp = mgmtSuperTableActionDecode, - .destroyFp = mgmtSuperTableActionDestroy, - .restoredFp = mgmtSuperTableActionRestored + .insertFp = mnodeSuperTableActionInsert, + .deleteFp = mnodeSuperTableActionDelete, + .updateFp = mnodeSuperTableActionUpdate, + .encodeFp = mnodeSuperTableActionEncode, + .decodeFp = mnodeSuperTableActionDecode, + .destroyFp = mnodeSuperTableActionDestroy, + .restoredFp = mnodeSuperTableActionRestored }; tsSuperTableSdb = sdbOpenTable(&tableDesc); @@ -530,63 +535,65 @@ static int32_t mgmtInitSuperTables() { return 0; } -static void mgmtCleanUpSuperTables() { +static void mnodeCleanupSuperTables() { sdbCloseTable(tsSuperTableSdb); } -int32_t mgmtInitTables() { - int32_t code = mgmtInitSuperTables(); +int32_t mnodeInitTables() { + int32_t code = mnodeInitSuperTables(); if (code != TSDB_CODE_SUCCESS) { return code; } - code = mgmtInitChildTables(); + code = mnodeInitChildTables(); if (code != TSDB_CODE_SUCCESS) { return code; } - mgmtAddShellMsgHandle(TSDB_MSG_TYPE_CM_TABLES_META, mgmtProcessMultiTableMetaMsg); - mgmtAddShellMsgHandle(TSDB_MSG_TYPE_CM_CREATE_TABLE, mgmtProcessCreateTableMsg); - mgmtAddShellMsgHandle(TSDB_MSG_TYPE_CM_DROP_TABLE, mgmtProcessDropTableMsg); - mgmtAddShellMsgHandle(TSDB_MSG_TYPE_CM_ALTER_TABLE, mgmtProcessAlterTableMsg); - mgmtAddShellMsgHandle(TSDB_MSG_TYPE_CM_TABLE_META, mgmtProcessTableMetaMsg); - mgmtAddShellMsgHandle(TSDB_MSG_TYPE_CM_STABLE_VGROUP, mgmtProcessSuperTableVgroupMsg); + mnodeAddReadMsgHandle(TSDB_MSG_TYPE_CM_TABLES_META, mnodeProcessMultiTableMetaMsg); + mnodeAddWriteMsgHandle(TSDB_MSG_TYPE_CM_CREATE_TABLE, mnodeProcessCreateTableMsg); + mnodeAddWriteMsgHandle(TSDB_MSG_TYPE_CM_DROP_TABLE, mnodeProcessDropTableMsg); + mnodeAddWriteMsgHandle(TSDB_MSG_TYPE_CM_ALTER_TABLE, mnodeProcessAlterTableMsg); + mnodeAddReadMsgHandle(TSDB_MSG_TYPE_CM_TABLE_META, mnodeProcessTableMetaMsg); + mnodeAddReadMsgHandle(TSDB_MSG_TYPE_CM_STABLE_VGROUP, mnodeProcessSuperTableVgroupMsg); - dnodeAddClientRspHandle(TSDB_MSG_TYPE_MD_CREATE_TABLE_RSP, mgmtProcessCreateChildTableRsp); - dnodeAddClientRspHandle(TSDB_MSG_TYPE_MD_DROP_TABLE_RSP, mgmtProcessDropChildTableRsp); - dnodeAddClientRspHandle(TSDB_MSG_TYPE_MD_DROP_STABLE_RSP, mgmtProcessDropSuperTableRsp); - dnodeAddClientRspHandle(TSDB_MSG_TYPE_MD_ALTER_TABLE_RSP, mgmtProcessAlterTableRsp); + mnodeAddPeerRspHandle(TSDB_MSG_TYPE_MD_CREATE_TABLE_RSP, mnodeProcessCreateChildTableRsp); + mnodeAddPeerRspHandle(TSDB_MSG_TYPE_MD_DROP_TABLE_RSP, mnodeProcessDropChildTableRsp); + mnodeAddPeerRspHandle(TSDB_MSG_TYPE_MD_DROP_STABLE_RSP, mnodeProcessDropSuperTableRsp); + mnodeAddPeerRspHandle(TSDB_MSG_TYPE_MD_ALTER_TABLE_RSP, mnodeProcessAlterTableRsp); - dnodeAddServerMsgHandle(TSDB_MSG_TYPE_DM_CONFIG_TABLE, mgmtProcessTableCfgMsg); + mnodeAddPeerMsgHandle(TSDB_MSG_TYPE_DM_CONFIG_TABLE, mnodeProcessTableCfgMsg); + + mnodeAddShowMetaHandle(TSDB_MGMT_TABLE_TABLE, mnodeGetShowTableMeta); + mnodeAddShowRetrieveHandle(TSDB_MGMT_TABLE_TABLE, mnodeRetrieveShowTables); + mnodeAddShowMetaHandle(TSDB_MGMT_TABLE_METRIC, mnodeGetShowSuperTableMeta); + mnodeAddShowRetrieveHandle(TSDB_MGMT_TABLE_METRIC, mnodeRetrieveShowSuperTables); + mnodeAddShowMetaHandle(TSDB_MGMT_TABLE_STREAMS, mnodeGetStreamMeta); + mnodeAddShowRetrieveHandle(TSDB_MGMT_TABLE_STREAMS, mnodeRetrieveStreams); - mgmtAddShellShowMetaHandle(TSDB_MGMT_TABLE_TABLE, mgmtGetShowTableMeta); - mgmtAddShellShowRetrieveHandle(TSDB_MGMT_TABLE_TABLE, mgmtRetrieveShowTables); - mgmtAddShellShowMetaHandle(TSDB_MGMT_TABLE_METRIC, mgmtGetShowSuperTableMeta); - mgmtAddShellShowRetrieveHandle(TSDB_MGMT_TABLE_METRIC, mgmtRetrieveShowSuperTables); - return TSDB_CODE_SUCCESS; } -static void *mgmtGetChildTable(char *tableId) { +static void *mnodeGetChildTable(char *tableId) { return sdbGetRow(tsChildTableSdb, tableId); } -static void *mgmtGetSuperTable(char *tableId) { +static void *mnodeGetSuperTable(char *tableId) { return sdbGetRow(tsSuperTableSdb, tableId); } -static void *mgmtGetSuperTableByUid(uint64_t uid) { +static void *mnodeGetSuperTableByUid(uint64_t uid) { SSuperTableObj *pStable = NULL; void *pIter = NULL; while (1) { - pIter = mgmtGetNextSuperTable(pIter, &pStable); + pIter = mnodeGetNextSuperTable(pIter, &pStable); if (pStable == NULL) break; if (pStable->uid == uid) { sdbFreeIter(pIter); return pStable; } - mgmtDecTableRef(pStable); + mnodeDecTableRef(pStable); } sdbFreeIter(pIter); @@ -594,13 +601,13 @@ static void *mgmtGetSuperTableByUid(uint64_t uid) { return NULL; } -void *mgmtGetTable(char *tableId) { - void *pTable = mgmtGetSuperTable(tableId); +void *mnodeGetTable(char *tableId) { + void *pTable = mnodeGetSuperTable(tableId); if (pTable != NULL) { return pTable; } - pTable = mgmtGetChildTable(tableId); + pTable = mnodeGetChildTable(tableId); if (pTable != NULL) { return pTable; } @@ -608,15 +615,15 @@ void *mgmtGetTable(char *tableId) { return NULL; } -void *mgmtGetNextChildTable(void *pIter, SChildTableObj **pTable) { +void *mnodeGetNextChildTable(void *pIter, SChildTableObj **pTable) { return sdbFetchRow(tsChildTableSdb, pIter, (void **)pTable); } -void *mgmtGetNextSuperTable(void *pIter, SSuperTableObj **pTable) { +void *mnodeGetNextSuperTable(void *pIter, SSuperTableObj **pTable) { return sdbFetchRow(tsSuperTableSdb, pIter, (void **)pTable); } -void mgmtIncTableRef(void *p1) { +void mnodeIncTableRef(void *p1) { STableObj *pTable = (STableObj *)p1; if (pTable->type == TSDB_SUPER_TABLE) { sdbIncRef(tsSuperTableSdb, pTable); @@ -625,7 +632,7 @@ void mgmtIncTableRef(void *p1) { } } -void mgmtDecTableRef(void *p1) { +void mnodeDecTableRef(void *p1) { if (p1 == NULL) return; STableObj *pTable = (STableObj *)p1; @@ -636,13 +643,13 @@ void mgmtDecTableRef(void *p1) { } } -void mgmtCleanUpTables() { - mgmtCleanUpChildTables(); - mgmtCleanUpSuperTables(); +void mnodeCleanupTables() { + mnodeCleanupChildTables(); + mnodeCleanupSuperTables(); } // todo move to name.h, add length of table name -static void mgmtExtractTableName(char* tableId, char* name) { +static void mnodeExtractTableName(char* tableId, char* name) { int pos = -1; int num = 0; for (pos = 0; tableId[pos] != 0; ++pos) { @@ -655,113 +662,106 @@ static void mgmtExtractTableName(char* tableId, char* name) { } } -static void mgmtProcessCreateTableMsg(SQueuedMsg *pMsg) { - SCMCreateTableMsg *pCreate = pMsg->pCont; +static int32_t mnodeProcessCreateTableMsg(SMnodeMsg *pMsg) { + SCMCreateTableMsg *pCreate = pMsg->rpcMsg.pCont; - if (pMsg->pDb == NULL) pMsg->pDb = mgmtGetDb(pCreate->db); + if (pMsg->pDb == NULL) pMsg->pDb = mnodeGetDb(pCreate->db); if (pMsg->pDb == NULL || pMsg->pDb->status != TSDB_DB_STATUS_READY) { mError("table:%s, failed to create, db not selected", pCreate->tableId); - mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_DB_NOT_SELECTED); - return; + return TSDB_CODE_DB_NOT_SELECTED; } - if (pMsg->pTable == NULL) pMsg->pTable = mgmtGetTable(pCreate->tableId); + if (pMsg->pTable == NULL) pMsg->pTable = mnodeGetTable(pCreate->tableId); if (pMsg->pTable != NULL && pMsg->retry == 0) { if (pCreate->getMeta) { mTrace("table:%s, continue to get meta", pCreate->tableId); - mgmtGetChildTableMeta(pMsg); + return mnodeGetChildTableMeta(pMsg); } else if (pCreate->igExists) { mTrace("table:%s, is already exist", pCreate->tableId); - mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_SUCCESS); + return TSDB_CODE_SUCCESS; } else { mError("table:%s, failed to create, table already exist", pCreate->tableId); - mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_TABLE_ALREADY_EXIST); + return TSDB_CODE_TABLE_ALREADY_EXIST; } - return; } if (pCreate->numOfTags != 0) { - mTrace("table:%s, create msg is received from thandle:%p", pCreate->tableId, pMsg->thandle); - mgmtProcessCreateSuperTableMsg(pMsg); + mTrace("table:%s, create stable msg is received from thandle:%p", pCreate->tableId, pMsg->rpcMsg.handle); + return mnodeProcessCreateSuperTableMsg(pMsg); } else { - mTrace("table:%s, create msg is received from thandle:%p", pCreate->tableId, pMsg->thandle); - mgmtProcessCreateChildTableMsg(pMsg); + mTrace("table:%s, create ctable msg is received from thandle:%p", pCreate->tableId, pMsg->rpcMsg.handle); + return mnodeProcessCreateChildTableMsg(pMsg); } } -static void mgmtProcessDropTableMsg(SQueuedMsg *pMsg) { - SCMDropTableMsg *pDrop = pMsg->pCont; - if (pMsg->pDb == NULL) pMsg->pDb = mgmtGetDbByTableId(pDrop->tableId); +static int32_t mnodeProcessDropTableMsg(SMnodeMsg *pMsg) { + SCMDropTableMsg *pDrop = pMsg->rpcMsg.pCont; + if (pMsg->pDb == NULL) pMsg->pDb = mnodeGetDbByTableId(pDrop->tableId); if (pMsg->pDb == NULL || pMsg->pDb->status != TSDB_DB_STATUS_READY) { mError("table:%s, failed to drop table, db not selected", pDrop->tableId); - mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_DB_NOT_SELECTED); - return; + return TSDB_CODE_DB_NOT_SELECTED; } - if (mgmtCheckIsMonitorDB(pMsg->pDb->name, tsMonitorDbName)) { + if (mnodeCheckIsMonitorDB(pMsg->pDb->name, tsMonitorDbName)) { mError("table:%s, failed to drop table, in monitor database", pDrop->tableId); - mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_MONITOR_DB_FORBIDDEN); - return; + return TSDB_CODE_MONITOR_DB_FORBIDDEN; } - if (pMsg->pTable == NULL) pMsg->pTable = mgmtGetTable(pDrop->tableId); + if (pMsg->pTable == NULL) pMsg->pTable = mnodeGetTable(pDrop->tableId); if (pMsg->pTable == NULL) { if (pDrop->igNotExists) { mTrace("table:%s, table is not exist, think drop success", pDrop->tableId); - mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_SUCCESS); - return; + return TSDB_CODE_SUCCESS; } else { mError("table:%s, failed to drop table, table not exist", pDrop->tableId); - mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_INVALID_TABLE); - return; + return TSDB_CODE_INVALID_TABLE_ID; } } if (pMsg->pTable->type == TSDB_SUPER_TABLE) { mPrint("table:%s, start to drop stable", pDrop->tableId); - mgmtProcessDropSuperTableMsg(pMsg); + return mnodeProcessDropSuperTableMsg(pMsg); } else { mPrint("table:%s, start to drop ctable", pDrop->tableId); - mgmtProcessDropChildTableMsg(pMsg); + return mnodeProcessDropChildTableMsg(pMsg); } } -static void mgmtProcessTableMetaMsg(SQueuedMsg *pMsg) { - SCMTableInfoMsg *pInfo = pMsg->pCont; +static int32_t mnodeProcessTableMetaMsg(SMnodeMsg *pMsg) { + SCMTableInfoMsg *pInfo = pMsg->rpcMsg.pCont; pInfo->createFlag = htons(pInfo->createFlag); - mTrace("table:%s, table meta msg is received from thandle:%p, createFlag:%d", pInfo->tableId, pMsg->thandle, pInfo->createFlag); + mTrace("table:%s, table meta msg is received from thandle:%p, createFlag:%d", pInfo->tableId, pMsg->rpcMsg.handle, pInfo->createFlag); - if (pMsg->pDb == NULL) pMsg->pDb = mgmtGetDbByTableId(pInfo->tableId); + if (pMsg->pDb == NULL) pMsg->pDb = mnodeGetDbByTableId(pInfo->tableId); if (pMsg->pDb == NULL || pMsg->pDb->status != TSDB_DB_STATUS_READY) { mError("table:%s, failed to get table meta, db not selected", pInfo->tableId); - mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_DB_NOT_SELECTED); - return; + return TSDB_CODE_DB_NOT_SELECTED; } - if (pMsg->pTable == NULL) pMsg->pTable = mgmtGetTable(pInfo->tableId); + if (pMsg->pTable == NULL) pMsg->pTable = mnodeGetTable(pInfo->tableId); if (pMsg->pTable == NULL) { if (!pInfo->createFlag) { mError("table:%s, failed to get table meta, table not exist", pInfo->tableId); - mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_INVALID_TABLE); + return TSDB_CODE_INVALID_TABLE_ID; } else { - mgmtAutoCreateChildTable(pMsg); + mTrace("table:%s, failed to get table meta, start auto create table ", pInfo->tableId); + return mnodeAutoCreateChildTable(pMsg); } } else { if (pMsg->pTable->type != TSDB_SUPER_TABLE) { - mgmtGetChildTableMeta(pMsg); + return mnodeGetChildTableMeta(pMsg); } else { - mgmtGetSuperTableMeta(pMsg); + return mnodeGetSuperTableMeta(pMsg); } } } -static void mgmtProcessCreateSuperTableMsg(SQueuedMsg *pMsg) { - SCMCreateTableMsg *pCreate = pMsg->pCont; +static int32_t mnodeProcessCreateSuperTableMsg(SMnodeMsg *pMsg) { + SCMCreateTableMsg *pCreate = pMsg->rpcMsg.pCont; SSuperTableObj *pStable = calloc(1, sizeof(SSuperTableObj)); if (pStable == NULL) { mError("table:%s, failed to create, no enough memory", pCreate->tableId); - mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_SERV_OUT_OF_MEMORY); - return; + return TSDB_CODE_SERV_OUT_OF_MEMORY; } pStable->info.tableId = strdup(pCreate->tableId); @@ -779,8 +779,7 @@ static void mgmtProcessCreateSuperTableMsg(SQueuedMsg *pMsg) { if (pStable->schema == NULL) { free(pStable); mError("table:%s, failed to create, no schema input", pCreate->tableId); - mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_INVALID_TABLE); - return; + return TSDB_CODE_INVALID_TABLE_ID; } memcpy(pStable->schema, pCreate->schema, numOfCols * sizeof(SSchema)); @@ -803,39 +802,39 @@ static void mgmtProcessCreateSuperTableMsg(SQueuedMsg *pMsg) { int32_t code = sdbInsertRow(&oper); if (code != TSDB_CODE_SUCCESS) { - mgmtDestroySuperTable(pStable); + mnodeDestroySuperTable(pStable); mError("table:%s, failed to create, sdb error", pCreate->tableId); - mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_SDB_ERROR); + return TSDB_CODE_SDB_ERROR; } else { mLPrint("table:%s, is created, tags:%d fields:%d", pStable->info.tableId, pStable->numOfTags, pStable->numOfColumns); - mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_SUCCESS); + return TSDB_CODE_SUCCESS; } } -static void mgmtProcessDropSuperTableMsg(SQueuedMsg *pMsg) { +static int32_t mnodeProcessDropSuperTableMsg(SMnodeMsg *pMsg) { SSuperTableObj *pStable = (SSuperTableObj *)pMsg->pTable; if (pStable->numOfTables != 0) { SHashMutableIterator *pIter = taosHashCreateIter(pStable->vgHash); while (taosHashIterNext(pIter)) { int32_t *pVgId = taosHashIterGet(pIter); - SVgObj *pVgroup = mgmtGetVgroup(*pVgId); + SVgObj *pVgroup = mnodeGetVgroup(*pVgId); if (pVgroup == NULL) break; SMDDropSTableMsg *pDrop = rpcMallocCont(sizeof(SMDDropSTableMsg)); pDrop->contLen = htonl(sizeof(SMDDropSTableMsg)); pDrop->vgId = htonl(pVgroup->vgId); pDrop->uid = htobe64(pStable->uid); - mgmtExtractTableName(pStable->info.tableId, pDrop->tableId); + mnodeExtractTableName(pStable->info.tableId, pDrop->tableId); mPrint("stable:%s, send drop stable msg to vgId:%d", pStable->info.tableId, pVgroup->vgId); - SRpcIpSet ipSet = mgmtGetIpSetFromVgroup(pVgroup); + SRpcIpSet ipSet = mnodeGetIpSetFromVgroup(pVgroup); SRpcMsg rpcMsg = {.pCont = pDrop, .contLen = sizeof(SMDDropSTableMsg), .msgType = TSDB_MSG_TYPE_MD_DROP_STABLE}; dnodeSendMsgToDnode(&ipSet, &rpcMsg); - mgmtDecVgroupRef(pVgroup); + mnodeDecVgroupRef(pVgroup); } taosHashDestroyIter(pIter); - mgmtDropAllChildTablesInStable(pStable); + mnodeDropAllChildTablesInStable(pStable); } SSdbOper oper = { @@ -846,10 +845,10 @@ static void mgmtProcessDropSuperTableMsg(SQueuedMsg *pMsg) { int32_t code = sdbDeleteRow(&oper); mLPrint("stable:%s, is dropped from sdb, result:%s", pStable->info.tableId, tstrerror(code)); - mgmtSendSimpleResp(pMsg->thandle, code); + return code; } -static int32_t mgmtFindSuperTableTagIndex(SSuperTableObj *pStable, const char *tagName) { +static int32_t mnodeFindSuperTableTagIndex(SSuperTableObj *pStable, const char *tagName) { SSchema *schema = (SSchema *) pStable->schema; for (int32_t tag = 0; tag < pStable->numOfTags; tag++) { if (strcasecmp(schema[pStable->numOfColumns + tag].name, tagName) == 0) { @@ -860,19 +859,19 @@ static int32_t mgmtFindSuperTableTagIndex(SSuperTableObj *pStable, const char *t return -1; } -static int32_t mgmtAddSuperTableTag(SSuperTableObj *pStable, SSchema schema[], int32_t ntags) { +static int32_t mnodeAddSuperTableTag(SSuperTableObj *pStable, SSchema schema[], int32_t ntags) { if (pStable->numOfTags + ntags > TSDB_MAX_TAGS) { mError("stable:%s, add tag, too many tags", pStable->info.tableId); return TSDB_CODE_TOO_MANY_TAGS; } for (int32_t i = 0; i < ntags; i++) { - if (mgmtFindSuperTableColumnIndex(pStable, schema[i].name) > 0) { + if (mnodeFindSuperTableColumnIndex(pStable, schema[i].name) > 0) { mError("stable:%s, add tag, column:%s already exist", pStable->info.tableId, schema[i].name); return TSDB_CODE_TAG_ALREAY_EXIST; } - if (mgmtFindSuperTableTagIndex(pStable, schema[i].name) > 0) { + if (mnodeFindSuperTableTagIndex(pStable, schema[i].name) > 0) { mError("stable:%s, add tag, tag:%s already exist", pStable->info.tableId, schema[i].name); return TSDB_CODE_FIELD_ALREAY_EXIST; } @@ -906,8 +905,8 @@ static int32_t mgmtAddSuperTableTag(SSuperTableObj *pStable, SSchema schema[], i return TSDB_CODE_SUCCESS; } -static int32_t mgmtDropSuperTableTag(SSuperTableObj *pStable, char *tagName) { - int32_t col = mgmtFindSuperTableTagIndex(pStable, tagName); +static int32_t mnodeDropSuperTableTag(SSuperTableObj *pStable, char *tagName) { + int32_t col = mnodeFindSuperTableTagIndex(pStable, tagName); if (col < 0) { mError("stable:%s, drop tag, tag:%s not exist", pStable->info.tableId, tagName); return TSDB_CODE_TAG_NOT_EXIST; @@ -933,8 +932,8 @@ static int32_t mgmtDropSuperTableTag(SSuperTableObj *pStable, char *tagName) { return TSDB_CODE_SUCCESS; } -static int32_t mgmtModifySuperTableTagName(SSuperTableObj *pStable, char *oldTagName, char *newTagName) { - int32_t col = mgmtFindSuperTableTagIndex(pStable, oldTagName); +static int32_t mnodeModifySuperTableTagName(SSuperTableObj *pStable, char *oldTagName, char *newTagName) { + int32_t col = mnodeFindSuperTableTagIndex(pStable, oldTagName); if (col < 0) { mError("stable:%s, failed to modify table tag, oldName: %s, newName: %s", pStable->info.tableId, oldTagName, newTagName); return TSDB_CODE_TAG_NOT_EXIST; @@ -946,7 +945,7 @@ static int32_t mgmtModifySuperTableTagName(SSuperTableObj *pStable, char *oldTag return TSDB_CODE_COL_NAME_TOO_LONG; } - if (mgmtFindSuperTableTagIndex(pStable, newTagName) >= 0) { + if (mnodeFindSuperTableTagIndex(pStable, newTagName) >= 0) { return TSDB_CODE_TAG_ALREAY_EXIST; } @@ -969,7 +968,7 @@ static int32_t mgmtModifySuperTableTagName(SSuperTableObj *pStable, char *oldTag return TSDB_CODE_SUCCESS; } -static int32_t mgmtFindSuperTableColumnIndex(SSuperTableObj *pStable, char *colName) { +static int32_t mnodeFindSuperTableColumnIndex(SSuperTableObj *pStable, char *colName) { SSchema *schema = (SSchema *) pStable->schema; for (int32_t col = 0; col < pStable->numOfColumns; col++) { if (strcasecmp(schema[col].name, colName) == 0) { @@ -980,19 +979,19 @@ static int32_t mgmtFindSuperTableColumnIndex(SSuperTableObj *pStable, char *colN return -1; } -static int32_t mgmtAddSuperTableColumn(SDbObj *pDb, SSuperTableObj *pStable, SSchema schema[], int32_t ncols) { +static int32_t mnodeAddSuperTableColumn(SDbObj *pDb, SSuperTableObj *pStable, SSchema schema[], int32_t ncols) { if (ncols <= 0) { mError("stable:%s, add column, ncols:%d <= 0", pStable->info.tableId); return TSDB_CODE_APP_ERROR; } for (int32_t i = 0; i < ncols; i++) { - if (mgmtFindSuperTableColumnIndex(pStable, schema[i].name) > 0) { + if (mnodeFindSuperTableColumnIndex(pStable, schema[i].name) > 0) { mError("stable:%s, add column, column:%s already exist", pStable->info.tableId, schema[i].name); return TSDB_CODE_FIELD_ALREAY_EXIST; } - if (mgmtFindSuperTableTagIndex(pStable, schema[i].name) > 0) { + if (mnodeFindSuperTableTagIndex(pStable, schema[i].name) > 0) { mError("stable:%s, add column, tag:%s already exist", pStable->info.tableId, schema[i].name); return TSDB_CODE_TAG_ALREAY_EXIST; } @@ -1013,10 +1012,10 @@ static int32_t mgmtAddSuperTableColumn(SDbObj *pDb, SSuperTableObj *pStable, SSc pStable->numOfColumns += ncols; pStable->sversion++; - SAcctObj *pAcct = mgmtGetAcct(pDb->acct); + SAcctObj *pAcct = mnodeGetAcct(pDb->acct); if (pAcct != NULL) { pAcct->acctInfo.numOfTimeSeries += (ncols * pStable->numOfTables); - mgmtDecAcctRef(pAcct); + mnodeDecAcctRef(pAcct); } SSdbOper oper = { @@ -1034,8 +1033,8 @@ static int32_t mgmtAddSuperTableColumn(SDbObj *pDb, SSuperTableObj *pStable, SSc return TSDB_CODE_SUCCESS; } -static int32_t mgmtDropSuperTableColumn(SDbObj *pDb, SSuperTableObj *pStable, char *colName) { - int32_t col = mgmtFindSuperTableColumnIndex(pStable, colName); +static int32_t mnodeDropSuperTableColumn(SDbObj *pDb, SSuperTableObj *pStable, char *colName) { + int32_t col = mnodeFindSuperTableColumnIndex(pStable, colName); if (col <= 0) { mError("stable:%s, drop column, column:%s not exist", pStable->info.tableId, colName); return TSDB_CODE_FIELD_NOT_EXIST; @@ -1050,10 +1049,10 @@ static int32_t mgmtDropSuperTableColumn(SDbObj *pDb, SSuperTableObj *pStable, ch int32_t schemaSize = sizeof(SSchema) * (pStable->numOfTags + pStable->numOfColumns); pStable->schema = realloc(pStable->schema, schemaSize); - SAcctObj *pAcct = mgmtGetAcct(pDb->acct); + SAcctObj *pAcct = mnodeGetAcct(pDb->acct); if (pAcct != NULL) { pAcct->acctInfo.numOfTimeSeries -= pStable->numOfTables; - mgmtDecAcctRef(pAcct); + mnodeDecAcctRef(pAcct); } SSdbOper oper = { @@ -1072,8 +1071,8 @@ static int32_t mgmtDropSuperTableColumn(SDbObj *pDb, SSuperTableObj *pStable, ch } // show super tables -static int32_t mgmtGetShowSuperTableMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) { - SDbObj *pDb = mgmtGetDb(pShow->db); +static int32_t mnodeGetShowSuperTableMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) { + SDbObj *pDb = mnodeGetDb(pShow->db); if (pDb == NULL) return TSDB_CODE_DB_NOT_SELECTED; int32_t cols = 0; @@ -1118,12 +1117,12 @@ static int32_t mgmtGetShowSuperTableMeta(STableMetaMsg *pMeta, SShowObj *pShow, pShow->numOfRows = pDb->numOfSuperTables; pShow->rowSize = pShow->offset[cols - 1] + pShow->bytes[cols - 1]; - mgmtDecDbRef(pDb); + mnodeDecDbRef(pDb); return 0; } // retrieve super tables -int32_t mgmtRetrieveShowSuperTables(SShowObj *pShow, char *data, int32_t rows, void *pConn) { +int32_t mnodeRetrieveShowSuperTables(SShowObj *pShow, char *data, int32_t rows, void *pConn) { int32_t numOfRows = 0; char * pWrite; int32_t cols = 0; @@ -1131,7 +1130,7 @@ int32_t mgmtRetrieveShowSuperTables(SShowObj *pShow, char *data, int32_t rows, v char prefix[20] = {0}; int32_t prefixLen; - SDbObj *pDb = mgmtGetDb(pShow->db); + SDbObj *pDb = mnodeGetDb(pShow->db); if (pDb == NULL) return 0; strcpy(prefix, pDb->name); @@ -1142,18 +1141,18 @@ int32_t mgmtRetrieveShowSuperTables(SShowObj *pShow, char *data, int32_t rows, v char stableName[TSDB_TABLE_NAME_LEN + 1] = {0}; while (numOfRows < rows) { - pShow->pIter = mgmtGetNextSuperTable(pShow->pIter, &pTable); + pShow->pIter = mnodeGetNextSuperTable(pShow->pIter, &pTable); if (pTable == NULL) break; if (strncmp(pTable->info.tableId, prefix, prefixLen)) { - mgmtDecTableRef(pTable); + mnodeDecTableRef(pTable); continue; } memset(stableName, 0, tListLen(stableName)); - mgmtExtractTableName(pTable->info.tableId, stableName); + mnodeExtractTableName(pTable->info.tableId, stableName); if (pShow->payloadLen > 0 && patternMatch(pShow->payload, stableName, TSDB_TABLE_NAME_LEN, &info) != TSDB_PATTERN_MATCH) { - mgmtDecTableRef(pTable); + mnodeDecTableRef(pTable); continue; } @@ -1185,16 +1184,16 @@ int32_t mgmtRetrieveShowSuperTables(SShowObj *pShow, char *data, int32_t rows, v cols++; numOfRows++; - mgmtDecTableRef(pTable); + mnodeDecTableRef(pTable); } pShow->numOfReads += numOfRows; - mgmtDecDbRef(pDb); + mnodeDecDbRef(pDb); return numOfRows; } -void mgmtDropAllSuperTables(SDbObj *pDropDb) { +void mnodeDropAllSuperTables(SDbObj *pDropDb) { void * pIter= NULL; int32_t numOfTables = 0; int32_t dbNameLen = strlen(pDropDb->name); @@ -1203,7 +1202,7 @@ void mgmtDropAllSuperTables(SDbObj *pDropDb) { mPrint("db:%s, all super tables will be dropped from sdb", pDropDb->name); while (1) { - pIter = mgmtGetNextSuperTable(pIter, &pTable); + pIter = mnodeGetNextSuperTable(pIter, &pTable); if (pTable == NULL) break; if (strncmp(pDropDb->name, pTable->info.tableId, dbNameLen) == 0) { @@ -1216,7 +1215,7 @@ void mgmtDropAllSuperTables(SDbObj *pDropDb) { numOfTables ++; } - mgmtDecTableRef(pTable); + mnodeDecTableRef(pTable); } sdbFreeIter(pIter); @@ -1224,7 +1223,7 @@ void mgmtDropAllSuperTables(SDbObj *pDropDb) { mPrint("db:%s, all super tables:%d is dropped from sdb", pDropDb->name, numOfTables); } -static int32_t mgmtSetSchemaFromSuperTable(SSchema *pSchema, SSuperTableObj *pTable) { +static int32_t mnodeSetSchemaFromSuperTable(SSchema *pSchema, SSuperTableObj *pTable) { int32_t numOfCols = pTable->numOfColumns + pTable->numOfTags; assert(numOfCols <= TSDB_MAX_COLUMNS); @@ -1239,7 +1238,7 @@ static int32_t mgmtSetSchemaFromSuperTable(SSchema *pSchema, SSuperTableObj *pTa return (pTable->numOfColumns + pTable->numOfTags) * sizeof(SSchema); } -static void mgmtGetSuperTableMeta(SQueuedMsg *pMsg) { +static int32_t mnodeGetSuperTableMeta(SMnodeMsg *pMsg) { SSuperTableObj *pTable = (SSuperTableObj *)pMsg->pTable; STableMetaMsg *pMeta = rpcMallocCont(sizeof(STableMetaMsg) + sizeof(SSchema) * (TSDB_MAX_TAGS + TSDB_MAX_COLUMNS + 16)); pMeta->uid = htobe64(pTable->uid); @@ -1249,54 +1248,62 @@ static void mgmtGetSuperTableMeta(SQueuedMsg *pMsg) { pMeta->numOfTags = (uint8_t)pTable->numOfTags; pMeta->numOfColumns = htons((int16_t)pTable->numOfColumns); pMeta->tableType = pTable->info.type; - pMeta->contLen = sizeof(STableMetaMsg) + mgmtSetSchemaFromSuperTable(pMeta->schema, pTable); + pMeta->contLen = sizeof(STableMetaMsg) + mnodeSetSchemaFromSuperTable(pMeta->schema, pTable); strncpy(pMeta->tableId, pTable->info.tableId, TSDB_TABLE_ID_LEN); - SRpcMsg rpcRsp = { - .handle = pMsg->thandle, - .pCont = pMeta, - .contLen = pMeta->contLen, - }; pMeta->contLen = htons(pMeta->contLen); - rpcSendResponse(&rpcRsp); + pMsg->rpcRsp.rsp = pMeta; + pMsg->rpcRsp.len = pMeta->contLen; + mTrace("stable:%s, uid:%" PRIu64 " table meta is retrieved", pTable->info.tableId, pTable->uid); + return TSDB_CODE_SUCCESS; } -static void mgmtProcessSuperTableVgroupMsg(SQueuedMsg *pMsg) { - SCMSTableVgroupMsg *pInfo = pMsg->pCont; +static int32_t mnodeProcessSuperTableVgroupMsg(SMnodeMsg *pMsg) { + SCMSTableVgroupMsg *pInfo = pMsg->rpcMsg.pCont; int32_t numOfTable = htonl(pInfo->numOfTables); // reserve space int32_t contLen = sizeof(SCMSTableVgroupRspMsg) + 32 * sizeof(SCMVgroupInfo) + sizeof(SVgroupsInfo); for (int32_t i = 0; i < numOfTable; ++i) { char *stableName = (char*)pInfo + sizeof(SCMSTableVgroupMsg) + (TSDB_TABLE_ID_LEN) * i; - SSuperTableObj *pTable = mgmtGetSuperTable(stableName); - if (pTable->vgHash != NULL) { + SSuperTableObj *pTable = mnodeGetSuperTable(stableName); + if (pTable != NULL && pTable->vgHash != NULL) { contLen += (taosHashGetSize(pTable->vgHash) * sizeof(SCMVgroupInfo) + sizeof(SVgroupsInfo)); - } - mgmtDecTableRef(pTable); + } + mnodeDecTableRef(pTable); } SCMSTableVgroupRspMsg *pRsp = rpcMallocCont(contLen); if (pRsp == NULL) { - mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_SERV_OUT_OF_MEMORY); - return; + return TSDB_CODE_SERV_OUT_OF_MEMORY; } - pRsp->numOfTables = htonl(numOfTable); - char* msg = (char*) pRsp + sizeof(SCMSTableVgroupRspMsg); + pRsp->numOfTables = 0; + char *msg = (char *)pRsp + sizeof(SCMSTableVgroupRspMsg); for (int32_t i = 0; i < numOfTable; ++i) { char *stableName = (char*)pInfo + sizeof(SCMSTableVgroupMsg) + (TSDB_TABLE_ID_LEN) * i; - SSuperTableObj *pTable = mgmtGetSuperTable(stableName); + SSuperTableObj *pTable = mnodeGetSuperTable(stableName); + if (pTable == NULL) { + mError("stable:%s, not exist while get stable vgroup info", stableName); + mnodeDecTableRef(pTable); + continue; + } + if (pTable->vgHash == NULL) { + mError("stable:%s, not vgroup exist while get stable vgroup info", stableName); + mnodeDecTableRef(pTable); + continue; + } + SVgroupsInfo *pVgroupInfo = (SVgroupsInfo *)msg; SHashMutableIterator *pIter = taosHashCreateIter(pTable->vgHash); int32_t vgSize = 0; while (taosHashIterNext(pIter)) { int32_t *pVgId = taosHashIterGet(pIter); - SVgObj * pVgroup = mgmtGetVgroup(*pVgId); + SVgObj * pVgroup = mnodeGetVgroup(*pVgId); if (pVgroup == NULL) continue; pVgroupInfo->vgroups[vgSize].vgId = htonl(pVgroup->vgId); @@ -1311,29 +1318,36 @@ static void mgmtProcessSuperTableVgroupMsg(SQueuedMsg *pMsg) { } vgSize++; - mgmtDecVgroupRef(pVgroup); + mnodeDecVgroupRef(pVgroup); } taosHashDestroyIter(pIter); + mnodeDecTableRef(pTable); pVgroupInfo->numOfVgroups = htonl(vgSize); // one table is done, try the next table msg += sizeof(SVgroupsInfo) + vgSize * sizeof(SCMVgroupInfo); + pRsp->numOfTables++; } - SRpcMsg rpcRsp = {0}; - rpcRsp.handle = pMsg->thandle; - rpcRsp.pCont = pRsp; - rpcRsp.contLen = msg - (char*) pRsp; - rpcSendResponse(&rpcRsp); + if (pRsp->numOfTables != numOfTable) { + rpcFreeCont(pRsp); + return TSDB_CODE_INVALID_TABLE_ID; + } else { + pRsp->numOfTables = htonl(pRsp->numOfTables); + pMsg->rpcRsp.rsp = pRsp; + pMsg->rpcRsp.len = msg - (char *)pRsp; + + return TSDB_CODE_SUCCESS; + } } -static void mgmtProcessDropSuperTableRsp(SRpcMsg *rpcMsg) { +static void mnodeProcessDropSuperTableRsp(SRpcMsg *rpcMsg) { mPrint("drop stable rsp received, result:%s", tstrerror(rpcMsg->code)); } -static void *mgmtBuildCreateChildTableMsg(SCMCreateTableMsg *pMsg, SChildTableObj *pTable) { +static void *mnodeBuildCreateChildTableMsg(SCMCreateTableMsg *pMsg, SChildTableObj *pTable) { STagData * pTagData = NULL; int32_t tagDataLen = 0; int32_t totalCols = 0; @@ -1354,7 +1368,7 @@ static void *mgmtBuildCreateChildTableMsg(SCMCreateTableMsg *pMsg, SChildTableOb return NULL; } - mgmtExtractTableName(pTable->info.tableId, pCreate->tableId); + mnodeExtractTableName(pTable->info.tableId, pCreate->tableId); pCreate->contLen = htonl(contLen); pCreate->vgId = htonl(pTable->vgId); pCreate->tableType = pTable->info.type; @@ -1364,7 +1378,7 @@ static void *mgmtBuildCreateChildTableMsg(SCMCreateTableMsg *pMsg, SChildTableOb pCreate->uid = htobe64(pTable->uid); if (pTable->info.type == TSDB_CHILD_TABLE) { - mgmtExtractTableName(pTable->superTable->info.tableId, pCreate->superTableId); + mnodeExtractTableName(pTable->superTable->info.tableId, pCreate->superTableId); pCreate->numOfColumns = htons(pTable->superTable->numOfColumns); pCreate->numOfTags = htons(pTable->superTable->numOfTags); pCreate->sversion = htonl(pTable->superTable->sversion); @@ -1394,13 +1408,16 @@ static void *mgmtBuildCreateChildTableMsg(SCMCreateTableMsg *pMsg, SChildTableOb if (pTable->info.type == TSDB_CHILD_TABLE && pMsg != NULL) { memcpy(pCreate->data + totalCols * sizeof(SSchema), pTagData->data, tagDataLen); - memcpy(pCreate->data + totalCols * sizeof(SSchema) + tagDataLen, pTable->sql, pTable->sqlLen); + } + + if (pTable->info.type == TSDB_STREAM_TABLE && pMsg != NULL) { + memcpy(pCreate->data + totalCols * sizeof(SSchema), pTable->sql, pTable->sqlLen); } return pCreate; } -static SChildTableObj* mgmtDoCreateChildTable(SCMCreateTableMsg *pCreate, SVgObj *pVgroup, int32_t tid) { +static SChildTableObj* mnodeDoCreateChildTable(SCMCreateTableMsg *pCreate, SVgObj *pVgroup, int32_t tid) { SChildTableObj *pTable = calloc(1, sizeof(SChildTableObj)); if (pTable == NULL) { mError("table:%s, failed to alloc memory", pCreate->tableId); @@ -1421,14 +1438,14 @@ static SChildTableObj* mgmtDoCreateChildTable(SCMCreateTableMsg *pCreate, SVgObj if (pTable->info.type == TSDB_CHILD_TABLE) { STagData *pTagData = (STagData *) pCreate->schema; // it is a tag key - SSuperTableObj *pSuperTable = mgmtGetSuperTable(pTagData->name); + SSuperTableObj *pSuperTable = mnodeGetSuperTable(pTagData->name); if (pSuperTable == NULL) { mError("table:%s, corresponding super table:%s does not exist", pCreate->tableId, pTagData->name); - free(pTable); - terrno = TSDB_CODE_INVALID_TABLE; + mnodeDestroyChildTable(pTable); + terrno = TSDB_CODE_INVALID_TABLE_ID; return NULL; } - mgmtDecTableRef(pSuperTable); + mnodeDecTableRef(pSuperTable); pTable->suid = pSuperTable->uid; pTable->uid = (((uint64_t)pTable->vgId) << 40) + ((((uint64_t)pTable->sid) & ((1ul << 24) - 1ul)) << 16) + @@ -1487,20 +1504,18 @@ static SChildTableObj* mgmtDoCreateChildTable(SCMCreateTableMsg *pCreate, SVgObj return pTable; } -static void mgmtProcessCreateChildTableMsg(SQueuedMsg *pMsg) { - SCMCreateTableMsg *pCreate = pMsg->pCont; +static int32_t mnodeProcessCreateChildTableMsg(SMnodeMsg *pMsg) { + SCMCreateTableMsg *pCreate = pMsg->rpcMsg.pCont; int32_t code = grantCheck(TSDB_GRANT_TIMESERIES); if (code != TSDB_CODE_SUCCESS) { mError("table:%s, failed to create, grant timeseries failed", pCreate->tableId); - mgmtSendSimpleResp(pMsg->thandle, code); - return; + return code; } - SVgObj *pVgroup = mgmtGetAvailableVgroup(pMsg->pDb); + SVgObj *pVgroup = mnodeGetAvailableVgroup(pMsg->pDb); if (pVgroup == NULL) { mTrace("table:%s, start to create a new vgroup", pCreate->tableId); - mgmtCreateVgroup(mgmtCloneQueuedMsg(pMsg), pMsg->pDb); - return; + return mnodeCreateVgroup(pMsg, pMsg->pDb); } if (pMsg->retry == 0) { @@ -1508,39 +1523,32 @@ static void mgmtProcessCreateChildTableMsg(SQueuedMsg *pMsg) { int32_t sid = taosAllocateId(pVgroup->idPool); if (sid <= 0) { mTrace("tables:%s, no enough sid in vgId:%d", pCreate->tableId, pVgroup->vgId); - mgmtCreateVgroup(mgmtCloneQueuedMsg(pMsg), pMsg->pDb); - return; + return mnodeCreateVgroup(pMsg, pMsg->pDb); } - pMsg->pTable = (STableObj *)mgmtDoCreateChildTable(pCreate, pVgroup, sid); + pMsg->pTable = (STableObj *)mnodeDoCreateChildTable(pCreate, pVgroup, sid); if (pMsg->pTable == NULL) { - mgmtSendSimpleResp(pMsg->thandle, terrno); - return; + return terrno; } - mgmtIncTableRef(pMsg->pTable); + mnodeIncTableRef(pMsg->pTable); } } else { - if (pMsg->pTable == NULL) pMsg->pTable = mgmtGetTable(pCreate->tableId); + if (pMsg->pTable == NULL) pMsg->pTable = mnodeGetTable(pCreate->tableId); } if (pMsg->pTable == NULL) { - mgmtSendSimpleResp(pMsg->thandle, terrno); - return; + return terrno; } - SMDCreateTableMsg *pMDCreate = mgmtBuildCreateChildTableMsg(pCreate, (SChildTableObj *) pMsg->pTable); + SMDCreateTableMsg *pMDCreate = mnodeBuildCreateChildTableMsg(pCreate, (SChildTableObj *)pMsg->pTable); if (pMDCreate == NULL) { - mgmtSendSimpleResp(pMsg->thandle, terrno); - return; + return terrno; } - SRpcIpSet ipSet = mgmtGetIpSetFromVgroup(pVgroup); - SQueuedMsg *newMsg = mgmtCloneQueuedMsg(pMsg); - newMsg->ahandle = pMsg->pTable; - newMsg->maxRetry = 10; + SRpcIpSet ipSet = mnodeGetIpSetFromVgroup(pVgroup); SRpcMsg rpcMsg = { - .handle = newMsg, + .handle = pMsg, .pCont = pMDCreate, .contLen = htonl(pMDCreate->contLen), .code = 0, @@ -1548,22 +1556,22 @@ static void mgmtProcessCreateChildTableMsg(SQueuedMsg *pMsg) { }; dnodeSendMsgToDnode(&ipSet, &rpcMsg); + + return TSDB_CODE_ACTION_IN_PROGRESS; } -static void mgmtProcessDropChildTableMsg(SQueuedMsg *pMsg) { +static int32_t mnodeProcessDropChildTableMsg(SMnodeMsg *pMsg) { SChildTableObj *pTable = (SChildTableObj *)pMsg->pTable; - if (pMsg->pVgroup == NULL) pMsg->pVgroup = mgmtGetVgroup(pTable->vgId); + if (pMsg->pVgroup == NULL) pMsg->pVgroup = mnodeGetVgroup(pTable->vgId); if (pMsg->pVgroup == NULL) { mError("table:%s, failed to drop ctable, vgroup not exist", pTable->info.tableId); - mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_OTHERS); - return; + return TSDB_CODE_OTHERS; } SMDDropTableMsg *pDrop = rpcMallocCont(sizeof(SMDDropTableMsg)); if (pDrop == NULL) { mError("table:%s, failed to drop ctable, no enough memory", pTable->info.tableId); - mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_SERV_OUT_OF_MEMORY); - return; + return TSDB_CODE_SERV_OUT_OF_MEMORY; } strcpy(pDrop->tableId, pTable->info.tableId); @@ -1572,13 +1580,11 @@ static void mgmtProcessDropChildTableMsg(SQueuedMsg *pMsg) { pDrop->sid = htonl(pTable->sid); pDrop->uid = htobe64(pTable->uid); - SRpcIpSet ipSet = mgmtGetIpSetFromVgroup(pMsg->pVgroup); + SRpcIpSet ipSet = mnodeGetIpSetFromVgroup(pMsg->pVgroup); mPrint("table:%s, send drop ctable msg", pDrop->tableId); - SQueuedMsg *newMsg = mgmtCloneQueuedMsg(pMsg); - newMsg->ahandle = pMsg->pTable; SRpcMsg rpcMsg = { - .handle = newMsg, + .handle = pMsg, .pCont = pDrop, .contLen = sizeof(SMDDropTableMsg), .code = 0, @@ -1586,13 +1592,15 @@ static void mgmtProcessDropChildTableMsg(SQueuedMsg *pMsg) { }; dnodeSendMsgToDnode(&ipSet, &rpcMsg); + + return TSDB_CODE_ACTION_IN_PROGRESS; } -static int32_t mgmtModifyChildTableTagValue(SChildTableObj *pTable, char *tagName, char *nContent) { +static int32_t mnodeModifyChildTableTagValue(SChildTableObj *pTable, char *tagName, char *nContent) { return TSDB_CODE_OPS_NOT_SUPPORT; } -static int32_t mgmtFindNormalTableColumnIndex(SChildTableObj *pTable, char *colName) { +static int32_t mnodeFindNormalTableColumnIndex(SChildTableObj *pTable, char *colName) { SSchema *schema = (SSchema *) pTable->schema; for (int32_t col = 0; col < pTable->numOfColumns; col++) { if (strcasecmp(schema[col].name, colName) == 0) { @@ -1603,14 +1611,14 @@ static int32_t mgmtFindNormalTableColumnIndex(SChildTableObj *pTable, char *colN return -1; } -static int32_t mgmtAddNormalTableColumn(SDbObj *pDb, SChildTableObj *pTable, SSchema schema[], int32_t ncols) { +static int32_t mnodeAddNormalTableColumn(SDbObj *pDb, SChildTableObj *pTable, SSchema schema[], int32_t ncols) { if (ncols <= 0) { mError("table:%s, add column, ncols:%d <= 0", pTable->info.tableId); return TSDB_CODE_APP_ERROR; } for (int32_t i = 0; i < ncols; i++) { - if (mgmtFindNormalTableColumnIndex(pTable, schema[i].name) > 0) { + if (mnodeFindNormalTableColumnIndex(pTable, schema[i].name) > 0) { mError("table:%s, add column, column:%s already exist", pTable->info.tableId, schema[i].name); return TSDB_CODE_FIELD_ALREAY_EXIST; } @@ -1629,10 +1637,10 @@ static int32_t mgmtAddNormalTableColumn(SDbObj *pDb, SChildTableObj *pTable, SSc pTable->numOfColumns += ncols; pTable->sversion++; - SAcctObj *pAcct = mgmtGetAcct(pDb->acct); + SAcctObj *pAcct = mnodeGetAcct(pDb->acct); if (pAcct != NULL) { pAcct->acctInfo.numOfTimeSeries += ncols; - mgmtDecAcctRef(pAcct); + mnodeDecAcctRef(pAcct); } SSdbOper oper = { @@ -1650,8 +1658,8 @@ static int32_t mgmtAddNormalTableColumn(SDbObj *pDb, SChildTableObj *pTable, SSc return TSDB_CODE_SUCCESS; } -static int32_t mgmtDropNormalTableColumn(SDbObj *pDb, SChildTableObj *pTable, char *colName) { - int32_t col = mgmtFindNormalTableColumnIndex(pTable, colName); +static int32_t mnodeDropNormalTableColumn(SDbObj *pDb, SChildTableObj *pTable, char *colName) { + int32_t col = mnodeFindNormalTableColumnIndex(pTable, colName); if (col <= 0) { mError("table:%s, drop column, column:%s not exist", pTable->info.tableId, colName); return TSDB_CODE_FIELD_NOT_EXIST; @@ -1661,10 +1669,10 @@ static int32_t mgmtDropNormalTableColumn(SDbObj *pDb, SChildTableObj *pTable, ch pTable->numOfColumns--; pTable->sversion++; - SAcctObj *pAcct = mgmtGetAcct(pDb->acct); + SAcctObj *pAcct = mnodeGetAcct(pDb->acct); if (pAcct != NULL) { pAcct->acctInfo.numOfTimeSeries--; - mgmtDecAcctRef(pAcct); + mnodeDecAcctRef(pAcct); } SSdbOper oper = { @@ -1682,7 +1690,7 @@ static int32_t mgmtDropNormalTableColumn(SDbObj *pDb, SChildTableObj *pTable, ch return TSDB_CODE_SUCCESS; } -static int32_t mgmtSetSchemaFromNormalTable(SSchema *pSchema, SChildTableObj *pTable) { +static int32_t mnodeSetSchemaFromNormalTable(SSchema *pSchema, SChildTableObj *pTable) { int32_t numOfCols = pTable->numOfColumns; for (int32_t i = 0; i < numOfCols; ++i) { strcpy(pSchema->name, pTable->schema[i].name); @@ -1695,7 +1703,7 @@ static int32_t mgmtSetSchemaFromNormalTable(SSchema *pSchema, SChildTableObj *pT return numOfCols * sizeof(SSchema); } -static int32_t mgmtDoGetChildTableMeta(SQueuedMsg *pMsg, STableMetaMsg *pMeta) { +static int32_t mnodeDoGetChildTableMeta(SMnodeMsg *pMsg, STableMetaMsg *pMeta) { SDbObj *pDb = pMsg->pDb; SChildTableObj *pTable = (SChildTableObj *)pMsg->pTable; @@ -1710,28 +1718,28 @@ static int32_t mgmtDoGetChildTableMeta(SQueuedMsg *pMsg, STableMetaMsg *pMeta) { pMeta->tversion = htons(pTable->superTable->tversion); pMeta->numOfTags = (int8_t)pTable->superTable->numOfTags; pMeta->numOfColumns = htons((int16_t)pTable->superTable->numOfColumns); - pMeta->contLen = sizeof(STableMetaMsg) + mgmtSetSchemaFromSuperTable(pMeta->schema, pTable->superTable); + pMeta->contLen = sizeof(STableMetaMsg) + mnodeSetSchemaFromSuperTable(pMeta->schema, pTable->superTable); } else { pMeta->sversion = htons(pTable->sversion); pMeta->tversion = 0; pMeta->numOfTags = 0; pMeta->numOfColumns = htons((int16_t)pTable->numOfColumns); - pMeta->contLen = sizeof(STableMetaMsg) + mgmtSetSchemaFromNormalTable(pMeta->schema, pTable); + pMeta->contLen = sizeof(STableMetaMsg) + mnodeSetSchemaFromNormalTable(pMeta->schema, pTable); } - if (pMsg->pVgroup == NULL) pMsg->pVgroup = mgmtGetVgroup(pTable->vgId); + if (pMsg->pVgroup == NULL) pMsg->pVgroup = mnodeGetVgroup(pTable->vgId); if (pMsg->pVgroup == NULL) { mError("table:%s, failed to get table meta, db not selected", pTable->info.tableId); return TSDB_CODE_INVALID_VGROUP_ID; } for (int32_t i = 0; i < pMsg->pVgroup->numOfVnodes; ++i) { - SDnodeObj *pDnode = mgmtGetDnode(pMsg->pVgroup->vnodeGid[i].dnodeId); + SDnodeObj *pDnode = mnodeGetDnode(pMsg->pVgroup->vnodeGid[i].dnodeId); if (pDnode == NULL) break; strcpy(pMeta->vgroup.ipAddr[i].fqdn, pDnode->dnodeFqdn); pMeta->vgroup.ipAddr[i].port = htons(pDnode->dnodePort + TSDB_PORT_DNODESHELL); pMeta->vgroup.numOfIps++; - mgmtDecDnodeRef(pDnode); + mnodeDecDnodeRef(pDnode); } pMeta->vgroup.vgId = htonl(pMsg->pVgroup->vgId); @@ -1740,16 +1748,15 @@ static int32_t mgmtDoGetChildTableMeta(SQueuedMsg *pMsg, STableMetaMsg *pMeta) { return TSDB_CODE_SUCCESS; } -static void mgmtAutoCreateChildTable(SQueuedMsg *pMsg) { - SCMTableInfoMsg *pInfo = pMsg->pCont; - STagData* pTag = (STagData*)pInfo->tags; +static int32_t mnodeAutoCreateChildTable(SMnodeMsg *pMsg) { + SCMTableInfoMsg *pInfo = pMsg->rpcMsg.pCont; + STagData *pTag = (STagData *)pInfo->tags; int32_t contLen = sizeof(SCMCreateTableMsg) + offsetof(STagData, data) + ntohl(pTag->dataLen); SCMCreateTableMsg *pCreateMsg = rpcMallocCont(contLen); if (pCreateMsg == NULL) { mError("table:%s, failed to create table while get meta info, no enough memory", pInfo->tableId); - mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_SERV_OUT_OF_MEMORY); - return; + return TSDB_CODE_SERV_OUT_OF_MEMORY; } strncpy(pCreateMsg->tableId, pInfo->tableId, tListLen(pInfo->tableId)); @@ -1759,35 +1766,33 @@ static void mgmtAutoCreateChildTable(SQueuedMsg *pMsg) { pCreateMsg->contLen = htonl(contLen); memcpy(pCreateMsg->schema, pInfo->tags, contLen - sizeof(SCMCreateTableMsg)); + mTrace("table:%s, start to create on demand, stable:%s", pInfo->tableId, ((STagData *)(pCreateMsg->schema))->name); - SQueuedMsg *newMsg = mgmtCloneQueuedMsg(pMsg); - newMsg->msgType = TSDB_MSG_TYPE_CM_CREATE_TABLE; - newMsg->pCont = pCreateMsg; - - mTrace("table:%s, start to create on demand, stable:%s", pInfo->tableId, pInfo->tags); - mgmtAddToShellQueue(newMsg); + rpcFreeCont(pMsg->rpcMsg.pCont); + pMsg->rpcMsg.msgType = TSDB_MSG_TYPE_CM_CREATE_TABLE; + pMsg->rpcMsg.pCont = pCreateMsg; + pMsg->rpcMsg.contLen = contLen; + + return TSDB_CODE_ACTION_NEED_REPROCESSED; } -static void mgmtGetChildTableMeta(SQueuedMsg *pMsg) { +static int32_t mnodeGetChildTableMeta(SMnodeMsg *pMsg) { STableMetaMsg *pMeta = rpcMallocCont(sizeof(STableMetaMsg) + sizeof(SSchema) * (TSDB_MAX_TAGS + TSDB_MAX_COLUMNS + 16)); if (pMeta == NULL) { mError("table:%s, failed to get table meta, no enough memory", pMsg->pTable->tableId); - mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_SERV_OUT_OF_MEMORY); - return; + return TSDB_CODE_SERV_OUT_OF_MEMORY; } - mgmtDoGetChildTableMeta(pMsg, pMeta); + mnodeDoGetChildTableMeta(pMsg, pMeta); - SRpcMsg rpcRsp = { - .handle = pMsg->thandle, - .pCont = pMeta, - .contLen = pMeta->contLen, - }; + pMsg->rpcRsp.len = pMeta->contLen; + pMsg->rpcRsp.rsp = pMeta; pMeta->contLen = htons(pMeta->contLen); - rpcSendResponse(&rpcRsp); + + return TSDB_CODE_SUCCESS; } -void mgmtDropAllChildTablesInVgroups(SVgObj *pVgroup) { +void mnodeDropAllChildTablesInVgroups(SVgObj *pVgroup) { void * pIter = NULL; int32_t numOfTables = 0; SChildTableObj *pTable = NULL; @@ -1795,7 +1800,7 @@ void mgmtDropAllChildTablesInVgroups(SVgObj *pVgroup) { mPrint("vgId:%d, all child tables will be dropped from sdb", pVgroup->vgId); while (1) { - pIter = mgmtGetNextChildTable(pIter, &pTable); + pIter = mnodeGetNextChildTable(pIter, &pTable); if (pTable == NULL) break; if (pTable->vgId == pVgroup->vgId) { @@ -1807,7 +1812,7 @@ void mgmtDropAllChildTablesInVgroups(SVgObj *pVgroup) { sdbDeleteRow(&oper); numOfTables++; } - mgmtDecTableRef(pTable); + mnodeDecTableRef(pTable); } sdbFreeIter(pIter); @@ -1815,7 +1820,7 @@ void mgmtDropAllChildTablesInVgroups(SVgObj *pVgroup) { mPrint("vgId:%d, all child tables is dropped from sdb", pVgroup->vgId); } -void mgmtDropAllChildTables(SDbObj *pDropDb) { +void mnodeDropAllChildTables(SDbObj *pDropDb) { void * pIter = NULL; int32_t numOfTables = 0; int32_t dbNameLen = strlen(pDropDb->name); @@ -1824,7 +1829,7 @@ void mgmtDropAllChildTables(SDbObj *pDropDb) { mPrint("db:%s, all child tables will be dropped from sdb", pDropDb->name); while (1) { - pIter = mgmtGetNextChildTable(pIter, &pTable); + pIter = mnodeGetNextChildTable(pIter, &pTable); if (pTable == NULL) break; if (strncmp(pDropDb->name, pTable->info.tableId, dbNameLen) == 0) { @@ -1836,7 +1841,7 @@ void mgmtDropAllChildTables(SDbObj *pDropDb) { sdbDeleteRow(&oper); numOfTables++; } - mgmtDecTableRef(pTable); + mnodeDecTableRef(pTable); } sdbFreeIter(pIter); @@ -1844,7 +1849,7 @@ void mgmtDropAllChildTables(SDbObj *pDropDb) { mPrint("db:%s, all child tables:%d is dropped from sdb", pDropDb->name, numOfTables); } -static void mgmtDropAllChildTablesInStable(SSuperTableObj *pStable) { +static void mnodeDropAllChildTablesInStable(SSuperTableObj *pStable) { void * pIter = NULL; int32_t numOfTables = 0; SChildTableObj *pTable = NULL; @@ -1852,7 +1857,7 @@ static void mgmtDropAllChildTablesInStable(SSuperTableObj *pStable) { mPrint("stable:%s, all child tables will dropped from sdb", pStable->info.tableId, numOfTables); while (1) { - pIter = mgmtGetNextChildTable(pIter, &pTable); + pIter = mnodeGetNextChildTable(pIter, &pTable); if (pTable == NULL) break; if (pTable->superTable == pStable) { @@ -1865,7 +1870,7 @@ static void mgmtDropAllChildTablesInStable(SSuperTableObj *pStable) { numOfTables++; } - mgmtDecTableRef(pTable); + mnodeDecTableRef(pTable); } sdbFreeIter(pIter); @@ -1873,77 +1878,62 @@ static void mgmtDropAllChildTablesInStable(SSuperTableObj *pStable) { mPrint("stable:%s, all child tables:%d is dropped from sdb", pStable->info.tableId, numOfTables); } -static SChildTableObj* mgmtGetTableByPos(int32_t vnode, int32_t sid) { - SVgObj *pVgroup = mgmtGetVgroup(vnode); +static SChildTableObj* mnodeGetTableByPos(int32_t vnode, int32_t sid) { + SVgObj *pVgroup = mnodeGetVgroup(vnode); if (pVgroup == NULL) return NULL; SChildTableObj *pTable = pVgroup->tableList[sid - 1]; - mgmtIncTableRef((STableObj *)pTable); + mnodeIncTableRef((STableObj *)pTable); - mgmtDecVgroupRef(pVgroup); + mnodeDecVgroupRef(pVgroup); return pTable; } -static void mgmtProcessTableCfgMsg(SRpcMsg *rpcMsg) { - SDMConfigTableMsg *pCfg = (SDMConfigTableMsg *) rpcMsg->pCont; - pCfg->dnode = htonl(pCfg->dnode); - pCfg->vnode = htonl(pCfg->vnode); - pCfg->sid = htonl(pCfg->sid); - mTrace("dnode:%s, vnode:%d, sid:%d, receive table config msg", taosIpStr(pCfg->dnode), pCfg->vnode, pCfg->sid); +static int32_t mnodeProcessTableCfgMsg(SMnodeMsg *pMsg) { + SDMConfigTableMsg *pCfg = pMsg->rpcMsg.pCont; + pCfg->dnodeId = htonl(pCfg->dnodeId); + pCfg->vgId = htonl(pCfg->vgId); + pCfg->sid = htonl(pCfg->sid); + mTrace("dnode:%d, vgId:%d sid:%d, receive table config msg", pCfg->dnodeId, pCfg->vgId, pCfg->sid); - SChildTableObj *pTable = mgmtGetTableByPos(pCfg->vnode, pCfg->sid); + SChildTableObj *pTable = mnodeGetTableByPos(pCfg->vgId, pCfg->sid); if (pTable == NULL) { - mError("dnode:%s, vnode:%d, sid:%d, table not found", taosIpStr(pCfg->dnode), pCfg->vnode, pCfg->sid); - mgmtSendSimpleResp(rpcMsg->handle, TSDB_CODE_NOT_ACTIVE_TABLE); - return; - } - - mgmtSendSimpleResp(rpcMsg->handle, TSDB_CODE_SUCCESS); - - SMDCreateTableMsg *pMDCreate = NULL; - pMDCreate = mgmtBuildCreateChildTableMsg(NULL, (SChildTableObj *) pTable); - if (pMDCreate == NULL) { - mgmtDecTableRef(pTable); - return; + mError("dnode:%d, vgId:%d sid:%d, table not found", pCfg->dnodeId, pCfg->vgId, pCfg->sid); + return TSDB_CODE_INVALID_TABLE_ID; } - SDnodeObj *pDnode = mgmtGetDnode(pCfg->dnode); - SRpcIpSet ipSet = mgmtGetIpSetFromIp(pDnode->dnodeEp); - SRpcMsg rpcRsp = { - .handle = NULL, - .pCont = pMDCreate, - .contLen = htonl(pMDCreate->contLen), - .code = 0, - .msgType = TSDB_MSG_TYPE_MD_CREATE_TABLE - }; - dnodeSendMsgToDnode(&ipSet, &rpcRsp); - - mgmtDecTableRef(pTable); - mgmtDecDnodeRef(pDnode); + SMDCreateTableMsg *pCreate = NULL; + pCreate = mnodeBuildCreateChildTableMsg(NULL, (SChildTableObj *)pTable); + mnodeDecTableRef(pTable); + + if (pCreate == NULL) return terrno; + + pMsg->rpcRsp.rsp = pCreate; + pMsg->rpcRsp.len = htonl(pCreate->contLen); + return TSDB_CODE_SUCCESS; } // handle drop child response -static void mgmtProcessDropChildTableRsp(SRpcMsg *rpcMsg) { +static void mnodeProcessDropChildTableRsp(SRpcMsg *rpcMsg) { if (rpcMsg->handle == NULL) return; - SQueuedMsg *queueMsg = rpcMsg->handle; - queueMsg->received++; + SMnodeMsg *mnodeMsg = rpcMsg->handle; + mnodeMsg->received++; - SChildTableObj *pTable = queueMsg->ahandle; - mPrint("table:%s, drop table rsp received, thandle:%p result:%s", pTable->info.tableId, queueMsg->thandle, tstrerror(rpcMsg->code)); + SChildTableObj *pTable = (SChildTableObj *)mnodeMsg->pTable; + assert(pTable); + mPrint("table:%s, drop table rsp received, thandle:%p result:%s", pTable->info.tableId, mnodeMsg->rpcMsg.handle, tstrerror(rpcMsg->code)); if (rpcMsg->code != TSDB_CODE_SUCCESS) { mError("table:%s, failed to drop in dnode, reason:%s", pTable->info.tableId, tstrerror(rpcMsg->code)); - mgmtSendSimpleResp(queueMsg->thandle, rpcMsg->code); - free(queueMsg); - mgmtDecTableRef(pTable); + dnodeSendRpcMnodeWriteRsp(mnodeMsg, rpcMsg->code); return; } - if (queueMsg->pVgroup == NULL) queueMsg->pVgroup = mgmtGetVgroup(pTable->vgId); - if (queueMsg->pVgroup == NULL) { + if (mnodeMsg->pVgroup == NULL) mnodeMsg->pVgroup = mnodeGetVgroup(pTable->vgId); + if (mnodeMsg->pVgroup == NULL) { mError("table:%s, failed to get vgroup", pTable->info.tableId); - mgmtSendSimpleResp(queueMsg->thandle, TSDB_CODE_INVALID_VGROUP_ID); + dnodeSendRpcMnodeWriteRsp(mnodeMsg, TSDB_CODE_INVALID_VGROUP_ID); return; } @@ -1956,39 +1946,39 @@ static void mgmtProcessDropChildTableRsp(SRpcMsg *rpcMsg) { int32_t code = sdbDeleteRow(&oper); if (code != TSDB_CODE_SUCCESS) { mError("table:%s, update ctables sdb error", pTable->info.tableId); - mgmtSendSimpleResp(queueMsg->thandle, TSDB_CODE_SDB_ERROR); + dnodeSendRpcMnodeWriteRsp(mnodeMsg, TSDB_CODE_SDB_ERROR); return; } - if (queueMsg->pVgroup->numOfTables <= 0) { - mPrint("vgId:%d, all tables is dropped, drop vgroup", queueMsg->pVgroup->vgId); - mgmtDropVgroup(queueMsg->pVgroup, NULL); + if (mnodeMsg->pVgroup->numOfTables <= 0) { + mPrint("vgId:%d, all tables is dropped, drop vgroup", mnodeMsg->pVgroup->vgId); + mnodeDropVgroup(mnodeMsg->pVgroup, NULL); } - mgmtSendSimpleResp(queueMsg->thandle, TSDB_CODE_SUCCESS); - mgmtFreeQueuedMsg(queueMsg); + dnodeSendRpcMnodeWriteRsp(mnodeMsg, TSDB_CODE_SUCCESS); } // handle create table response from dnode // if failed, drop the table cached -static void mgmtProcessCreateChildTableRsp(SRpcMsg *rpcMsg) { +static void mnodeProcessCreateChildTableRsp(SRpcMsg *rpcMsg) { if (rpcMsg->handle == NULL) return; - SQueuedMsg *queueMsg = rpcMsg->handle; - queueMsg->received++; + SMnodeMsg *mnodeMsg = rpcMsg->handle; + mnodeMsg->received++; - SChildTableObj *pTable = queueMsg->ahandle; - mTrace("table:%s, create table rsp received, thandle:%p result:%s", pTable->info.tableId, queueMsg->thandle, + SChildTableObj *pTable = (SChildTableObj *)mnodeMsg->pTable; + assert(pTable); + mTrace("table:%s, create table rsp received, thandle:%p result:%s", pTable->info.tableId, mnodeMsg->rpcMsg.handle, tstrerror(rpcMsg->code)); if (rpcMsg->code != TSDB_CODE_SUCCESS) { - if (queueMsg->retry++ < queueMsg->maxRetry) { + if (mnodeMsg->retry++ < 10) { mTrace("table:%s, create table rsp received, retry:%d thandle:%p result:%s", pTable->info.tableId, - queueMsg->retry, queueMsg->thandle, tstrerror(rpcMsg->code)); - mgmtDealyedAddToShellQueue(queueMsg); + mnodeMsg->retry, mnodeMsg->rpcMsg.handle, tstrerror(rpcMsg->code)); + dnodeDelayReprocessMnodeWriteMsg(mnodeMsg); } else { mError("table:%s, failed to create in dnode, thandle:%p result:%s", pTable->info.tableId, - queueMsg->thandle, tstrerror(rpcMsg->code)); + mnodeMsg->rpcMsg.handle, tstrerror(rpcMsg->code)); SSdbOper oper = { .type = SDB_OPER_GLOBAL, @@ -1997,38 +1987,35 @@ static void mgmtProcessCreateChildTableRsp(SRpcMsg *rpcMsg) { }; sdbDeleteRow(&oper); - mgmtSendSimpleResp(queueMsg->thandle, rpcMsg->code); - mgmtFreeQueuedMsg(queueMsg); + dnodeSendRpcMnodeWriteRsp(mnodeMsg, rpcMsg->code); } } else { - mTrace("table:%s, created in dnode, thandle:%p result:%s", pTable->info.tableId, queueMsg->thandle, + mTrace("table:%s, created in dnode, thandle:%p result:%s", pTable->info.tableId, mnodeMsg->rpcMsg.handle, tstrerror(rpcMsg->code)); - SCMCreateTableMsg *pCreate = queueMsg->pCont; + SCMCreateTableMsg *pCreate = mnodeMsg->rpcMsg.pCont; if (pCreate->getMeta) { mTrace("table:%s, continue to get meta", pTable->info.tableId); - queueMsg->retry = 0; - mgmtAddToShellQueue(queueMsg); + mnodeMsg->retry = 0; + dnodeReprocessMnodeWriteMsg(mnodeMsg); } else { - mgmtSendSimpleResp(queueMsg->thandle, rpcMsg->code); - mgmtFreeQueuedMsg(queueMsg); + dnodeSendRpcMnodeWriteRsp(mnodeMsg, rpcMsg->code); } } } // not implemented yet -static void mgmtProcessAlterTableRsp(SRpcMsg *rpcMsg) { +static void mnodeProcessAlterTableRsp(SRpcMsg *rpcMsg) { mTrace("alter table rsp received, handle:%p code:%s", rpcMsg->handle, tstrerror(rpcMsg->code)); } -static void mgmtProcessMultiTableMetaMsg(SQueuedMsg *pMsg) { - SCMMultiTableInfoMsg *pInfo = pMsg->pCont; +static int32_t mnodeProcessMultiTableMetaMsg(SMnodeMsg *pMsg) { + SCMMultiTableInfoMsg *pInfo = pMsg->rpcMsg.pCont; pInfo->numOfTables = htonl(pInfo->numOfTables); int32_t totalMallocLen = 4 * 1024 * 1024; // first malloc 4 MB, subsequent reallocation as twice SMultiTableMeta *pMultiMeta = rpcMallocCont(totalMallocLen); if (pMultiMeta == NULL) { - mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_SERV_OUT_OF_MEMORY); - return; + return TSDB_CODE_SERV_OUT_OF_MEMORY; } pMultiMeta->contLen = sizeof(SMultiTableMeta); @@ -2036,12 +2023,12 @@ static void mgmtProcessMultiTableMetaMsg(SQueuedMsg *pMsg) { for (int32_t t = 0; t < pInfo->numOfTables; ++t) { char * tableId = (char *)(pInfo->tableIds + t * TSDB_TABLE_ID_LEN + 1); - SChildTableObj *pTable = mgmtGetChildTable(tableId); + SChildTableObj *pTable = mnodeGetChildTable(tableId); if (pTable == NULL) continue; - if (pMsg->pDb == NULL) pMsg->pDb = mgmtGetDbByTableId(tableId); + if (pMsg->pDb == NULL) pMsg->pDb = mnodeGetDbByTableId(tableId); if (pMsg->pDb == NULL) { - mgmtDecTableRef(pTable); + mnodeDecTableRef(pTable); continue; } @@ -2050,35 +2037,33 @@ static void mgmtProcessMultiTableMetaMsg(SQueuedMsg *pMsg) { totalMallocLen *= 2; pMultiMeta = rpcReallocCont(pMultiMeta, totalMallocLen); if (pMultiMeta == NULL) { - mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_SERV_OUT_OF_MEMORY); - mgmtDecTableRef(pTable); - return; + mnodeDecTableRef(pTable); + return TSDB_CODE_SERV_OUT_OF_MEMORY; } else { t--; - mgmtDecTableRef(pTable); + mnodeDecTableRef(pTable); continue; } } STableMetaMsg *pMeta = (STableMetaMsg *)(pMultiMeta->metas + pMultiMeta->contLen); - int32_t code = mgmtDoGetChildTableMeta(pMsg, pMeta); + int32_t code = mnodeDoGetChildTableMeta(pMsg, pMeta); if (code == TSDB_CODE_SUCCESS) { pMultiMeta->numOfTables ++; pMultiMeta->contLen += pMeta->contLen; } - mgmtDecTableRef(pTable); + mnodeDecTableRef(pTable); } - SRpcMsg rpcRsp = {0}; - rpcRsp.handle = pMsg->thandle; - rpcRsp.pCont = pMultiMeta; - rpcRsp.contLen = pMultiMeta->contLen; - rpcSendResponse(&rpcRsp); + pMsg->rpcRsp.rsp = pMultiMeta; + pMsg->rpcRsp.len = pMultiMeta->contLen; + + return TSDB_CODE_SUCCESS; } -static int32_t mgmtGetShowTableMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) { - SDbObj *pDb = mgmtGetDb(pShow->db); +static int32_t mnodeGetShowTableMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) { + SDbObj *pDb = mnodeGetDb(pShow->db); if (pDb == NULL) return TSDB_CODE_DB_NOT_SELECTED; int32_t cols = 0; @@ -2119,11 +2104,11 @@ static int32_t mgmtGetShowTableMeta(STableMetaMsg *pMeta, SShowObj *pShow, void pShow->numOfRows = pDb->numOfTables; pShow->rowSize = pShow->offset[cols - 1] + pShow->bytes[cols - 1]; - mgmtDecDbRef(pDb); + mnodeDecDbRef(pDb); return 0; } -static void mgmtVacuumResult(char *data, int32_t numOfCols, int32_t rows, int32_t capacity, SShowObj *pShow) { +static void mnodeVacuumResult(char *data, int32_t numOfCols, int32_t rows, int32_t capacity, SShowObj *pShow) { if (rows < capacity) { for (int32_t i = 0; i < numOfCols; ++i) { memmove(data + pShow->offset[i] * rows, data + pShow->offset[i] * capacity, pShow->bytes[i] * rows); @@ -2131,8 +2116,8 @@ static void mgmtVacuumResult(char *data, int32_t numOfCols, int32_t rows, int32_ } } -static int32_t mgmtRetrieveShowTables(SShowObj *pShow, char *data, int32_t rows, void *pConn) { - SDbObj *pDb = mgmtGetDb(pShow->db); +static int32_t mnodeRetrieveShowTables(SShowObj *pShow, char *data, int32_t rows, void *pConn) { + SDbObj *pDb = mnodeGetDb(pShow->db); if (pDb == NULL) return 0; int32_t numOfRows = 0; @@ -2145,22 +2130,22 @@ static int32_t mgmtRetrieveShowTables(SShowObj *pShow, char *data, int32_t rows, int32_t prefixLen = strlen(prefix); while (numOfRows < rows) { - pShow->pIter = mgmtGetNextChildTable(pShow->pIter, &pTable); + pShow->pIter = mnodeGetNextChildTable(pShow->pIter, &pTable); if (pTable == NULL) break; // not belong to current db if (strncmp(pTable->info.tableId, prefix, prefixLen)) { - mgmtDecTableRef(pTable); + mnodeDecTableRef(pTable); continue; } char tableName[TSDB_TABLE_NAME_LEN + 1] = {0}; // pattern compare for table name - mgmtExtractTableName(pTable->info.tableId, tableName); + mnodeExtractTableName(pTable->info.tableId, tableName); if (pShow->payloadLen > 0 && patternMatch(pShow->payload, tableName, TSDB_TABLE_NAME_LEN, &info) != TSDB_PATTERN_MATCH) { - mgmtDecTableRef(pTable); + mnodeDecTableRef(pTable); continue; } @@ -2188,47 +2173,44 @@ static int32_t mgmtRetrieveShowTables(SShowObj *pShow, char *data, int32_t rows, memset(tableName, 0, tListLen(tableName)); if (pTable->info.type == TSDB_CHILD_TABLE) { - mgmtExtractTableName(pTable->superTable->info.tableId, tableName); + mnodeExtractTableName(pTable->superTable->info.tableId, tableName); STR_WITH_MAXSIZE_TO_VARSTR(pWrite, tableName, TSDB_TABLE_NAME_LEN); } cols++; numOfRows++; - mgmtDecTableRef(pTable); + mnodeDecTableRef(pTable); } pShow->numOfReads += numOfRows; const int32_t NUM_OF_COLUMNS = 4; - mgmtVacuumResult(data, NUM_OF_COLUMNS, numOfRows, rows, pShow); - mgmtDecDbRef(pDb); + mnodeVacuumResult(data, NUM_OF_COLUMNS, numOfRows, rows, pShow); + mnodeDecDbRef(pDb); return numOfRows; } -static void mgmtProcessAlterTableMsg(SQueuedMsg *pMsg) { - SCMAlterTableMsg *pAlter = pMsg->pCont; - mTrace("table:%s, alter table msg is received from thandle:%p", pAlter->tableId, pMsg->thandle); +static int32_t mnodeProcessAlterTableMsg(SMnodeMsg *pMsg) { + SCMAlterTableMsg *pAlter = pMsg->rpcMsg.pCont; + mTrace("table:%s, alter table msg is received from thandle:%p", pAlter->tableId, pMsg->rpcMsg.handle); - if (pMsg->pDb == NULL) pMsg->pDb = mgmtGetDbByTableId(pAlter->tableId); + if (pMsg->pDb == NULL) pMsg->pDb = mnodeGetDbByTableId(pAlter->tableId); if (pMsg->pDb == NULL || pMsg->pDb->status != TSDB_DB_STATUS_READY) { mError("table:%s, failed to alter table, db not selected", pAlter->tableId); - mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_DB_NOT_SELECTED); - return; + return TSDB_CODE_DB_NOT_SELECTED; } - if (mgmtCheckIsMonitorDB(pMsg->pDb->name, tsMonitorDbName)) { + if (mnodeCheckIsMonitorDB(pMsg->pDb->name, tsMonitorDbName)) { mError("table:%s, failed to alter table, its log db", pAlter->tableId); - mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_MONITOR_DB_FORBIDDEN); - return; + return TSDB_CODE_MONITOR_DB_FORBIDDEN; } - if (pMsg->pTable == NULL) pMsg->pTable = mgmtGetTable(pAlter->tableId); + if (pMsg->pTable == NULL) pMsg->pTable = mnodeGetTable(pAlter->tableId); if (pMsg->pTable == NULL) { mError("table:%s, failed to alter table, table not exist", pMsg->pTable->tableId); - mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_INVALID_TABLE); - return; + return TSDB_CODE_INVALID_TABLE_ID; } pAlter->type = htons(pAlter->type); @@ -2237,8 +2219,7 @@ static void mgmtProcessAlterTableMsg(SQueuedMsg *pMsg) { if (pAlter->numOfCols > 2) { mError("table:%s, error numOfCols:%d in alter table", pAlter->tableId, pAlter->numOfCols); - mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_APP_ERROR); - return; + return TSDB_CODE_APP_ERROR; } for (int32_t i = 0; i < pAlter->numOfCols; ++i) { @@ -2250,15 +2231,15 @@ static void mgmtProcessAlterTableMsg(SQueuedMsg *pMsg) { SSuperTableObj *pTable = (SSuperTableObj *)pMsg->pTable; mTrace("table:%s, start to alter stable", pAlter->tableId); if (pAlter->type == TSDB_ALTER_TABLE_ADD_TAG_COLUMN) { - code = mgmtAddSuperTableTag(pTable, pAlter->schema, 1); + code = mnodeAddSuperTableTag(pTable, pAlter->schema, 1); } else if (pAlter->type == TSDB_ALTER_TABLE_DROP_TAG_COLUMN) { - code = mgmtDropSuperTableTag(pTable, pAlter->schema[0].name); + code = mnodeDropSuperTableTag(pTable, pAlter->schema[0].name); } else if (pAlter->type == TSDB_ALTER_TABLE_CHANGE_TAG_COLUMN) { - code = mgmtModifySuperTableTagName(pTable, pAlter->schema[0].name, pAlter->schema[1].name); + code = mnodeModifySuperTableTagName(pTable, pAlter->schema[0].name, pAlter->schema[1].name); } else if (pAlter->type == TSDB_ALTER_TABLE_ADD_COLUMN) { - code = mgmtAddSuperTableColumn(pMsg->pDb, pTable, pAlter->schema, 1); + code = mnodeAddSuperTableColumn(pMsg->pDb, pTable, pAlter->schema, 1); } else if (pAlter->type == TSDB_ALTER_TABLE_DROP_COLUMN) { - code = mgmtDropSuperTableColumn(pMsg->pDb, pTable, pAlter->schema[0].name); + code = mnodeDropSuperTableColumn(pMsg->pDb, pTable, pAlter->schema[0].name); } else { } } else { @@ -2266,14 +2247,126 @@ static void mgmtProcessAlterTableMsg(SQueuedMsg *pMsg) { SChildTableObj *pTable = (SChildTableObj *)pMsg->pTable; if (pAlter->type == TSDB_ALTER_TABLE_UPDATE_TAG_VAL) { char *tagVal = (char*)(pAlter->schema + pAlter->numOfCols); - code = mgmtModifyChildTableTagValue(pTable, pAlter->schema[0].name, tagVal); + code = mnodeModifyChildTableTagValue(pTable, pAlter->schema[0].name, tagVal); } else if (pAlter->type == TSDB_ALTER_TABLE_ADD_COLUMN) { - code = mgmtAddNormalTableColumn(pMsg->pDb, pTable, pAlter->schema, 1); + code = mnodeAddNormalTableColumn(pMsg->pDb, pTable, pAlter->schema, 1); } else if (pAlter->type == TSDB_ALTER_TABLE_DROP_COLUMN) { - code = mgmtDropNormalTableColumn(pMsg->pDb, pTable, pAlter->schema[0].name); + code = mnodeDropNormalTableColumn(pMsg->pDb, pTable, pAlter->schema[0].name); } else { } } - mgmtSendSimpleResp(pMsg->thandle, code); + return code; +} + +static int32_t mnodeGetStreamMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) { + SDbObj *pDb = mnodeGetDb(pShow->db); + if (pDb == NULL) return TSDB_CODE_DB_NOT_SELECTED; + + int32_t cols = 0; + SSchema *pSchema = pMeta->schema; + + pShow->bytes[cols] = TSDB_TABLE_NAME_LEN + VARSTR_HEADER_SIZE; + pSchema[cols].type = TSDB_DATA_TYPE_BINARY; + strcpy(pSchema[cols].name, "table_name"); + pSchema[cols].bytes = htons(pShow->bytes[cols]); + cols++; + + pShow->bytes[cols] = 8; + pSchema[cols].type = TSDB_DATA_TYPE_TIMESTAMP; + strcpy(pSchema[cols].name, "created_time"); + pSchema[cols].bytes = htons(pShow->bytes[cols]); + cols++; + + pShow->bytes[cols] = 2; + pSchema[cols].type = TSDB_DATA_TYPE_SMALLINT; + strcpy(pSchema[cols].name, "columns"); + pSchema[cols].bytes = htons(pShow->bytes[cols]); + cols++; + + pShow->bytes[cols] = TSDB_MAX_SQL_SHOW_LEN + VARSTR_HEADER_SIZE; + pSchema[cols].type = TSDB_DATA_TYPE_BINARY; + strcpy(pSchema[cols].name, "sql"); + pSchema[cols].bytes = htons(pShow->bytes[cols]); + cols++; + + pMeta->numOfColumns = htons(cols); + pShow->numOfColumns = cols; + + pShow->offset[0] = 0; + for (int32_t i = 1; i < cols; ++i) { + pShow->offset[i] = pShow->offset[i - 1] + pShow->bytes[i - 1]; + } + + pShow->numOfRows = pDb->numOfTables; + pShow->rowSize = pShow->offset[cols - 1] + pShow->bytes[cols - 1]; + + mnodeDecDbRef(pDb); + return 0; +} + +static int32_t mnodeRetrieveStreams(SShowObj *pShow, char *data, int32_t rows, void *pConn) { + SDbObj *pDb = mnodeGetDb(pShow->db); + if (pDb == NULL) return 0; + + + int32_t numOfRows = 0; + SChildTableObj *pTable = NULL; + SPatternCompareInfo info = PATTERN_COMPARE_INFO_INITIALIZER; + + char prefix[64] = {0}; + strcpy(prefix, pDb->name); + strcat(prefix, TS_PATH_DELIMITER); + int32_t prefixLen = strlen(prefix); + + while (numOfRows < rows) { + pShow->pIter = mnodeGetNextChildTable(pShow->pIter, &pTable); + if (pTable == NULL) break; + + // not belong to current db + if (strncmp(pTable->info.tableId, prefix, prefixLen) || pTable->info.type != TSDB_STREAM_TABLE) { + mnodeDecTableRef(pTable); + continue; + } + + char tableName[TSDB_TABLE_NAME_LEN + 1] = {0}; + + // pattern compare for table name + mnodeExtractTableName(pTable->info.tableId, tableName); + + if (pShow->payloadLen > 0 && patternMatch(pShow->payload, tableName, TSDB_TABLE_NAME_LEN, &info) != TSDB_PATTERN_MATCH) { + mnodeDecTableRef(pTable); + continue; + } + + int32_t cols = 0; + + char *pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows; + + STR_WITH_MAXSIZE_TO_VARSTR(pWrite, tableName, TSDB_TABLE_NAME_LEN); + cols++; + + pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows; + *(int64_t *) pWrite = pTable->createdTime; + cols++; + + pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows; + *(int16_t *)pWrite = pTable->numOfColumns; + cols++; + + pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows; + STR_WITH_MAXSIZE_TO_VARSTR(pWrite, pTable->sql, TSDB_MAX_SQL_SHOW_LEN); + cols++; + + numOfRows++; + mnodeDecTableRef(pTable); + } + + pShow->numOfReads += numOfRows; + const int32_t NUM_OF_COLUMNS = 4; + + mnodeVacuumResult(data, NUM_OF_COLUMNS, numOfRows, rows, pShow); + mnodeDecDbRef(pDb); + + return numOfRows; } diff --git a/src/mnode/src/mgmtUser.c b/src/mnode/src/mnodeUser.c similarity index 69% rename from src/mnode/src/mgmtUser.c rename to src/mnode/src/mnodeUser.c index 62a98c41707c11083818ad96af4c2fdb7744ffb4..a1edaaa0a8bb7cb40a0abe948a3f2e951c409e50 100644 --- a/src/mnode/src/mgmtUser.c +++ b/src/mnode/src/mnodeUser.c @@ -21,36 +21,39 @@ #include "tglobal.h" #include "tgrant.h" #include "tdataformat.h" +#include "mnode.h" #include "dnode.h" -#include "mgmtDef.h" -#include "mgmtInt.h" -#include "mgmtAcct.h" -#include "mgmtMnode.h" -#include "mgmtSdb.h" -#include "mgmtShell.h" -#include "mgmtUser.h" +#include "mnodeDef.h" +#include "mnodeInt.h" +#include "mnodeAcct.h" +#include "mnodeMnode.h" +#include "mnodeSdb.h" +#include "mnodeShow.h" +#include "mnodeUser.h" +#include "mnodeWrite.h" +#include "mnodePeer.h" static void * tsUserSdb = NULL; static int32_t tsUserUpdateSize = 0; -static int32_t mgmtGetUserMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn); -static int32_t mgmtRetrieveUsers(SShowObj *pShow, char *data, int32_t rows, void *pConn); -static void mgmtProcessCreateUserMsg(SQueuedMsg *pMsg); -static void mgmtProcessAlterUserMsg(SQueuedMsg *pMsg); -static void mgmtProcessDropUserMsg(SQueuedMsg *pMsg); -static void mgmtProcessAuthMsg(SRpcMsg *rpcMsg); - -static int32_t mgmtUserActionDestroy(SSdbOper *pOper) { +static int32_t mnodeGetUserMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn); +static int32_t mnodeRetrieveUsers(SShowObj *pShow, char *data, int32_t rows, void *pConn); +static int32_t mnodeProcessCreateUserMsg(SMnodeMsg *pMsg); +static int32_t mnodeProcessAlterUserMsg(SMnodeMsg *pMsg); +static int32_t mnodeProcessDropUserMsg(SMnodeMsg *pMsg); +static int32_t mnodeProcessAuthMsg(SMnodeMsg *pMsg); + +static int32_t mnodeUserActionDestroy(SSdbOper *pOper) { tfree(pOper->pObj); return TSDB_CODE_SUCCESS; } -static int32_t mgmtUserActionInsert(SSdbOper *pOper) { +static int32_t mnodeUserActionInsert(SSdbOper *pOper) { SUserObj *pUser = pOper->pObj; - SAcctObj *pAcct = mgmtGetAcct(pUser->acct); + SAcctObj *pAcct = mnodeGetAcct(pUser->acct); if (pAcct != NULL) { - mgmtAddUserToAcct(pAcct, pUser); - mgmtDecAcctRef(pAcct); + mnodeAddUserToAcct(pAcct, pUser); + mnodeDecAcctRef(pAcct); } else { mError("user:%s, acct:%s info not exist in sdb", pUser->user, pUser->acct); return TSDB_CODE_INVALID_ACCT; @@ -59,37 +62,37 @@ static int32_t mgmtUserActionInsert(SSdbOper *pOper) { return TSDB_CODE_SUCCESS; } -static int32_t mgmtUserActionDelete(SSdbOper *pOper) { +static int32_t mnodeUserActionDelete(SSdbOper *pOper) { SUserObj *pUser = pOper->pObj; - SAcctObj *pAcct = mgmtGetAcct(pUser->acct); + SAcctObj *pAcct = mnodeGetAcct(pUser->acct); if (pAcct != NULL) { - mgmtDropUserFromAcct(pAcct, pUser); - mgmtDecAcctRef(pAcct); + mnodeDropUserFromAcct(pAcct, pUser); + mnodeDecAcctRef(pAcct); } return TSDB_CODE_SUCCESS; } -static int32_t mgmtUserActionUpdate(SSdbOper *pOper) { +static int32_t mnodeUserActionUpdate(SSdbOper *pOper) { SUserObj *pUser = pOper->pObj; - SUserObj *pSaved = mgmtGetUser(pUser->user); + SUserObj *pSaved = mnodeGetUser(pUser->user); if (pUser != pSaved) { memcpy(pSaved, pUser, tsUserUpdateSize); free(pUser); } - mgmtDecUserRef(pSaved); + mnodeDecUserRef(pSaved); return TSDB_CODE_SUCCESS; } -static int32_t mgmtUserActionEncode(SSdbOper *pOper) { +static int32_t mnodeUserActionEncode(SSdbOper *pOper) { SUserObj *pUser = pOper->pObj; memcpy(pOper->rowData, pUser, tsUserUpdateSize); pOper->rowSize = tsUserUpdateSize; return TSDB_CODE_SUCCESS; } -static int32_t mgmtUserActionDecode(SSdbOper *pOper) { +static int32_t mnodeUserActionDecode(SSdbOper *pOper) { SUserObj *pUser = (SUserObj *)calloc(1, sizeof(SUserObj)); if (pUser == NULL) return TSDB_CODE_SERV_OUT_OF_MEMORY; @@ -98,19 +101,19 @@ static int32_t mgmtUserActionDecode(SSdbOper *pOper) { return TSDB_CODE_SUCCESS; } -static int32_t mgmtUserActionRestored() { +static int32_t mnodeUserActionRestored() { if (dnodeIsFirstDeploy()) { - SAcctObj *pAcct = mgmtGetAcct("root"); - mgmtCreateUser(pAcct, "root", "taosdata"); - mgmtCreateUser(pAcct, "monitor", tsInternalPass); - mgmtCreateUser(pAcct, "_root", tsInternalPass); - mgmtDecAcctRef(pAcct); + SAcctObj *pAcct = mnodeGetAcct("root"); + mnodeCreateUser(pAcct, "root", "taosdata"); + mnodeCreateUser(pAcct, "monitor", tsInternalPass); + mnodeCreateUser(pAcct, "_root", tsInternalPass); + mnodeDecAcctRef(pAcct); } return TSDB_CODE_SUCCESS; } -int32_t mgmtInitUsers() { +int32_t mnodeInitUsers() { SUserObj tObj; tsUserUpdateSize = (int8_t *)tObj.updateEnd - (int8_t *)&tObj; @@ -121,13 +124,13 @@ int32_t mgmtInitUsers() { .maxRowSize = tsUserUpdateSize, .refCountPos = (int8_t *)(&tObj.refCount) - (int8_t *)&tObj, .keyType = SDB_KEY_STRING, - .insertFp = mgmtUserActionInsert, - .deleteFp = mgmtUserActionDelete, - .updateFp = mgmtUserActionUpdate, - .encodeFp = mgmtUserActionEncode, - .decodeFp = mgmtUserActionDecode, - .destroyFp = mgmtUserActionDestroy, - .restoredFp = mgmtUserActionRestored + .insertFp = mnodeUserActionInsert, + .deleteFp = mnodeUserActionDelete, + .updateFp = mnodeUserActionUpdate, + .encodeFp = mnodeUserActionEncode, + .decodeFp = mnodeUserActionDecode, + .destroyFp = mnodeUserActionDestroy, + .restoredFp = mnodeUserActionRestored }; tsUserSdb = sdbOpenTable(&tableDesc); @@ -136,38 +139,38 @@ int32_t mgmtInitUsers() { return -1; } - mgmtAddShellMsgHandle(TSDB_MSG_TYPE_CM_CREATE_USER, mgmtProcessCreateUserMsg); - mgmtAddShellMsgHandle(TSDB_MSG_TYPE_CM_ALTER_USER, mgmtProcessAlterUserMsg); - mgmtAddShellMsgHandle(TSDB_MSG_TYPE_CM_DROP_USER, mgmtProcessDropUserMsg); - mgmtAddShellShowMetaHandle(TSDB_MGMT_TABLE_USER, mgmtGetUserMeta); - mgmtAddShellShowRetrieveHandle(TSDB_MGMT_TABLE_USER, mgmtRetrieveUsers); - dnodeAddServerMsgHandle(TSDB_MSG_TYPE_DM_AUTH, mgmtProcessAuthMsg); + mnodeAddWriteMsgHandle(TSDB_MSG_TYPE_CM_CREATE_USER, mnodeProcessCreateUserMsg); + mnodeAddWriteMsgHandle(TSDB_MSG_TYPE_CM_ALTER_USER, mnodeProcessAlterUserMsg); + mnodeAddWriteMsgHandle(TSDB_MSG_TYPE_CM_DROP_USER, mnodeProcessDropUserMsg); + mnodeAddShowMetaHandle(TSDB_MGMT_TABLE_USER, mnodeGetUserMeta); + mnodeAddShowRetrieveHandle(TSDB_MGMT_TABLE_USER, mnodeRetrieveUsers); + mnodeAddPeerMsgHandle(TSDB_MSG_TYPE_DM_AUTH, mnodeProcessAuthMsg); mTrace("table:%s, hash is created", tableDesc.tableName); return 0; } -void mgmtCleanUpUsers() { +void mnodeCleanupUsers() { sdbCloseTable(tsUserSdb); } -SUserObj *mgmtGetUser(char *name) { +SUserObj *mnodeGetUser(char *name) { return (SUserObj *)sdbGetRow(tsUserSdb, name); } -void *mgmtGetNextUser(void *pIter, SUserObj **pUser) { +void *mnodeGetNextUser(void *pIter, SUserObj **pUser) { return sdbFetchRow(tsUserSdb, pIter, (void **)pUser); } -void mgmtIncUserRef(SUserObj *pUser) { +void mnodeIncUserRef(SUserObj *pUser) { return sdbIncRef(tsUserSdb, pUser); } -void mgmtDecUserRef(SUserObj *pUser) { +void mnodeDecUserRef(SUserObj *pUser) { return sdbDecRef(tsUserSdb, pUser); } -static int32_t mgmtUpdateUser(SUserObj *pUser) { +static int32_t mnodeUpdateUser(SUserObj *pUser) { SSdbOper oper = { .type = SDB_OPER_GLOBAL, .table = tsUserSdb, @@ -182,7 +185,7 @@ static int32_t mgmtUpdateUser(SUserObj *pUser) { return code; } -int32_t mgmtCreateUser(SAcctObj *pAcct, char *name, char *pass) { +int32_t mnodeCreateUser(SAcctObj *pAcct, char *name, char *pass) { int32_t code = acctCheck(pAcct, ACCT_GRANT_USER); if (code != TSDB_CODE_SUCCESS) { return code; @@ -196,10 +199,10 @@ int32_t mgmtCreateUser(SAcctObj *pAcct, char *name, char *pass) { return TSDB_CODE_INVALID_PASS_FORMAT; } - SUserObj *pUser = mgmtGetUser(name); + SUserObj *pUser = mnodeGetUser(name); if (pUser != NULL) { mTrace("user:%s, is already there", name); - mgmtDecUserRef(pUser); + mnodeDecUserRef(pUser); return TSDB_CODE_USER_ALREADY_EXIST; } @@ -235,7 +238,7 @@ int32_t mgmtCreateUser(SAcctObj *pAcct, char *name, char *pass) { return code; } -static int32_t mgmtDropUser(SUserObj *pUser) { +static int32_t mnodeDropUser(SUserObj *pUser) { SSdbOper oper = { .type = SDB_OPER_GLOBAL, .table = tsUserSdb, @@ -250,8 +253,8 @@ static int32_t mgmtDropUser(SUserObj *pUser) { return code; } -static int32_t mgmtGetUserMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) { - SUserObj *pUser = mgmtGetUserFromConn(pConn); +static int32_t mnodeGetUserMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) { + SUserObj *pUser = mnodeGetUserFromConn(pConn); if (pUser == NULL) { return TSDB_CODE_NO_USER_FROM_CONN; } @@ -289,18 +292,18 @@ static int32_t mgmtGetUserMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pCon pShow->numOfRows = pUser->pAcct->acctInfo.numOfUsers; pShow->rowSize = pShow->offset[cols - 1] + pShow->bytes[cols - 1]; - mgmtDecUserRef(pUser); + mnodeDecUserRef(pUser); return 0; } -static int32_t mgmtRetrieveUsers(SShowObj *pShow, char *data, int32_t rows, void *pConn) { +static int32_t mnodeRetrieveUsers(SShowObj *pShow, char *data, int32_t rows, void *pConn) { int32_t numOfRows = 0; SUserObj *pUser = NULL; int32_t cols = 0; char *pWrite; while (numOfRows < rows) { - pShow->pIter = mgmtGetNextUser(pShow->pIter, &pUser); + pShow->pIter = mnodeGetNextUser(pShow->pIter, &pUser); if (pUser == NULL) break; cols = 0; @@ -327,30 +330,30 @@ static int32_t mgmtRetrieveUsers(SShowObj *pShow, char *data, int32_t rows, void cols++; numOfRows++; - mgmtDecUserRef(pUser); + mnodeDecUserRef(pUser); } pShow->numOfReads += numOfRows; return numOfRows; } -SUserObj *mgmtGetUserFromConn(void *pConn) { +SUserObj *mnodeGetUserFromConn(void *pConn) { SRpcConnInfo connInfo; if (rpcGetConnInfo(pConn, &connInfo) == 0) { - return mgmtGetUser(connInfo.user); + return mnodeGetUser(connInfo.user); } else { mError("can not get user from conn:%p", pConn); return NULL; } } -static void mgmtProcessCreateUserMsg(SQueuedMsg *pMsg) { +static int32_t mnodeProcessCreateUserMsg(SMnodeMsg *pMsg) { int32_t code; SUserObj *pOperUser = pMsg->pUser; if (pOperUser->superAuth) { - SCMCreateUserMsg *pCreate = pMsg->pCont; - code = mgmtCreateUser(pOperUser->pAcct, pCreate->user, pCreate->pass); + SCMCreateUserMsg *pCreate = pMsg->rpcMsg.pCont; + code = mnodeCreateUser(pOperUser->pAcct, pCreate->user, pCreate->pass); if (code == TSDB_CODE_SUCCESS) { mLPrint("user:%s, is created by %s", pCreate->user, pOperUser->user); } @@ -359,24 +362,22 @@ static void mgmtProcessCreateUserMsg(SQueuedMsg *pMsg) { code = TSDB_CODE_NO_RIGHTS; } - mgmtSendSimpleResp(pMsg->thandle, code); + return code; } -static void mgmtProcessAlterUserMsg(SQueuedMsg *pMsg) { +static int32_t mnodeProcessAlterUserMsg(SMnodeMsg *pMsg) { int32_t code; SUserObj *pOperUser = pMsg->pUser; - SCMAlterUserMsg *pAlter = pMsg->pCont; - SUserObj *pUser = mgmtGetUser(pAlter->user); + SCMAlterUserMsg *pAlter = pMsg->rpcMsg.pCont; + SUserObj *pUser = mnodeGetUser(pAlter->user); if (pUser == NULL) { - mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_INVALID_USER); - return; + return TSDB_CODE_INVALID_USER; } if (strcmp(pUser->user, "monitor") == 0 || (strcmp(pUser->user + 1, pUser->acct) == 0 && pUser->user[0] == '_')) { - mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_NO_RIGHTS); - mgmtDecUserRef(pUser); - return; + mnodeDecUserRef(pUser); + return TSDB_CODE_NO_RIGHTS; } if ((pAlter->flag & TSDB_ALTER_USER_PASSWD) != 0) { @@ -398,14 +399,12 @@ static void mgmtProcessAlterUserMsg(SQueuedMsg *pMsg) { if (hasRight) { memset(pUser->pass, 0, sizeof(pUser->pass)); taosEncryptPass((uint8_t*)pAlter->pass, strlen(pAlter->pass), pUser->pass); - code = mgmtUpdateUser(pUser); + code = mnodeUpdateUser(pUser); mLPrint("user:%s, password is altered by %s, result:%s", pUser->user, pOperUser->user, tstrerror(code)); } else { mError("user:%s, no rights to alter user", pOperUser->user); code = TSDB_CODE_NO_RIGHTS; } - - mgmtSendSimpleResp(pMsg->thandle, code); } else if ((pAlter->flag & TSDB_ALTER_USER_PRIVILEGES) != 0) { bool hasRight = false; @@ -441,38 +440,35 @@ static void mgmtProcessAlterUserMsg(SQueuedMsg *pMsg) { pUser->writeAuth = 1; } - code = mgmtUpdateUser(pUser); + code = mnodeUpdateUser(pUser); mLPrint("user:%s, privilege is altered by %s, result:%s", pUser->user, pOperUser->user, tstrerror(code)); } else { mError("user:%s, no rights to alter user", pOperUser->user); code = TSDB_CODE_NO_RIGHTS; } - - mgmtSendSimpleResp(pMsg->thandle, code); } else { mError("user:%s, no rights to alter user", pOperUser->user); - mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_NO_RIGHTS); + code = TSDB_CODE_NO_RIGHTS; } - mgmtDecUserRef(pUser); + mnodeDecUserRef(pUser); + return code; } -static void mgmtProcessDropUserMsg(SQueuedMsg *pMsg) { +static int32_t mnodeProcessDropUserMsg(SMnodeMsg *pMsg) { int32_t code; SUserObj *pOperUser = pMsg->pUser; - SCMDropUserMsg *pDrop = pMsg->pCont; - SUserObj *pUser = mgmtGetUser(pDrop->user); + SCMDropUserMsg *pDrop = pMsg->rpcMsg.pCont; + SUserObj *pUser = mnodeGetUser(pDrop->user); if (pUser == NULL) { - mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_INVALID_USER); - return; + return TSDB_CODE_INVALID_USER; } if (strcmp(pUser->user, "monitor") == 0 || strcmp(pUser->user, pUser->acct) == 0 || (strcmp(pUser->user + 1, pUser->acct) == 0 && pUser->user[0] == '_')) { - mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_NO_RIGHTS); - mgmtDecUserRef(pUser); - return ; + mnodeDecUserRef(pUser); + return TSDB_CODE_NO_RIGHTS; } bool hasRight = false; @@ -491,7 +487,7 @@ static void mgmtProcessDropUserMsg(SQueuedMsg *pMsg) { } if (hasRight) { - code = mgmtDropUser(pUser); + code = mnodeDropUser(pUser); if (code == TSDB_CODE_SUCCESS) { mLPrint("user:%s, is dropped by %s, result:%s", pUser->user, pOperUser->user, tstrerror(code)); } @@ -499,18 +495,18 @@ static void mgmtProcessDropUserMsg(SQueuedMsg *pMsg) { code = TSDB_CODE_NO_RIGHTS; } - mgmtSendSimpleResp(pMsg->thandle, code); - mgmtDecUserRef(pUser); + mnodeDecUserRef(pUser); + return code; } -void mgmtDropAllUsers(SAcctObj *pAcct) { +void mnodeDropAllUsers(SAcctObj *pAcct) { void * pIter = NULL; int32_t numOfUsers = 0; int32_t acctNameLen = strlen(pAcct->user); SUserObj *pUser = NULL; while (1) { - pIter = mgmtGetNextUser(pIter, &pUser); + pIter = mnodeGetNextUser(pIter, &pUser); if (pUser == NULL) break; if (strncmp(pUser->acct, pAcct->user, acctNameLen) == 0) { @@ -523,7 +519,7 @@ void mgmtDropAllUsers(SAcctObj *pAcct) { numOfUsers++; } - mgmtDecUserRef(pUser); + mnodeDecUserRef(pUser); } sdbFreeIter(pIter); @@ -531,14 +527,14 @@ void mgmtDropAllUsers(SAcctObj *pAcct) { mTrace("acct:%s, all users:%d is dropped from sdb", pAcct->user, numOfUsers); } -int32_t mgmtRetriveAuth(char *user, char *spi, char *encrypt, char *secret, char *ckey) { +int32_t mnodeRetriveAuth(char *user, char *spi, char *encrypt, char *secret, char *ckey) { if (!sdbIsMaster()) { *secret = 0; mTrace("user:%s, failed to auth user, reason:%s", user, tstrerror(TSDB_CODE_NOT_READY)); return TSDB_CODE_NOT_READY; } - SUserObj *pUser = mgmtGetUser(user); + SUserObj *pUser = mnodeGetUser(user); if (pUser == NULL) { *secret = 0; mError("user:%s, failed to auth user, reason:%s", user, tstrerror(TSDB_CODE_INVALID_USER)); @@ -549,21 +545,18 @@ int32_t mgmtRetriveAuth(char *user, char *spi, char *encrypt, char *secret, char *ckey = 0; memcpy(secret, pUser->pass, TSDB_KEY_LEN); - mgmtDecUserRef(pUser); + mnodeDecUserRef(pUser); mTrace("user:%s, auth info is returned", user); return TSDB_CODE_SUCCESS; } } -static void mgmtProcessAuthMsg(SRpcMsg *rpcMsg) { - SRpcMsg rpcRsp = {.handle = rpcMsg->handle, .pCont = NULL, .contLen = 0, .code = 0, .msgType = 0}; - - SDMAuthMsg *pAuthMsg = rpcMsg->pCont; +static int32_t mnodeProcessAuthMsg(SMnodeMsg *pMsg) { + SDMAuthMsg *pAuthMsg = pMsg->rpcMsg.pCont; SDMAuthRsp *pAuthRsp = rpcMallocCont(sizeof(SDMAuthRsp)); - rpcRsp.code = mgmtRetriveAuth(pAuthMsg->user, &pAuthRsp->spi, &pAuthRsp->encrypt, pAuthRsp->secret, pAuthRsp->ckey); - rpcRsp.pCont = pAuthRsp; - rpcRsp.contLen = sizeof(SDMAuthRsp); + pMsg->rpcRsp.rsp = pAuthRsp; + pMsg->rpcRsp.len = sizeof(SDMAuthRsp); - rpcSendResponse(&rpcRsp); + return mnodeRetriveAuth(pAuthMsg->user, &pAuthRsp->spi, &pAuthRsp->encrypt, pAuthRsp->secret, pAuthRsp->ckey); } diff --git a/src/mnode/src/mgmtVgroup.c b/src/mnode/src/mnodeVgroup.c similarity index 69% rename from src/mnode/src/mgmtVgroup.c rename to src/mnode/src/mnodeVgroup.c index 960863d665504e0c6d6e6b9cef54581f84c325b2..9c24f7eaf3e3e6d0d88e2a27fe4f9b72db1956f2 100644 --- a/src/mnode/src/mgmtVgroup.c +++ b/src/mnode/src/mnodeVgroup.c @@ -23,30 +23,32 @@ #include "ttime.h" #include "tbalance.h" #include "tglobal.h" -#include "dnode.h" #include "tdataformat.h" -#include "mgmtDef.h" -#include "mgmtInt.h" -#include "mgmtDb.h" -#include "mgmtDnode.h" -#include "mgmtMnode.h" -#include "mgmtProfile.h" -#include "mgmtSdb.h" -#include "mgmtShell.h" -#include "mgmtTable.h" -#include "mgmtVgroup.h" +#include "dnode.h" +#include "mnode.h" +#include "mnodeDef.h" +#include "mnodeInt.h" +#include "mnodeDb.h" +#include "mnodeDnode.h" +#include "mnodeMnode.h" +#include "mnodeProfile.h" +#include "mnodeSdb.h" +#include "mnodeShow.h" +#include "mnodeTable.h" +#include "mnodeVgroup.h" +#include "mnodePeer.h" static void *tsVgroupSdb = NULL; static int32_t tsVgUpdateSize = 0; -static int32_t mgmtGetVgroupMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn); -static int32_t mgmtRetrieveVgroups(SShowObj *pShow, char *data, int32_t rows, void *pConn); -static void mgmtProcessCreateVnodeRsp(SRpcMsg *rpcMsg); -static void mgmtProcessDropVnodeRsp(SRpcMsg *rpcMsg); -static void mgmtProcessVnodeCfgMsg(SRpcMsg *rpcMsg) ; -static void mgmtSendDropVgroupMsg(SVgObj *pVgroup, void *ahandle); +static int32_t mnodeGetVgroupMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn); +static int32_t mnodeRetrieveVgroups(SShowObj *pShow, char *data, int32_t rows, void *pConn); +static void mnodeProcessCreateVnodeRsp(SRpcMsg *rpcMsg); +static void mnodeProcessDropVnodeRsp(SRpcMsg *rpcMsg); +static int32_t mnodeProcessVnodeCfgMsg(SMnodeMsg *pMsg) ; +static void mnodeSendDropVgroupMsg(SVgObj *pVgroup, void *ahandle); -static int32_t mgmtVgroupActionDestroy(SSdbOper *pOper) { +static int32_t mnodeVgroupActionDestroy(SSdbOper *pOper) { SVgObj *pVgroup = pOper->pObj; if (pVgroup->idPool) { taosIdPoolCleanUp(pVgroup->idPool); @@ -60,11 +62,11 @@ static int32_t mgmtVgroupActionDestroy(SSdbOper *pOper) { return TSDB_CODE_SUCCESS; } -static int32_t mgmtVgroupActionInsert(SSdbOper *pOper) { +static int32_t mnodeVgroupActionInsert(SSdbOper *pOper) { SVgObj *pVgroup = pOper->pObj; // refer to db - SDbObj *pDb = mgmtGetDb(pVgroup->dbName); + SDbObj *pDb = mnodeGetDb(pVgroup->dbName); if (pDb == NULL) { return TSDB_CODE_INVALID_DB; } @@ -88,40 +90,40 @@ static int32_t mgmtVgroupActionInsert(SSdbOper *pOper) { } for (int32_t i = 0; i < pVgroup->numOfVnodes; ++i) { - SDnodeObj *pDnode = mgmtGetDnode(pVgroup->vnodeGid[i].dnodeId); + SDnodeObj *pDnode = mnodeGetDnode(pVgroup->vnodeGid[i].dnodeId); if (pDnode != NULL) { pVgroup->vnodeGid[i].pDnode = pDnode; atomic_add_fetch_32(&pDnode->openVnodes, 1); - mgmtDecDnodeRef(pDnode); + mnodeDecDnodeRef(pDnode); } } - mgmtAddVgroupIntoDb(pVgroup); + mnodeAddVgroupIntoDb(pVgroup); return TSDB_CODE_SUCCESS; } -static int32_t mgmtVgroupActionDelete(SSdbOper *pOper) { +static int32_t mnodeVgroupActionDelete(SSdbOper *pOper) { SVgObj *pVgroup = pOper->pObj; if (pVgroup->pDb != NULL) { - mgmtRemoveVgroupFromDb(pVgroup); + mnodeRemoveVgroupFromDb(pVgroup); } - mgmtDecDbRef(pVgroup->pDb); + mnodeDecDbRef(pVgroup->pDb); for (int32_t i = 0; i < pVgroup->numOfVnodes; ++i) { - SDnodeObj *pDnode = mgmtGetDnode(pVgroup->vnodeGid[i].dnodeId); + SDnodeObj *pDnode = mnodeGetDnode(pVgroup->vnodeGid[i].dnodeId); if (pDnode != NULL) { atomic_sub_fetch_32(&pDnode->openVnodes, 1); } - mgmtDecDnodeRef(pDnode); + mnodeDecDnodeRef(pDnode); } return TSDB_CODE_SUCCESS; } -static void mgmtVgroupUpdateIdPool(SVgObj *pVgroup) { +static void mnodeVgroupUpdateIdPool(SVgObj *pVgroup) { int32_t oldTables = taosIdPoolMaxSize(pVgroup->idPool); SDbObj *pDb = pVgroup->pDb; if (pDb != NULL) { @@ -135,9 +137,9 @@ static void mgmtVgroupUpdateIdPool(SVgObj *pVgroup) { } } -static int32_t mgmtVgroupActionUpdate(SSdbOper *pOper) { +static int32_t mnodeVgroupActionUpdate(SSdbOper *pOper) { SVgObj *pNew = pOper->pObj; - SVgObj *pVgroup = mgmtGetVgroup(pNew->vgId); + SVgObj *pVgroup = mnodeGetVgroup(pNew->vgId); if (pVgroup != pNew) { for (int32_t i = 0; i < pVgroup->numOfVnodes; ++i) { @@ -151,24 +153,24 @@ static int32_t mgmtVgroupActionUpdate(SSdbOper *pOper) { free(pNew); for (int32_t i = 0; i < pVgroup->numOfVnodes; ++i) { - SDnodeObj *pDnode = mgmtGetDnode(pVgroup->vnodeGid[i].dnodeId); + SDnodeObj *pDnode = mnodeGetDnode(pVgroup->vnodeGid[i].dnodeId); pVgroup->vnodeGid[i].pDnode = pDnode; if (pDnode != NULL) { atomic_add_fetch_32(&pDnode->openVnodes, 1); } - mgmtDecDnodeRef(pDnode); + mnodeDecDnodeRef(pDnode); } } - mgmtVgroupUpdateIdPool(pVgroup); + mnodeVgroupUpdateIdPool(pVgroup); - mgmtDecVgroupRef(pVgroup); + mnodeDecVgroupRef(pVgroup); mTrace("vgId:%d, is updated, numOfVnode:%d", pVgroup->vgId, pVgroup->numOfVnodes); return TSDB_CODE_SUCCESS; } -static int32_t mgmtVgroupActionEncode(SSdbOper *pOper) { +static int32_t mnodeVgroupActionEncode(SSdbOper *pOper) { SVgObj *pVgroup = pOper->pObj; memcpy(pOper->rowData, pVgroup, tsVgUpdateSize); SVgObj *pTmpVgroup = pOper->rowData; @@ -181,7 +183,7 @@ static int32_t mgmtVgroupActionEncode(SSdbOper *pOper) { return TSDB_CODE_SUCCESS; } -static int32_t mgmtVgroupActionDecode(SSdbOper *pOper) { +static int32_t mnodeVgroupActionDecode(SSdbOper *pOper) { SVgObj *pVgroup = (SVgObj *) calloc(1, sizeof(SVgObj)); if (pVgroup == NULL) return TSDB_CODE_SERV_OUT_OF_MEMORY; @@ -190,11 +192,11 @@ static int32_t mgmtVgroupActionDecode(SSdbOper *pOper) { return TSDB_CODE_SUCCESS; } -static int32_t mgmtVgroupActionRestored() { +static int32_t mnodeVgroupActionRestored() { return 0; } -int32_t mgmtInitVgroups() { +int32_t mnodeInitVgroups() { SVgObj tObj; tsVgUpdateSize = (int8_t *)tObj.updateEnd - (int8_t *)&tObj; @@ -205,13 +207,13 @@ int32_t mgmtInitVgroups() { .maxRowSize = tsVgUpdateSize, .refCountPos = (int8_t *)(&tObj.refCount) - (int8_t *)&tObj, .keyType = SDB_KEY_AUTO, - .insertFp = mgmtVgroupActionInsert, - .deleteFp = mgmtVgroupActionDelete, - .updateFp = mgmtVgroupActionUpdate, - .encodeFp = mgmtVgroupActionEncode, - .decodeFp = mgmtVgroupActionDecode, - .destroyFp = mgmtVgroupActionDestroy, - .restoredFp = mgmtVgroupActionRestored, + .insertFp = mnodeVgroupActionInsert, + .deleteFp = mnodeVgroupActionDelete, + .updateFp = mnodeVgroupActionUpdate, + .encodeFp = mnodeVgroupActionEncode, + .decodeFp = mnodeVgroupActionDecode, + .destroyFp = mnodeVgroupActionDestroy, + .restoredFp = mnodeVgroupActionRestored, }; tsVgroupSdb = sdbOpenTable(&tableDesc); @@ -220,30 +222,30 @@ int32_t mgmtInitVgroups() { return -1; } - mgmtAddShellShowMetaHandle(TSDB_MGMT_TABLE_VGROUP, mgmtGetVgroupMeta); - mgmtAddShellShowRetrieveHandle(TSDB_MGMT_TABLE_VGROUP, mgmtRetrieveVgroups); - dnodeAddClientRspHandle(TSDB_MSG_TYPE_MD_CREATE_VNODE_RSP, mgmtProcessCreateVnodeRsp); - dnodeAddClientRspHandle(TSDB_MSG_TYPE_MD_DROP_VNODE_RSP, mgmtProcessDropVnodeRsp); - dnodeAddServerMsgHandle(TSDB_MSG_TYPE_DM_CONFIG_VNODE, mgmtProcessVnodeCfgMsg); + mnodeAddShowMetaHandle(TSDB_MGMT_TABLE_VGROUP, mnodeGetVgroupMeta); + mnodeAddShowRetrieveHandle(TSDB_MGMT_TABLE_VGROUP, mnodeRetrieveVgroups); + mnodeAddPeerRspHandle(TSDB_MSG_TYPE_MD_CREATE_VNODE_RSP, mnodeProcessCreateVnodeRsp); + mnodeAddPeerRspHandle(TSDB_MSG_TYPE_MD_DROP_VNODE_RSP, mnodeProcessDropVnodeRsp); + mnodeAddPeerMsgHandle(TSDB_MSG_TYPE_DM_CONFIG_VNODE, mnodeProcessVnodeCfgMsg); mTrace("table:vgroups is created"); return 0; } -void mgmtIncVgroupRef(SVgObj *pVgroup) { +void mnodeIncVgroupRef(SVgObj *pVgroup) { return sdbIncRef(tsVgroupSdb, pVgroup); } -void mgmtDecVgroupRef(SVgObj *pVgroup) { +void mnodeDecVgroupRef(SVgObj *pVgroup) { return sdbDecRef(tsVgroupSdb, pVgroup); } -SVgObj *mgmtGetVgroup(int32_t vgId) { +SVgObj *mnodeGetVgroup(int32_t vgId) { return (SVgObj *)sdbGetRow(tsVgroupSdb, &vgId); } -void mgmtUpdateVgroup(SVgObj *pVgroup) { +void mnodeUpdateVgroup(SVgObj *pVgroup) { SSdbOper oper = { .type = SDB_OPER_GLOBAL, .table = tsVgroupSdb, @@ -251,10 +253,10 @@ void mgmtUpdateVgroup(SVgObj *pVgroup) { }; sdbUpdateRow(&oper); - mgmtSendCreateVgroupMsg(pVgroup, NULL); + mnodeSendCreateVgroupMsg(pVgroup, NULL); } -void mgmtUpdateVgroupStatus(SVgObj *pVgroup, SDnodeObj *pDnode, SVnodeLoad *pVload) { +void mnodeUpdateVgroupStatus(SVgObj *pVgroup, SDnodeObj *pDnode, SVnodeLoad *pVload) { bool dnodeExist = false; for (int32_t i = 0; i < pVgroup->numOfVnodes; ++i) { SVnodeGid *pVgid = &pVgroup->vnodeGid[i]; @@ -269,9 +271,9 @@ void mgmtUpdateVgroupStatus(SVgObj *pVgroup, SDnodeObj *pDnode, SVnodeLoad *pVlo } if (!dnodeExist) { - SRpcIpSet ipSet = mgmtGetIpSetFromIp(pDnode->dnodeEp); + SRpcIpSet ipSet = mnodeGetIpSetFromIp(pDnode->dnodeEp); mError("vgId:%d, dnode:%d not exist in mnode, drop it", pVload->vgId, pDnode->dnodeId); - mgmtSendDropVnodeMsg(pVload->vgId, &ipSet, NULL); + mnodeSendDropVnodeMsg(pVload->vgId, &ipSet, NULL); return; } @@ -282,22 +284,22 @@ void mgmtUpdateVgroupStatus(SVgObj *pVgroup, SDnodeObj *pDnode, SVnodeLoad *pVlo } if (pVload->cfgVersion != pVgroup->pDb->cfgVersion || pVload->replica != pVgroup->numOfVnodes) { - mError("dnode:%d, vgId:%d, vnode cfgVersion:%d repica:%d not match with mgmt cfgVersion:%d replica:%d", + mError("dnode:%d, vgId:%d, vnode cfgVersion:%d repica:%d not match with mnode cfgVersion:%d replica:%d", pDnode->dnodeId, pVload->vgId, pVload->cfgVersion, pVload->replica, pVgroup->pDb->cfgVersion, pVgroup->numOfVnodes); - mgmtSendCreateVgroupMsg(pVgroup, NULL); + mnodeSendCreateVgroupMsg(pVgroup, NULL); } } -SVgObj *mgmtGetAvailableVgroup(SDbObj *pDb) { +SVgObj *mnodeGetAvailableVgroup(SDbObj *pDb) { return pDb->pHead; } -void *mgmtGetNextVgroup(void *pIter, SVgObj **pVgroup) { +void *mnodeGetNextVgroup(void *pIter, SVgObj **pVgroup) { return sdbFetchRow(tsVgroupSdb, pIter, (void **)pVgroup); } -void mgmtCreateVgroup(SQueuedMsg *pMsg, SDbObj *pDb) { +int32_t mnodeCreateVgroup(SMnodeMsg *pMsg, SDbObj *pDb) { SVgObj *pVgroup = (SVgObj *)calloc(1, sizeof(SVgObj)); strcpy(pVgroup->dbName, pDb->name); pVgroup->numOfVnodes = pDb->cfg.replications; @@ -305,9 +307,7 @@ void mgmtCreateVgroup(SQueuedMsg *pMsg, SDbObj *pDb) { if (balanceAllocVnodes(pVgroup) != 0) { mError("db:%s, no enough dnode to alloc %d vnodes to vgroup", pDb->name, pVgroup->numOfVnodes); free(pVgroup); - mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_NO_ENOUGH_DNODES); - mgmtFreeQueuedMsg(pMsg); - return; + return TSDB_CODE_NO_ENOUGH_DNODES; } SSdbOper oper = { @@ -320,10 +320,7 @@ void mgmtCreateVgroup(SQueuedMsg *pMsg, SDbObj *pDb) { int32_t code = sdbInsertRow(&oper); if (code != TSDB_CODE_SUCCESS) { tfree(pVgroup); - code = TSDB_CODE_SDB_ERROR; - mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_SDB_ERROR); - mgmtFreeQueuedMsg(pMsg); - return; + return TSDB_CODE_SDB_ERROR; } mPrint("vgId:%d, is created in mnode, db:%s replica:%d", pVgroup->vgId, pDb->name, pVgroup->numOfVnodes); @@ -331,17 +328,20 @@ void mgmtCreateVgroup(SQueuedMsg *pMsg, SDbObj *pDb) { mPrint("vgId:%d, index:%d, dnode:%d", pVgroup->vgId, i, pVgroup->vnodeGid[i].dnodeId); } - pMsg->ahandle = pVgroup; + mnodeIncVgroupRef(pVgroup); + pMsg->pVgroup = pVgroup; pMsg->expected = pVgroup->numOfVnodes; - mgmtSendCreateVgroupMsg(pVgroup, pMsg); + mnodeSendCreateVgroupMsg(pVgroup, pMsg); + + return TSDB_CODE_ACTION_IN_PROGRESS; } -void mgmtDropVgroup(SVgObj *pVgroup, void *ahandle) { +void mnodeDropVgroup(SVgObj *pVgroup, void *ahandle) { if (ahandle != NULL) { - mgmtSendDropVgroupMsg(pVgroup, ahandle); + mnodeSendDropVgroupMsg(pVgroup, ahandle); } else { mTrace("vgId:%d, replica:%d is deleting from sdb", pVgroup->vgId, pVgroup->numOfVnodes); - mgmtSendDropVgroupMsg(pVgroup, NULL); + mnodeSendDropVgroupMsg(pVgroup, NULL); SSdbOper oper = { .type = SDB_OPER_GLOBAL, .table = tsVgroupSdb, @@ -351,12 +351,12 @@ void mgmtDropVgroup(SVgObj *pVgroup, void *ahandle) { } } -void mgmtCleanUpVgroups() { +void mnodeCleanupVgroups() { sdbCloseTable(tsVgroupSdb); } -int32_t mgmtGetVgroupMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) { - SDbObj *pDb = mgmtGetDb(pShow->db); +int32_t mnodeGetVgroupMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) { + SDbObj *pDb = mnodeGetDb(pShow->db); if (pDb == NULL) { return TSDB_CODE_DB_NOT_SELECTED; } @@ -380,15 +380,15 @@ int32_t mgmtGetVgroupMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) { SVgObj *pVgroup = NULL; STableObj *pTable = NULL; if (pShow->payloadLen > 0 ) { - pTable = mgmtGetTable(pShow->payload); + pTable = mnodeGetTable(pShow->payload); if (NULL == pTable || pTable->type == TSDB_SUPER_TABLE) { - mgmtDecTableRef(pTable); + mnodeDecTableRef(pTable); return TSDB_CODE_INVALID_TABLE_ID; } - mgmtDecTableRef(pTable); - pVgroup = mgmtGetVgroup(((SChildTableObj*)pTable)->vgId); + mnodeDecTableRef(pTable); + pVgroup = mnodeGetVgroup(((SChildTableObj*)pTable)->vgId); if (NULL == pVgroup) return TSDB_CODE_INVALID_TABLE_ID; - mgmtDecVgroupRef(pVgroup); + mnodeDecVgroupRef(pVgroup); maxReplica = pVgroup->numOfVnodes > maxReplica ? pVgroup->numOfVnodes : maxReplica; } else { SVgObj *pVgroup = pDb->pHead; @@ -434,19 +434,19 @@ int32_t mgmtGetVgroupMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) { pShow->pIter = pVgroup; } - mgmtDecDbRef(pDb); + mnodeDecDbRef(pDb); return 0; } -int32_t mgmtRetrieveVgroups(SShowObj *pShow, char *data, int32_t rows, void *pConn) { +int32_t mnodeRetrieveVgroups(SShowObj *pShow, char *data, int32_t rows, void *pConn) { int32_t numOfRows = 0; SVgObj *pVgroup = NULL; int32_t maxReplica = 0; int32_t cols = 0; char * pWrite; - SDbObj *pDb = mgmtGetDb(pShow->db); + SDbObj *pDb = mnodeGetDb(pShow->db); if (pDb == NULL) return 0; pVgroup = pDb->pHead; @@ -483,7 +483,7 @@ int32_t mgmtRetrieveVgroups(SShowObj *pShow, char *data, int32_t rows, void *pCo cols++; pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows; - char *role = mgmtGetMnodeRoleStr(pVgroup->vnodeGid[i].role); + char *role = mnodeGetMnodeRoleStr(pVgroup->vnodeGid[i].role); STR_TO_VARSTR(pWrite, role); cols++; } else { @@ -502,12 +502,12 @@ int32_t mgmtRetrieveVgroups(SShowObj *pShow, char *data, int32_t rows, void *pCo } pShow->numOfReads += numOfRows; - mgmtDecDbRef(pDb); + mnodeDecDbRef(pDb); return numOfRows; } -void mgmtAddTableIntoVgroup(SVgObj *pVgroup, SChildTableObj *pTable) { +void mnodeAddTableIntoVgroup(SVgObj *pVgroup, SChildTableObj *pTable) { if (pTable->sid >= 1 && pVgroup->tableList[pTable->sid - 1] == NULL) { pVgroup->tableList[pTable->sid - 1] = pTable; taosIdPoolMarkStatus(pVgroup->idPool, pTable->sid); @@ -515,24 +515,24 @@ void mgmtAddTableIntoVgroup(SVgObj *pVgroup, SChildTableObj *pTable) { } if (pVgroup->numOfTables >= pVgroup->pDb->cfg.maxTables) { - mgmtMoveVgroupToTail(pVgroup); + mnodeMoveVgroupToTail(pVgroup); } - mgmtIncVgroupRef(pVgroup); + mnodeIncVgroupRef(pVgroup); } -void mgmtRemoveTableFromVgroup(SVgObj *pVgroup, SChildTableObj *pTable) { +void mnodeRemoveTableFromVgroup(SVgObj *pVgroup, SChildTableObj *pTable) { if (pTable->sid >= 1 && pVgroup->tableList[pTable->sid - 1] != NULL) { pVgroup->tableList[pTable->sid - 1] = NULL; taosFreeId(pVgroup->idPool, pTable->sid); pVgroup->numOfTables--; } - mgmtMoveVgroupToHead(pVgroup); - mgmtDecVgroupRef(pVgroup); + mnodeMoveVgroupToHead(pVgroup); + mnodeDecVgroupRef(pVgroup); } -SMDCreateVnodeMsg *mgmtBuildCreateVnodeMsg(SVgObj *pVgroup) { +SMDCreateVnodeMsg *mnodeBuildCreateVnodeMsg(SVgObj *pVgroup) { SDbObj *pDb = pVgroup->pDb; if (pDb == NULL) return NULL; @@ -571,7 +571,7 @@ SMDCreateVnodeMsg *mgmtBuildCreateVnodeMsg(SVgObj *pVgroup) { return pVnode; } -SRpcIpSet mgmtGetIpSetFromVgroup(SVgObj *pVgroup) { +SRpcIpSet mnodeGetIpSetFromVgroup(SVgObj *pVgroup) { SRpcIpSet ipSet = { .numOfIps = pVgroup->numOfVnodes, .inUse = 0, @@ -583,7 +583,7 @@ SRpcIpSet mgmtGetIpSetFromVgroup(SVgObj *pVgroup) { return ipSet; } -SRpcIpSet mgmtGetIpSetFromIp(char *ep) { +SRpcIpSet mnodeGetIpSetFromIp(char *ep) { SRpcIpSet ipSet; ipSet.numOfIps = 1; @@ -593,9 +593,9 @@ SRpcIpSet mgmtGetIpSetFromIp(char *ep) { return ipSet; } -void mgmtSendCreateVnodeMsg(SVgObj *pVgroup, SRpcIpSet *ipSet, void *ahandle) { +void mnodeSendCreateVnodeMsg(SVgObj *pVgroup, SRpcIpSet *ipSet, void *ahandle) { mTrace("vgId:%d, send create vnode:%d msg, ahandle:%p", pVgroup->vgId, pVgroup->vgId, ahandle); - SMDCreateVnodeMsg *pCreate = mgmtBuildCreateVnodeMsg(pVgroup); + SMDCreateVnodeMsg *pCreate = mnodeBuildCreateVnodeMsg(pVgroup); SRpcMsg rpcMsg = { .handle = ahandle, .pCont = pCreate, @@ -606,34 +606,33 @@ void mgmtSendCreateVnodeMsg(SVgObj *pVgroup, SRpcIpSet *ipSet, void *ahandle) { dnodeSendMsgToDnode(ipSet, &rpcMsg); } -void mgmtSendCreateVgroupMsg(SVgObj *pVgroup, void *ahandle) { +void mnodeSendCreateVgroupMsg(SVgObj *pVgroup, void *ahandle) { mTrace("vgId:%d, send create all vnodes msg, ahandle:%p", pVgroup->vgId, ahandle); for (int32_t i = 0; i < pVgroup->numOfVnodes; ++i) { - SRpcIpSet ipSet = mgmtGetIpSetFromIp(pVgroup->vnodeGid[i].pDnode->dnodeEp); - mgmtSendCreateVnodeMsg(pVgroup, &ipSet, ahandle); + SRpcIpSet ipSet = mnodeGetIpSetFromIp(pVgroup->vnodeGid[i].pDnode->dnodeEp); + mnodeSendCreateVnodeMsg(pVgroup, &ipSet, ahandle); } } -static void mgmtProcessCreateVnodeRsp(SRpcMsg *rpcMsg) { +static void mnodeProcessCreateVnodeRsp(SRpcMsg *rpcMsg) { if (rpcMsg->handle == NULL) return; - SQueuedMsg *queueMsg = rpcMsg->handle; - queueMsg->received++; + SMnodeMsg *mnodeMsg = rpcMsg->handle; + mnodeMsg->received++; if (rpcMsg->code == TSDB_CODE_SUCCESS) { - queueMsg->code = rpcMsg->code; - queueMsg->successed++; + mnodeMsg->code = rpcMsg->code; + mnodeMsg->successed++; } - SVgObj *pVgroup = queueMsg->ahandle; + SVgObj *pVgroup = mnodeMsg->pVgroup; mTrace("vgId:%d, create vnode rsp received, result:%s received:%d successed:%d expected:%d, thandle:%p ahandle:%p", - pVgroup->vgId, tstrerror(rpcMsg->code), queueMsg->received, queueMsg->successed, queueMsg->expected, - queueMsg->thandle, rpcMsg->handle); + pVgroup->vgId, tstrerror(rpcMsg->code), mnodeMsg->received, mnodeMsg->successed, mnodeMsg->expected, + mnodeMsg->rpcMsg.handle, rpcMsg->handle); - if (queueMsg->received != queueMsg->expected) return; + if (mnodeMsg->received != mnodeMsg->expected) return; - if (queueMsg->received == queueMsg->successed) { - SQueuedMsg *newMsg = mgmtCloneQueuedMsg(queueMsg); - mgmtAddToShellQueue(newMsg); + if (mnodeMsg->received == mnodeMsg->successed) { + dnodeReprocessMnodeWriteMsg(mnodeMsg); } else { SSdbOper oper = { .type = SDB_OPER_GLOBAL, @@ -644,14 +643,12 @@ static void mgmtProcessCreateVnodeRsp(SRpcMsg *rpcMsg) { if (code != 0) { code = TSDB_CODE_SDB_ERROR; } - - mgmtSendSimpleResp(queueMsg->thandle, rpcMsg->code); - } - mgmtFreeQueuedMsg(queueMsg); + dnodeSendRpcMnodeWriteRsp(mnodeMsg, code); + } } -static SMDDropVnodeMsg *mgmtBuildDropVnodeMsg(int32_t vgId) { +static SMDDropVnodeMsg *mnodeBuildDropVnodeMsg(int32_t vgId) { SMDDropVnodeMsg *pDrop = rpcMallocCont(sizeof(SMDDropVnodeMsg)); if (pDrop == NULL) return NULL; @@ -659,8 +656,8 @@ static SMDDropVnodeMsg *mgmtBuildDropVnodeMsg(int32_t vgId) { return pDrop; } -void mgmtSendDropVnodeMsg(int32_t vgId, SRpcIpSet *ipSet, void *ahandle) { - SMDDropVnodeMsg *pDrop = mgmtBuildDropVnodeMsg(vgId); +void mnodeSendDropVnodeMsg(int32_t vgId, SRpcIpSet *ipSet, void *ahandle) { + SMDDropVnodeMsg *pDrop = mnodeBuildDropVnodeMsg(vgId); SRpcMsg rpcMsg = { .handle = ahandle, .pCont = pDrop, @@ -671,32 +668,32 @@ void mgmtSendDropVnodeMsg(int32_t vgId, SRpcIpSet *ipSet, void *ahandle) { dnodeSendMsgToDnode(ipSet, &rpcMsg); } -static void mgmtSendDropVgroupMsg(SVgObj *pVgroup, void *ahandle) { +static void mnodeSendDropVgroupMsg(SVgObj *pVgroup, void *ahandle) { mTrace("vgId:%d, send drop all vnodes msg, ahandle:%p", pVgroup->vgId, ahandle); for (int32_t i = 0; i < pVgroup->numOfVnodes; ++i) { - SRpcIpSet ipSet = mgmtGetIpSetFromIp(pVgroup->vnodeGid[i].pDnode->dnodeEp); + SRpcIpSet ipSet = mnodeGetIpSetFromIp(pVgroup->vnodeGid[i].pDnode->dnodeEp); mTrace("vgId:%d, send drop vnode msg to dnode:%d, ahandle:%p", pVgroup->vgId, pVgroup->vnodeGid[i].dnodeId, ahandle); - mgmtSendDropVnodeMsg(pVgroup->vgId, &ipSet, ahandle); + mnodeSendDropVnodeMsg(pVgroup->vgId, &ipSet, ahandle); } } -static void mgmtProcessDropVnodeRsp(SRpcMsg *rpcMsg) { +static void mnodeProcessDropVnodeRsp(SRpcMsg *rpcMsg) { mTrace("drop vnode rsp is received, handle:%p", rpcMsg->handle); if (rpcMsg->handle == NULL) return; - SQueuedMsg *queueMsg = rpcMsg->handle; - queueMsg->received++; + SMnodeMsg *mnodeMsg = rpcMsg->handle; + mnodeMsg->received++; if (rpcMsg->code == TSDB_CODE_SUCCESS) { - queueMsg->code = rpcMsg->code; - queueMsg->successed++; + mnodeMsg->code = rpcMsg->code; + mnodeMsg->successed++; } - SVgObj *pVgroup = queueMsg->ahandle; + SVgObj *pVgroup = mnodeMsg->pVgroup; mTrace("vgId:%d, drop vnode rsp received, result:%s received:%d successed:%d expected:%d, thandle:%p ahandle:%p", - pVgroup->vgId, tstrerror(rpcMsg->code), queueMsg->received, queueMsg->successed, queueMsg->expected, - queueMsg->thandle, rpcMsg->handle); + pVgroup->vgId, tstrerror(rpcMsg->code), mnodeMsg->received, mnodeMsg->successed, mnodeMsg->expected, + mnodeMsg->rpcMsg.handle, rpcMsg->handle); - if (queueMsg->received != queueMsg->expected) return; + if (mnodeMsg->received != mnodeMsg->expected) return; SSdbOper oper = { .type = SDB_OPER_GLOBAL, @@ -708,41 +705,35 @@ static void mgmtProcessDropVnodeRsp(SRpcMsg *rpcMsg) { code = TSDB_CODE_SDB_ERROR; } - SQueuedMsg *newMsg = mgmtCloneQueuedMsg(queueMsg); - mgmtAddToShellQueue(newMsg); - - queueMsg->pCont = NULL; - mgmtFreeQueuedMsg(queueMsg); + dnodeReprocessMnodeWriteMsg(mnodeMsg); } -static void mgmtProcessVnodeCfgMsg(SRpcMsg *rpcMsg) { - SDMConfigVnodeMsg *pCfg = (SDMConfigVnodeMsg *) rpcMsg->pCont; +static int32_t mnodeProcessVnodeCfgMsg(SMnodeMsg *pMsg) { + SDMConfigVnodeMsg *pCfg = pMsg->rpcMsg.pCont; pCfg->dnodeId = htonl(pCfg->dnodeId); pCfg->vgId = htonl(pCfg->vgId); - SDnodeObj *pDnode = mgmtGetDnode(pCfg->dnodeId); + SDnodeObj *pDnode = mnodeGetDnode(pCfg->dnodeId); if (pDnode == NULL) { mTrace("dnode:%s, invalid dnode", taosIpStr(pCfg->dnodeId), pCfg->vgId); - mgmtSendSimpleResp(rpcMsg->handle, TSDB_CODE_NOT_ACTIVE_VNODE); - return; + return TSDB_CODE_INVALID_VGROUP_ID; } - mgmtDecDnodeRef(pDnode); + mnodeDecDnodeRef(pDnode); - SVgObj *pVgroup = mgmtGetVgroup(pCfg->vgId); + SVgObj *pVgroup = mnodeGetVgroup(pCfg->vgId); if (pVgroup == NULL) { mTrace("dnode:%s, vgId:%d, no vgroup info", taosIpStr(pCfg->dnodeId), pCfg->vgId); - mgmtSendSimpleResp(rpcMsg->handle, TSDB_CODE_NOT_ACTIVE_VNODE); - return; + return TSDB_CODE_INVALID_VGROUP_ID; } - mgmtDecVgroupRef(pVgroup); + mnodeDecVgroupRef(pVgroup); - mgmtSendSimpleResp(rpcMsg->handle, TSDB_CODE_SUCCESS); + SRpcIpSet ipSet = mnodeGetIpSetFromIp(pDnode->dnodeEp); + mnodeSendCreateVnodeMsg(pVgroup, &ipSet, NULL); - SRpcIpSet ipSet = mgmtGetIpSetFromIp(pDnode->dnodeEp); - mgmtSendCreateVnodeMsg(pVgroup, &ipSet, NULL); + return TSDB_CODE_SUCCESS; } -void mgmtDropAllDnodeVgroups(SDnodeObj *pDropDnode) { +void mnodeDropAllDnodeVgroups(SDnodeObj *pDropDnode) { void * pIter = NULL; SVgObj *pVgroup = NULL; int32_t numOfVgroups = 0; @@ -750,11 +741,11 @@ void mgmtDropAllDnodeVgroups(SDnodeObj *pDropDnode) { mPrint("dnode:%d, all vgroups will be dropped from sdb", pDropDnode->dnodeId); while (1) { - pIter = mgmtGetNextVgroup(pIter, &pVgroup); + pIter = mnodeGetNextVgroup(pIter, &pVgroup); if (pVgroup == NULL) break; if (pVgroup->vnodeGid[0].dnodeId == pDropDnode->dnodeId) { - mgmtDropAllChildTablesInVgroups(pVgroup); + mnodeDropAllChildTablesInVgroups(pVgroup); SSdbOper oper = { .type = SDB_OPER_LOCAL, .table = tsVgroupSdb, @@ -763,7 +754,7 @@ void mgmtDropAllDnodeVgroups(SDnodeObj *pDropDnode) { sdbDeleteRow(&oper); numOfVgroups++; } - mgmtDecVgroupRef(pVgroup); + mnodeDecVgroupRef(pVgroup); } sdbFreeIter(pIter); @@ -771,21 +762,21 @@ void mgmtDropAllDnodeVgroups(SDnodeObj *pDropDnode) { mPrint("dnode:%d, all vgroups is dropped from sdb", pDropDnode->dnodeId); } -void mgmtUpdateAllDbVgroups(SDbObj *pAlterDb) { +void mnodeUpdateAllDbVgroups(SDbObj *pAlterDb) { void * pIter = NULL; SVgObj *pVgroup = NULL; mPrint("db:%s, all vgroups will be update in sdb", pAlterDb->name); while (1) { - pIter = mgmtGetNextVgroup(pIter, &pVgroup); + pIter = mnodeGetNextVgroup(pIter, &pVgroup); if (pVgroup == NULL) break; if (pVgroup->pDb == pAlterDb) { - mgmtVgroupUpdateIdPool(pVgroup); + mnodeVgroupUpdateIdPool(pVgroup); } - mgmtDecVgroupRef(pVgroup); + mnodeDecVgroupRef(pVgroup); } sdbFreeIter(pIter); @@ -793,14 +784,14 @@ void mgmtUpdateAllDbVgroups(SDbObj *pAlterDb) { mPrint("db:%s, all vgroups is updated in sdb", pAlterDb->name); } -void mgmtDropAllDbVgroups(SDbObj *pDropDb, bool sendMsg) { +void mnodeDropAllDbVgroups(SDbObj *pDropDb) { void * pIter = NULL; int32_t numOfVgroups = 0; SVgObj *pVgroup = NULL; mPrint("db:%s, all vgroups will be dropped from sdb", pDropDb->name); while (1) { - pIter = mgmtGetNextVgroup(pIter, &pVgroup); + pIter = mnodeGetNextVgroup(pIter, &pVgroup); if (pVgroup == NULL) break; if (pVgroup->pDb == pDropDb) { @@ -811,16 +802,34 @@ void mgmtDropAllDbVgroups(SDbObj *pDropDb, bool sendMsg) { }; sdbDeleteRow(&oper); numOfVgroups++; - - if (sendMsg) { - mgmtSendDropVgroupMsg(pVgroup, NULL); - } } - mgmtDecVgroupRef(pVgroup); + mnodeDecVgroupRef(pVgroup); } sdbFreeIter(pIter); mPrint("db:%s, all vgroups:%d is dropped from sdb", pDropDb->name, numOfVgroups); } + +void mnodeSendDropAllDbVgroupsMsg(SDbObj *pDropDb) { + void * pIter = NULL; + int32_t numOfVgroups = 0; + SVgObj *pVgroup = NULL; + + mPrint("db:%s, all vgroups will be dropped in dnode", pDropDb->name); + while (1) { + pIter = mnodeGetNextVgroup(pIter, &pVgroup); + if (pVgroup == NULL) break; + + if (pVgroup->pDb == pDropDb) { + mnodeSendDropVgroupMsg(pVgroup, NULL); + } + + mnodeDecVgroupRef(pVgroup); + } + + sdbFreeIter(pIter); + + mPrint("db:%s, all vgroups:%d drop msg is sent to dnode", pDropDb->name, numOfVgroups); +} \ No newline at end of file diff --git a/src/mnode/src/mnodeWrite.c b/src/mnode/src/mnodeWrite.c new file mode 100644 index 0000000000000000000000000000000000000000..8b3d82d32a57974a36206f58ddb7ef991832d498 --- /dev/null +++ b/src/mnode/src/mnodeWrite.c @@ -0,0 +1,82 @@ +/* + * Copyright (c) 2019 TAOS Data, Inc. + * + * This program is free software: you can use, redistribute, and/or modify + * it under the terms of the GNU Affero General Public License, version 3 + * or later ("AGPL"), as published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ + +#define _DEFAULT_SOURCE +#include "os.h" +#include "taosdef.h" +#include "tsched.h" +#include "tbalance.h" +#include "tgrant.h" +#include "tglobal.h" +#include "trpc.h" +#include "mnode.h" +#include "dnode.h" +#include "mnodeDef.h" +#include "mnodeInt.h" +#include "mnodeAcct.h" +#include "mnodeDnode.h" +#include "mnodeMnode.h" +#include "mnodeDb.h" +#include "mnodeSdb.h" +#include "mnodeVgroup.h" +#include "mnodeUser.h" +#include "mnodeTable.h" +#include "mnodeShow.h" + +static int32_t (*tsMnodeProcessWriteMsgFp[TSDB_MSG_TYPE_MAX])(SMnodeMsg *); + +void mnodeAddWriteMsgHandle(uint8_t msgType, int32_t (*fp)(SMnodeMsg *mnodeMsg)) { + tsMnodeProcessWriteMsgFp[msgType] = fp; +} + +int32_t mnodeProcessWrite(SMnodeMsg *pMsg) { + if (pMsg->rpcMsg.pCont == NULL) { + mError("%p, msg:%s in mwrite queue, content is null", pMsg->rpcMsg.ahandle, taosMsg[pMsg->rpcMsg.msgType]); + return TSDB_CODE_INVALID_MSG_LEN; + } + + if (!sdbIsMaster()) { + SMnodeRsp *rpcRsp = &pMsg->rpcRsp; + SRpcIpSet *ipSet = rpcMallocCont(sizeof(SRpcIpSet)); + mnodeGetMnodeIpSetForShell(ipSet); + rpcRsp->rsp = ipSet; + rpcRsp->len = sizeof(SRpcIpSet); + + mTrace("%p, msg:%s in mwrite queue, will be redireced inUse:%d", pMsg->rpcMsg.ahandle, taosMsg[pMsg->rpcMsg.msgType], ipSet->inUse); + for (int32_t i = 0; i < ipSet->numOfIps; ++i) { + mTrace("mnode index:%d ip:%s:%d", i, ipSet->fqdn[i], htons(ipSet->port[i])); + } + + return TSDB_CODE_REDIRECT; + } + + if (tsMnodeProcessWriteMsgFp[pMsg->rpcMsg.msgType] == NULL) { + mError("%p, msg:%s in mwrite queue, not processed", pMsg->rpcMsg.ahandle, taosMsg[pMsg->rpcMsg.msgType]); + return TSDB_CODE_MSG_NOT_PROCESSED; + } + + int32_t code = mnodeInitMsg(pMsg); + if (code != TSDB_CODE_SUCCESS) { + mError("%p, msg:%s in mwrite queue, not processed reason:%s", pMsg->rpcMsg.ahandle, taosMsg[pMsg->rpcMsg.msgType], tstrerror(code)); + return code; + } + + if (!pMsg->pUser->writeAuth) { + mError("%p, msg:%s in mwrite queue, not processed, no write auth", pMsg->rpcMsg.ahandle, taosMsg[pMsg->rpcMsg.msgType]); + return TSDB_CODE_NO_RIGHTS; + } + + return (*tsMnodeProcessWriteMsgFp[pMsg->rpcMsg.msgType])(pMsg); +} diff --git a/src/plugins/http/inc/httpLog.h b/src/plugins/http/inc/httpLog.h index 0e09f277f0de9a5d1860fc63e4765476af57df9b..a940dcf6ec0036ef7ebc82d1d3456f2ef70aaabe 100644 --- a/src/plugins/http/inc/httpLog.h +++ b/src/plugins/http/inc/httpLog.h @@ -21,7 +21,7 @@ extern int32_t httpDebugFlag; #define httpError(...) { if (httpDebugFlag & DEBUG_ERROR) { taosPrintLog("ERROR HTP ", 255, __VA_ARGS__); }} -#define httpWarn(...) { if (httpDebugFlag & DEBUG_WARN) { taosPrintLog("WARN HTP ", httpDebugFlag, __VA_ARGS__); }} +#define httpWarn(...) { if (httpDebugFlag & DEBUG_WARN) { taosPrintLog("WARN HTP ", httpDebugFlag, __VA_ARGS__); }} #define httpTrace(...) { if (httpDebugFlag & DEBUG_TRACE) { taosPrintLog("HTP ", httpDebugFlag, __VA_ARGS__); }} #define httpDump(...) { if (httpDebugFlag & DEBUG_TRACE) { taosPrintLongString("HTP ", httpDebugFlag, __VA_ARGS__); }} #define httpPrint(...) { taosPrintLog("HTP ", 255, __VA_ARGS__); } diff --git a/src/plugins/http/src/httpJson.c b/src/plugins/http/src/httpJson.c index cbca19b57f0b2875ec08688ba9bcfe09a36c9116..74e3c409a1cc6932c7f3929d73f7da6469176e03 100644 --- a/src/plugins/http/src/httpJson.c +++ b/src/plugins/http/src/httpJson.c @@ -445,10 +445,10 @@ void httpJsonPairStatus(JsonBuf* buf, int code) { httpJsonItemToken(buf); if (code == TSDB_CODE_DB_NOT_SELECTED) { httpJsonPair(buf, "desc", 4, "failed to create database", 23); - } else if (code == TSDB_CODE_INVALID_TABLE) { + } else if (code == TSDB_CODE_INVALID_TABLE_ID) { httpJsonPair(buf, "desc", 4, "failed to create table", 22); } else httpJsonPair(buf, "desc", 4, (char*)tstrerror(code), (int)strlen(tstrerror(code))); } } -} \ No newline at end of file +} diff --git a/src/plugins/http/src/tgHandle.c b/src/plugins/http/src/tgHandle.c index 1a55c325d5cfb1ccf6d6a6f83da7f2e2880cad83..61f9da63688f6d0f541a1fbae56b4d2037978269 100644 --- a/src/plugins/http/src/tgHandle.c +++ b/src/plugins/http/src/tgHandle.c @@ -281,7 +281,7 @@ int tgReadSchema(char *fileName) { } void tgInitHandle(HttpServer *pServer) { - char fileName[256] = {0}; + char fileName[TSDB_FILENAME_LEN*2] = {0}; sprintf(fileName, "%s/taos.telegraf.cfg", configDir); if (tgReadSchema(fileName) <= 0) { tgFreeSchemas(); diff --git a/src/plugins/http/src/tgJson.c b/src/plugins/http/src/tgJson.c index cb9c42e7920c57f276c52d013870e0623d77ee43..27059010b8713340de639feba6e0fb05c2554afb 100644 --- a/src/plugins/http/src/tgJson.c +++ b/src/plugins/http/src/tgJson.c @@ -111,7 +111,7 @@ bool tgCheckFinished(struct HttpContext *pContext, HttpSqlCmd *cmd, int code) { pContext->ipstr); return false; } - } else if (code == TSDB_CODE_INVALID_TABLE) { + } else if (code == TSDB_CODE_INVALID_TABLE_ID) { cmd->cmdState = HTTP_CMD_STATE_RUN_FINISHED; if (multiCmds->cmds[multiCmds->pos - 1].cmdState == HTTP_CMD_STATE_NOT_RUN_YET) { multiCmds->pos = (int16_t)(multiCmds->pos - 2); @@ -151,4 +151,4 @@ void tgSetNextCmd(struct HttpContext *pContext, HttpSqlCmd *cmd, int code) { } else { multiCmds->pos++; } -} \ No newline at end of file +} diff --git a/src/plugins/monitor/src/monitorMain.c b/src/plugins/monitor/src/monitorMain.c index fc958c1b1fb408afcd5c6a4d27a9eb4632731bcf..0c27233289ce054625e3512374748206f54176df 100644 --- a/src/plugins/monitor/src/monitorMain.c +++ b/src/plugins/monitor/src/monitorMain.c @@ -28,7 +28,7 @@ #include "monitor.h" #define monitorError(...) { if (monitorDebugFlag & DEBUG_ERROR) { taosPrintLog("ERROR MON ", 255, __VA_ARGS__); }} -#define monitorWarn(...) { if (monitorDebugFlag & DEBUG_WARN) { taosPrintLog("WARN MON ", monitorDebugFlag, __VA_ARGS__); }} +#define monitorWarn(...) { if (monitorDebugFlag & DEBUG_WARN) { taosPrintLog("WARN MON ", monitorDebugFlag, __VA_ARGS__); }} #define monitorTrace(...) { if (monitorDebugFlag & DEBUG_TRACE) { taosPrintLog("MON ", monitorDebugFlag, __VA_ARGS__); }} #define monitorPrint(...) { taosPrintLog("MON ", 255, __VA_ARGS__); } diff --git a/src/plugins/mqtt/inc/mqttLog.h b/src/plugins/mqtt/inc/mqttLog.h index 735678a326c0842c1c3084ad203255991a0e3615..a7ae2fc1ae70151a4d2133f3be53bbdafcdb50e2 100644 --- a/src/plugins/mqtt/inc/mqttLog.h +++ b/src/plugins/mqtt/inc/mqttLog.h @@ -26,7 +26,7 @@ extern int32_t mqttDebugFlag; } #define mqttWarn(...) \ if ( mqttDebugFlag & DEBUG_WARN) { \ - taosPrintLog("WARN MQT ", mqttDebugFlag, __VA_ARGS__); \ + taosPrintLog("WARN MQT ", mqttDebugFlag, __VA_ARGS__); \ } #define mqttTrace(...) \ if ( mqttDebugFlag & DEBUG_TRACE) { \ diff --git a/src/query/src/qExecutor.c b/src/query/src/qExecutor.c index 47ad633e34bd8b9d4051bc6888c27dba21924fcf..72d018d3158156eeb5d665b9f258266141f0b5f2 100644 --- a/src/query/src/qExecutor.c +++ b/src/query/src/qExecutor.c @@ -5298,7 +5298,7 @@ static int32_t convertQueryMsg(SQueryTableMsg *pQueryMsg, SArray **pTableIdList, if (pColFilter->filterstr) { pColFilter->len = htobe64(pFilterMsg->len); - pColFilter->pz = (int64_t) calloc(1, pColFilter->len); + pColFilter->pz = (int64_t) calloc(1, pColFilter->len + 1 * TSDB_NCHAR_SIZE); // note: null-terminator memcpy((void *)pColFilter->pz, pMsg, pColFilter->len); pMsg += (pColFilter->len + 1); } else { diff --git a/src/rpc/inc/rpcLog.h b/src/rpc/inc/rpcLog.h index 5094d817572a66da4f5cfcebbd17172fe0a58d89..10ef974298ad962a9d003841dabc7a1c7db12ab8 100644 --- a/src/rpc/inc/rpcLog.h +++ b/src/rpc/inc/rpcLog.h @@ -25,7 +25,7 @@ extern "C" { extern int32_t rpcDebugFlag; #define tError(...) { if (rpcDebugFlag & DEBUG_ERROR) { taosPrintLog("ERROR RPC ", rpcDebugFlag, __VA_ARGS__); }} -#define tWarn(...) { if (rpcDebugFlag & DEBUG_WARN) { taosPrintLog("WARN RPC ", rpcDebugFlag, __VA_ARGS__); }} +#define tWarn(...) { if (rpcDebugFlag & DEBUG_WARN) { taosPrintLog("WARN RPC ", rpcDebugFlag, __VA_ARGS__); }} #define tTrace(...) { if (rpcDebugFlag & DEBUG_TRACE) { taosPrintLog("RPC ", rpcDebugFlag, __VA_ARGS__); }} #define tDump(x, y) { if (rpcDebugFlag & DEBUG_DUMP) { taosDumpData((unsigned char *)x, y); }} #define tPrint(...) { taosPrintLog("RPC ", 255, __VA_ARGS__); } diff --git a/src/rpc/src/rpcMain.c b/src/rpc/src/rpcMain.c index a8bb2fd65b2091e33eb26d3f738cbfe4f736f69e..fa46c3a0f35aaa8076815bf6f7de6d8f802f590b 100644 --- a/src/rpc/src/rpcMain.c +++ b/src/rpc/src/rpcMain.c @@ -420,13 +420,13 @@ void rpcSendResponse(const SRpcMsg *pRsp) { pConn->rspMsgLen = msgLen; if (pMsg->code == TSDB_CODE_ACTION_IN_PROGRESS) pConn->inTranId--; - rpcUnlockConn(pConn); - taosTmrStopA(&pConn->pTimer); // taosTmrReset(rpcProcessIdleTimer, pRpc->idleTime, pConn, pRpc->tmrCtrl, &pConn->pIdleTimer); rpcSendMsgToPeer(pConn, msg, msgLen); pConn->secured = 1; // connection shall be secured + rpcUnlockConn(pConn); + return; } @@ -594,7 +594,10 @@ static SRpcConn *rpcAllocateServerConn(SRpcInfo *pRpc, SRecvInfo *pRecv) { // check if it is already allocated SRpcConn **ppConn = (SRpcConn **)(taosHashGet(pRpc->hash, hashstr, size)); if (ppConn) pConn = *ppConn; - if (pConn) return pConn; + if (pConn) { + pConn->secured = 0; + return pConn; + } int sid = taosAllocateId(pRpc->idPool); if (sid <= 0) { @@ -1092,10 +1095,10 @@ static void rpcSendReqToServer(SRpcInfo *pRpc, SRpcReqContext *pContext) { pConn->reqMsgLen = msgLen; pConn->pContext = pContext; - rpcUnlockConn(pConn); - taosTmrReset(rpcProcessRetryTimer, tsRpcTimer, pConn, pRpc->tmrCtrl, &pConn->pTimer); rpcSendMsgToPeer(pConn, msg, msgLen); + + rpcUnlockConn(pConn); } static void rpcSendMsgToPeer(SRpcConn *pConn, void *msg, int msgLen) { diff --git a/src/tsdb/inc/tsdbMain.h b/src/tsdb/inc/tsdbMain.h index 63d3eb349b75717895441c0e2661b86db3db2c67..9dd5136c959f30bc56804b87f9f63692d1cc0eca 100644 --- a/src/tsdb/inc/tsdbMain.h +++ b/src/tsdb/inc/tsdbMain.h @@ -85,12 +85,14 @@ typedef struct STable { TSKEY lastKey; // lastkey inserted in this table, initialized as 0, TODO: make a structure struct STable *next; // TODO: remove the next struct STable *prev; - tstr * name; // NOTE: there a flexible string here + tstr * name; // NOTE: there a flexible string here + char * sql; + void * cqhandle; } STable; #define TSDB_GET_TABLE_LAST_KEY(tb) ((tb)->lastKey) -void * tsdbEncodeTable(STable *pTable, int *contLen); +void tsdbEncodeTable(STable *pTable, char *buf, int *contLen); STable *tsdbDecodeTable(void *cont, int contLen); void tsdbFreeEncode(void *cont); @@ -109,6 +111,7 @@ typedef struct { SMetaFile *mfh; // meta file handle int maxRowBytes; int maxCols; + void * pRepo; } STsdbMeta; // element put in skiplist for each table @@ -117,7 +120,7 @@ typedef struct STableIndexElem { STable* pTable; } STableIndexElem; -STsdbMeta *tsdbInitMeta(char *rootDir, int32_t maxTables); +STsdbMeta *tsdbInitMeta(char *rootDir, int32_t maxTables, void *pRepo); int32_t tsdbFreeMeta(STsdbMeta *pMeta); STSchema * tsdbGetTableSchema(STsdbMeta *pMeta, STable *pTable); STSchema * tsdbGetTableTagSchema(STsdbMeta *pMeta, STable *pTable); diff --git a/src/tsdb/src/tsdbMain.c b/src/tsdb/src/tsdbMain.c index 55342ffd38add8f8975945addaedf466955af969..9c8e57d18aa6bcc98d9acb562bd83f070ff97281 100644 --- a/src/tsdb/src/tsdbMain.c +++ b/src/tsdb/src/tsdbMain.c @@ -189,9 +189,9 @@ _err: * * @return a TSDB repository handle on success, NULL for failure and the error number is set */ -TsdbRepoT *tsdbOpenRepo(char *tsdbDir, STsdbAppH *pAppH) { +TsdbRepoT *tsdbOpenRepo(char *rootDir, STsdbAppH *pAppH) { char dataDir[128] = "\0"; - if (access(tsdbDir, F_OK | W_OK | R_OK) < 0) { + if (access(rootDir, F_OK | W_OK | R_OK) < 0) { return NULL; } @@ -200,12 +200,12 @@ TsdbRepoT *tsdbOpenRepo(char *tsdbDir, STsdbAppH *pAppH) { return NULL; } - pRepo->rootDir = strdup(tsdbDir); + pRepo->rootDir = strdup(rootDir); tsdbRestoreCfg(pRepo, &(pRepo->config)); if (pAppH) pRepo->appH = *pAppH; - pRepo->tsdbMeta = tsdbInitMeta(tsdbDir, pRepo->config.maxTables); + pRepo->tsdbMeta = tsdbInitMeta(rootDir, pRepo->config.maxTables, pRepo); if (pRepo->tsdbMeta == NULL) { free(pRepo->rootDir); free(pRepo); @@ -446,7 +446,7 @@ int32_t tsdbInsertData(TsdbRepoT *repo, SSubmitMsg *pMsg, SShellSubmitRspMsg * p */ int tsdbInitTableCfg(STableCfg *config, ETableType type, uint64_t uid, int32_t tid) { if (config == NULL) return -1; - if (type != TSDB_NORMAL_TABLE && type != TSDB_CHILD_TABLE) return -1; + if (type != TSDB_CHILD_TABLE && type != TSDB_NORMAL_TABLE && type != TSDB_STREAM_TABLE) return -1; memset((void *)config, 0, sizeof(STableCfg)); @@ -455,6 +455,7 @@ int tsdbInitTableCfg(STableCfg *config, ETableType type, uint64_t uid, int32_t t config->tableId.uid = uid; config->tableId.tid = tid; config->name = NULL; + config->sql = NULL; return 0; } @@ -540,12 +541,26 @@ int tsdbTableSetSName(STableCfg *config, char *sname, bool dup) { return 0; } +int tsdbTableSetStreamSql(STableCfg *config, char *sql, bool dup) { + if (config->type != TSDB_STREAM_TABLE) return -1; + + if (dup) { + config->sql = strdup(sql); + if (config->sql == NULL) return -1; + } else { + config->sql = sql; + } + + return 0; +} + void tsdbClearTableCfg(STableCfg *config) { if (config->schema) tdFreeSchema(config->schema); if (config->tagSchema) tdFreeSchema(config->tagSchema); if (config->tagValues) tdFreeDataRow(config->tagValues); tfree(config->name); tfree(config->sname); + tfree(config->sql); } int tsdbInitSubmitBlkIter(SSubmitBlk *pBlock, SSubmitBlkIter *pIter) { @@ -936,7 +951,7 @@ static SSkipListIterator **tsdbCreateTableIters(STsdbMeta *pMeta, int maxTables) for (int tid = 1; tid < maxTables; tid++) { STable *pTable = pMeta->tables[tid]; - if (pTable == NULL || pTable->imem == NULL) continue; + if (pTable == NULL || pTable->imem == NULL || pTable->imem->numOfRows == 0) continue; iters[tid] = tSkipListCreateIter(pTable->imem->pData); if (iters[tid] == NULL) goto _err; @@ -968,12 +983,12 @@ static void *tsdbCommitData(void *arg) { SRWHelper whelper = {{0}}; if (pCache->imem == NULL) return NULL; - tsdbPrint("vgId: %d, starting to commit....", pRepo->config.tsdbId); + tsdbPrint("vgId:%d, starting to commit....", pRepo->config.tsdbId); // Create the iterator to read from cache SSkipListIterator **iters = tsdbCreateTableIters(pMeta, pCfg->maxTables); if (iters == NULL) { - // TODO: deal with the error + ASSERT(0); return NULL; } @@ -1015,6 +1030,7 @@ _exit: } } tsdbUnLockRepo(arg); + tsdbPrint("vgId:%d, commit over....", pRepo->config.tsdbId); return NULL; } diff --git a/src/tsdb/src/tsdbMeta.c b/src/tsdb/src/tsdbMeta.c index 95680f95c4565a500a00b14e8dfec9de91d902fe..0d9e6a9cf8c6c5438be023def71657eaaa2e9da6 100644 --- a/src/tsdb/src/tsdbMeta.c +++ b/src/tsdb/src/tsdbMeta.c @@ -15,7 +15,6 @@ static int32_t tsdbCheckTableCfg(STableCfg *pCfg); static int tsdbAddTableToMeta(STsdbMeta *pMeta, STable *pTable, bool addIdx); static int tsdbAddTableIntoIndex(STsdbMeta *pMeta, STable *pTable); static int tsdbRemoveTableFromIndex(STsdbMeta *pMeta, STable *pTable); -static int tsdbEstimateTableEncodeSize(STable *pTable); static int tsdbRemoveTableFromMeta(STsdbMeta *pMeta, STable *pTable, bool rmFromIdx); /** @@ -28,16 +27,10 @@ static int tsdbRemoveTableFromMeta(STsdbMeta *pMeta, STable *pTable, bool rm * @return binary content for success * NULL fro failure */ -void *tsdbEncodeTable(STable *pTable, int *contLen) { - if (pTable == NULL) return NULL; - - *contLen = tsdbEstimateTableEncodeSize(pTable); - if (*contLen < 0) return NULL; +void tsdbEncodeTable(STable *pTable, char *buf, int *contLen) { + if (pTable == NULL) return; - void *ret = calloc(1, *contLen); - if (ret == NULL) return NULL; - - void *ptr = ret; + void *ptr = buf; T_APPEND_MEMBER(ptr, pTable, STable, type); // Encode name, todo refactor *(int *)ptr = varDataLen(pTable->name); @@ -54,12 +47,17 @@ void *tsdbEncodeTable(STable *pTable, int *contLen) { ptr = tdEncodeSchema(ptr, pTable->schema); ptr = tdEncodeSchema(ptr, pTable->tagSchema); } else if (pTable->type == TSDB_CHILD_TABLE) { - dataRowCpy(ptr, pTable->tagVal); + tdTagRowCpy(ptr, pTable->tagVal); + ptr = POINTER_SHIFT(ptr, dataRowLen(pTable->tagVal) + ((STagRow *)pTable->tagVal)->dataLen); } else { ptr = tdEncodeSchema(ptr, pTable->schema); } - return ret; + if (pTable->type == TSDB_STREAM_TABLE) { + ptr = taosEncodeString(ptr, pTable->sql); + } + + *contLen = (char *)ptr - buf; } /** @@ -96,11 +94,16 @@ STable *tsdbDecodeTable(void *cont, int contLen) { pTable->schema = tdDecodeSchema(&ptr); pTable->tagSchema = tdDecodeSchema(&ptr); } else if (pTable->type == TSDB_CHILD_TABLE) { - pTable->tagVal = tdDataRowDup(ptr); + pTable->tagVal = tdTagRowDecode(ptr); + ptr = POINTER_SHIFT(ptr, dataRowLen(pTable->tagVal) + ((STagRow *)pTable->tagVal)->dataLen); } else { pTable->schema = tdDecodeSchema(&ptr); } + if (pTable->type == TSDB_STREAM_TABLE) { + ptr = taosDecodeString(ptr, &(pTable->sql)); + } + return pTable; } @@ -114,8 +117,10 @@ static char* getTagIndexKey(const void* pData) { SDataRow row = elem->pTable->tagVal; STSchema* pSchema = tsdbGetTableTagSchema(elem->pMeta, elem->pTable); STColumn* pCol = &pSchema->columns[DEFAULT_TAG_INDEX_COLUMN]; - - return tdGetRowDataOfCol(row, pCol->type, TD_DATA_ROW_HEAD_SIZE + pCol->offset); + int16_t type = 0; + void * res = tdQueryTagByID(row, pCol->colId,&type); + ASSERT(type == pCol->type); + return res; } int tsdbRestoreTable(void *pHandle, void *cont, int contLen) { @@ -137,6 +142,7 @@ int tsdbRestoreTable(void *pHandle, void *cont, int contLen) { void tsdbOrgMeta(void *pHandle) { STsdbMeta *pMeta = (STsdbMeta *)pHandle; + STsdbRepo *pRepo = (STsdbRepo *)pMeta->pRepo; for (int i = 1; i < pMeta->maxTables; i++) { STable *pTable = pMeta->tables[i]; @@ -144,13 +150,20 @@ void tsdbOrgMeta(void *pHandle) { tsdbAddTableIntoIndex(pMeta, pTable); } } + + for (int i = 0; i < pMeta->maxTables; i++) { + STable *pTable = pMeta->tables[i]; + if (pTable && pTable->type == TSDB_STREAM_TABLE) { + pTable->cqhandle = (*pRepo->appH.cqCreateFunc)(pRepo->appH.cqH, i, pTable->sql, tsdbGetTableSchema(pMeta, pTable)); + } + } } /** * Initialize the meta handle * ASSUMPTIONS: VALID PARAMETER */ -STsdbMeta *tsdbInitMeta(char *rootDir, int32_t maxTables) { +STsdbMeta *tsdbInitMeta(char *rootDir, int32_t maxTables, void *pRepo) { STsdbMeta *pMeta = (STsdbMeta *)malloc(sizeof(STsdbMeta)); if (pMeta == NULL) return NULL; @@ -160,6 +173,7 @@ STsdbMeta *tsdbInitMeta(char *rootDir, int32_t maxTables) { pMeta->tables = (STable **)calloc(maxTables, sizeof(STable *)); pMeta->maxRowBytes = 0; pMeta->maxCols = 0; + pMeta->pRepo = pRepo; if (pMeta->tables == NULL) { free(pMeta); return NULL; @@ -184,13 +198,16 @@ STsdbMeta *tsdbInitMeta(char *rootDir, int32_t maxTables) { } int32_t tsdbFreeMeta(STsdbMeta *pMeta) { + STsdbRepo *pRepo = (STsdbRepo *)pMeta->pRepo; if (pMeta == NULL) return 0; tsdbCloseMetaFile(pMeta->mfh); for (int i = 1; i < pMeta->maxTables; i++) { if (pMeta->tables[i] != NULL) { - tsdbFreeTable(pMeta->tables[i]); + STable *pTable = pMeta->tables[i]; + if (pTable->type == TSDB_STREAM_TABLE) (*pRepo->appH.cqDropFunc)(pTable->cqhandle); + tsdbFreeTable(pTable); } } @@ -211,7 +228,7 @@ int32_t tsdbFreeMeta(STsdbMeta *pMeta) { } STSchema *tsdbGetTableSchema(STsdbMeta *pMeta, STable *pTable) { - if (pTable->type == TSDB_NORMAL_TABLE || pTable->type == TSDB_SUPER_TABLE) { + if (pTable->type == TSDB_NORMAL_TABLE || pTable->type == TSDB_SUPER_TABLE || pTable->type == TSDB_STREAM_TABLE) { return pTable->schema; } else if (pTable->type == TSDB_CHILD_TABLE) { STable *pSuper = tsdbGetTableByUid(pMeta, pTable->superUid); @@ -255,8 +272,9 @@ int32_t tsdbGetTableTagVal(TsdbRepoT* repo, STableId* id, int32_t colId, int16_t } SDataRow row = (SDataRow)pTable->tagVal; - char* d = tdGetRowDataOfCol(row, pCol->type, TD_DATA_ROW_HEAD_SIZE + pCol->offset); - + int16_t tagtype = 0; + char* d = tdQueryTagByID(row, pCol->colId, &tagtype); + //ASSERT((int8_t)tagtype == pCol->type) *val = d; *type = pCol->type; *bytes = pCol->bytes; @@ -283,6 +301,76 @@ char* tsdbGetTableName(TsdbRepoT *repo, const STableId* id, int16_t* bytes) { } } +static STable *tsdbNewTable(STableCfg *pCfg, bool isSuper) { + STable *pTable = NULL; + size_t tsize = 0; + + pTable = (STable *)calloc(1, sizeof(STable)); + if (pTable == NULL) { + terrno = TSDB_CODE_SERV_OUT_OF_MEMORY; + goto _err; + } + + pTable->type = pCfg->type; + + if (isSuper) { + pTable->type = TSDB_SUPER_TABLE; + pTable->tableId.uid = pCfg->superUid; + pTable->tableId.tid = -1; + pTable->superUid = TSDB_INVALID_SUPER_TABLE_ID; + pTable->schema = tdDupSchema(pCfg->schema); + pTable->tagSchema = tdDupSchema(pCfg->tagSchema); + + tsize = strnlen(pCfg->sname, TSDB_TABLE_NAME_LEN); + pTable->name = calloc(1, tsize + VARSTR_HEADER_SIZE + 1); + if (pTable->name == NULL) { + terrno = TSDB_CODE_SERV_OUT_OF_MEMORY; + goto _err; + } + STR_WITH_SIZE_TO_VARSTR(pTable->name, pCfg->sname, tsize); + + STColumn *pColSchema = schemaColAt(pTable->tagSchema, 0); + pTable->pIndex = tSkipListCreate(TSDB_SUPER_TABLE_SL_LEVEL, pColSchema->type, pColSchema->bytes, 1, 0, 0, + getTagIndexKey); // Allow duplicate key, no lock + if (pTable->pIndex == NULL) { + terrno = TSDB_CODE_SERV_OUT_OF_MEMORY; + goto _err; + } + } else { + pTable->type = pCfg->type; + pTable->tableId.uid = pCfg->tableId.uid; + pTable->tableId.tid = pCfg->tableId.tid; + pTable->lastKey = TSKEY_INITIAL_VAL; + + tsize = strnlen(pCfg->name, TSDB_TABLE_NAME_LEN); + pTable->name = calloc(1, tsize + VARSTR_HEADER_SIZE + 1); + if (pTable->name == NULL) { + terrno = TSDB_CODE_SERV_OUT_OF_MEMORY; + goto _err; + } + STR_WITH_SIZE_TO_VARSTR(pTable->name, pCfg->name, tsize); + + if (pCfg->type == TSDB_CHILD_TABLE) { + pTable->superUid = pCfg->superUid; + pTable->tagVal = tdDataRowDup(pCfg->tagValues); + } else if (pCfg->type == TSDB_NORMAL_TABLE) { + pTable->superUid = -1; + pTable->schema = tdDupSchema(pCfg->schema); + } else { + ASSERT(pCfg->type == TSDB_STREAM_TABLE); + pTable->superUid = -1; + pTable->schema = tdDupSchema(pCfg->schema); + pTable->sql = strdup(pCfg->sql); + } + } + + return pTable; + +_err: + tsdbFreeTable(pTable); + return NULL; +} + int tsdbCreateTable(TsdbRepoT *repo, STableCfg *pCfg) { STsdbRepo *pRepo = (STsdbRepo *)repo; STsdbMeta *pMeta = pRepo->tsdbMeta; @@ -303,61 +391,19 @@ int tsdbCreateTable(TsdbRepoT *repo, STableCfg *pCfg) { super = tsdbGetTableByUid(pMeta, pCfg->superUid); if (super == NULL) { // super table not exists, try to create it newSuper = 1; - // TODO: use function to implement create table object - super = (STable *)calloc(1, sizeof(STable)); + super = tsdbNewTable(pCfg, true); if (super == NULL) return -1; - - super->type = TSDB_SUPER_TABLE; - super->tableId.uid = pCfg->superUid; - super->tableId.tid = -1; - super->superUid = TSDB_INVALID_SUPER_TABLE_ID; - super->schema = tdDupSchema(pCfg->schema); - super->tagSchema = tdDupSchema(pCfg->tagSchema); - super->tagVal = NULL; - - // todo refactor extract method - size_t size = strnlen(pCfg->sname, TSDB_TABLE_NAME_LEN); - super->name = calloc(1, size + VARSTR_HEADER_SIZE + 1); - STR_WITH_SIZE_TO_VARSTR(super->name, pCfg->sname, size); - - // index the first tag column - STColumn* pColSchema = schemaColAt(super->tagSchema, 0); - super->pIndex = tSkipListCreate(TSDB_SUPER_TABLE_SL_LEVEL, pColSchema->type, pColSchema->bytes, - 1, 0, 1, getTagIndexKey); // Allow duplicate key, no lock - - if (super->pIndex == NULL) { - tdFreeSchema(super->schema); - tdFreeSchema(super->tagSchema); - tdFreeDataRow(super->tagVal); - free(super); - return -1; - } } else { if (super->type != TSDB_SUPER_TABLE) return -1; } } - STable *table = (STable *)calloc(1, sizeof(STable)); + STable *table = tsdbNewTable(pCfg, false); if (table == NULL) { - if (newSuper) tsdbFreeTable(super); - return -1; - } - - table->tableId = pCfg->tableId; - - size_t size = strnlen(pCfg->name, TSDB_TABLE_NAME_LEN); - table->name = calloc(1, size + VARSTR_HEADER_SIZE + 1); - STR_WITH_SIZE_TO_VARSTR(table->name, pCfg->name, size); - - table->lastKey = 0; - if (IS_CREATE_STABLE(pCfg)) { // TSDB_CHILD_TABLE - table->type = TSDB_CHILD_TABLE; - table->superUid = pCfg->superUid; - table->tagVal = tdDataRowDup(pCfg->tagValues); - } else { // TSDB_NORMAL_TABLE - table->type = TSDB_NORMAL_TABLE; - table->superUid = -1; - table->schema = tdDupSchema(pCfg->schema); + if (newSuper) { + tsdbFreeTable(super); + return -1; + } } // Register to meta @@ -372,15 +418,15 @@ int tsdbCreateTable(TsdbRepoT *repo, STableCfg *pCfg) { // Write to meta file int bufLen = 0; + char *buf = malloc(1024*1024); if (newSuper) { - void *buf = tsdbEncodeTable(super, &bufLen); + tsdbEncodeTable(super, buf, &bufLen); tsdbInsertMetaRecord(pMeta->mfh, super->tableId.uid, buf, bufLen); - tsdbFreeEncode(buf); } - void *buf = tsdbEncodeTable(table, &bufLen); + tsdbEncodeTable(table, buf, &bufLen); tsdbInsertMetaRecord(pMeta->mfh, table->tableId.uid, buf, bufLen); - tsdbFreeEncode(buf); + tfree(buf); return 0; } @@ -438,13 +484,18 @@ static void tsdbFreeMemTable(SMemTable *pMemTable) { } static int tsdbFreeTable(STable *pTable) { - // TODO: finish this function + if (pTable == NULL) return 0; + if (pTable->type == TSDB_CHILD_TABLE) { - tdFreeDataRow(pTable->tagVal); + tdFreeTagRow(pTable->tagVal); } else { tdFreeSchema(pTable->schema); } + if (pTable->type == TSDB_STREAM_TABLE) { + tfree(pTable->sql); + } + // Free content if (TSDB_TABLE_IS_SUPER_TABLE(pTable)) { tdFreeSchema(pTable->tagSchema); @@ -473,6 +524,7 @@ STable *tsdbGetTableByUid(STsdbMeta *pMeta, uint64_t uid) { } static int tsdbAddTableToMeta(STsdbMeta *pMeta, STable *pTable, bool addIdx) { + STsdbRepo *pRepo = (STsdbRepo *)pMeta->pRepo; if (pTable->type == TSDB_SUPER_TABLE) { // add super table to the linked list if (pMeta->superList == NULL) { @@ -491,6 +543,9 @@ static int tsdbAddTableToMeta(STsdbMeta *pMeta, STable *pTable, bool addIdx) { if (pTable->type == TSDB_CHILD_TABLE && addIdx) { // add STABLE to the index tsdbAddTableIntoIndex(pMeta, pTable); } + if (pTable->type == TSDB_STREAM_TABLE && addIdx) { + pTable->cqhandle = (*pRepo->appH.cqCreateFunc)(pRepo->appH.cqH, pTable->tableId.tid, pTable->sql, tsdbGetTableSchema(pMeta, pTable)); + } pMeta->nTables++; } @@ -522,7 +577,6 @@ static int tsdbRemoveTableFromMeta(STsdbMeta *pMeta, STable *pTable, bool rmFrom tSkipListDestroyIter(pIter); - // TODO: Remove the table from the list if (pTable->prev != NULL) { pTable->prev->next = pTable->next; if (pTable->next != NULL) { @@ -536,6 +590,9 @@ static int tsdbRemoveTableFromMeta(STsdbMeta *pMeta, STable *pTable, bool rmFrom if (pTable->type == TSDB_CHILD_TABLE && rmFromIdx) { tsdbRemoveTableFromIndex(pMeta, pTable); } + if (pTable->type == TSDB_STREAM_TABLE && rmFromIdx) { + // TODO + } pMeta->nTables--; } @@ -579,7 +636,9 @@ static int tsdbRemoveTableFromIndex(STsdbMeta *pMeta, STable *pTable) { STSchema* pSchema = tsdbGetTableTagSchema(pMeta, pTable); STColumn* pCol = &pSchema->columns[DEFAULT_TAG_INDEX_COLUMN]; - char* key = tdGetRowDataOfCol(pTable->tagVal, pCol->type, TD_DATA_ROW_HEAD_SIZE + pCol->offset); + int16_t tagtype = 0; + char* key = tdQueryTagByID(pTable->tagVal, pCol->colId, &tagtype); + ASSERT(pCol->type == tagtype); SArray* res = tSkipListGet(pSTable->pIndex, key); size_t size = taosArrayGetSize(res); @@ -598,25 +657,6 @@ static int tsdbRemoveTableFromIndex(STsdbMeta *pMeta, STable *pTable) { return 0; } -static int tsdbEstimateTableEncodeSize(STable *pTable) { - int size = 0; - size += T_MEMBER_SIZE(STable, type); - size += sizeof(int) + varDataLen(pTable->name); - size += T_MEMBER_SIZE(STable, tableId); - size += T_MEMBER_SIZE(STable, superUid); - size += T_MEMBER_SIZE(STable, sversion); - - if (pTable->type == TSDB_SUPER_TABLE) { - size += tdGetSchemaEncodeSize(pTable->schema); - size += tdGetSchemaEncodeSize(pTable->tagSchema); - } else if (pTable->type == TSDB_CHILD_TABLE) { - size += dataRowLen(pTable->tagVal); - } else { - size += tdGetSchemaEncodeSize(pTable->schema); - } - - return size; -} char *getTSTupleKey(const void * data) { SDataRow row = (SDataRow)data; diff --git a/src/tsdb/src/tsdbRead.c b/src/tsdb/src/tsdbRead.c index 595217debb75b32ff150bec6a4231da3a0f0f683..2220ebfd88abf6933ae4c88003b26b1c9be9b1b4 100644 --- a/src/tsdb/src/tsdbRead.c +++ b/src/tsdb/src/tsdbRead.c @@ -1753,9 +1753,9 @@ int32_t tableGroupComparFn(const void *p1, const void *p2, const void *param) { STColumn* pCol = schemaColAt(pTableGroupSupp->pTagSchema, colIndex); bytes = pCol->bytes; type = pCol->type; - - f1 = tdGetRowDataOfCol(pTable1->tagVal, pCol->type, TD_DATA_ROW_HEAD_SIZE + pCol->offset); - f2 = tdGetRowDataOfCol(pTable2->tagVal, pCol->type, TD_DATA_ROW_HEAD_SIZE + pCol->offset); + int16_t tgtype1, tgtype2 = 0; + f1 = tdQueryTagByID(pTable1->tagVal, pCol->colId, &tgtype1); + f2 = tdQueryTagByID(pTable2->tagVal, pCol->colId, &tgtype2); } int32_t ret = doCompare(f1, f2, type, bytes); @@ -1843,12 +1843,14 @@ bool indexedNodeFilterFp(const void* pNode, void* param) { val = (char*) elem->pTable->name; type = TSDB_DATA_TYPE_BINARY; } else { - STSchema* pTSchema = (STSchema*) pInfo->param; // todo table schema is identical to stable schema?? - - int32_t offset = pTSchema->columns[pInfo->colIndex].offset; - val = tdGetRowDataOfCol(elem->pTable->tagVal, pInfo->sch.type, TD_DATA_ROW_HEAD_SIZE + offset); - } - +// STSchema* pTSchema = (STSchema*) pInfo->param; // todo table schema is identical to stable schema?? + int16_t type; + // int32_t offset = pTSchema->columns[pInfo->colIndex].offset; + // val = tdGetRowDataOfCol(elem->pTable->tagVal, pInfo->sch.type, TD_DATA_ROW_HEAD_SIZE + offset); + val = tdQueryTagByID(elem->pTable->tagVal, pInfo->sch.colId, &type); + // ASSERT(pInfo->sch.type == type); + } + //todo :the val is possible to be null, so check it out carefully int32_t ret = 0; if (type == TSDB_DATA_TYPE_BINARY || type == TSDB_DATA_TYPE_NCHAR) { if (pInfo->optr == TSDB_RELATION_IN) { diff --git a/src/util/inc/tcoding.h b/src/util/inc/tcoding.h index b4f7f596c56ff6bde8242881d1f8e06a7264986a..cc9caf71d04bdceba3dab1916470d8803863992f 100644 --- a/src/util/inc/tcoding.h +++ b/src/util/inc/tcoding.h @@ -217,6 +217,28 @@ static FORCE_INLINE void *taosDecodeVariant64(void *buf, uint64_t *value) { return NULL; // error happened } +static FORCE_INLINE void *taosEncodeString(void *buf, char *value) { + size_t size = strlen(value); + + buf = taosEncodeVariant64(buf, size); + memcpy(buf, value, size); + + return POINTER_SHIFT(buf, size); +} + +static FORCE_INLINE void *taosDecodeString(void *buf, char **value) { + uint64_t size = 0; + + buf = taosDecodeVariant64(buf, &size); + *value = (char *)malloc(size + 1); + if (*value == NULL) return NULL; + memcpy(*value, buf, size); + + (*value)[size] = '\0'; + + return POINTER_SHIFT(buf, size); +} + #ifdef __cplusplus } #endif diff --git a/src/util/inc/tutil.h b/src/util/inc/tutil.h index 3a0a1920af4cb2dd841bbeadd973eea85830019e..5dcb6e406f5c8dccd989ec0e2e8e1d2542177020 100644 --- a/src/util/inc/tutil.h +++ b/src/util/inc/tutil.h @@ -145,7 +145,7 @@ bool taosMbsToUcs4(char *mbs, size_t mbs_len, char *ucs4, int32_t ucs4_max_len, int tasoUcs4Compare(void* f1_ucs4, void *f2_ucs4, int bytes); -bool taosUcs4ToMbs(void *ucs4, int32_t ucs4_max_len, char *mbs); +int32_t taosUcs4ToMbs(void *ucs4, int32_t ucs4_max_len, char *mbs); bool taosValidateEncodec(const char *encodec); diff --git a/src/util/src/tconfig.c b/src/util/src/tconfig.c index 32a3df844c2be3bbbf634af796e56fb2c47a05ed..203a34fd15341acf263e74166717a14d4787a23d 100644 --- a/src/util/src/tconfig.c +++ b/src/util/src/tconfig.c @@ -241,7 +241,12 @@ void taosReadGlobalLogCfg() { wordexp_t full_path; wordexp(configDir, &full_path, 0); - if (full_path.we_wordv != NULL && full_path.we_wordv[0] != NULL) { + if (full_path.we_wordv != NULL && full_path.we_wordv[0] != NULL) { + if (strlen(full_path.we_wordv[0]) > TSDB_FILENAME_LEN - 1) { + printf("\nconfig file: %s path overflow max len %d, all variables are set to default\n", full_path.we_wordv[0], TSDB_FILENAME_LEN - 1); + wordfree(&full_path); + return; + } strcpy(configDir, full_path.we_wordv[0]); } else { printf("configDir:%s not there, use default value: /etc/taos", configDir); diff --git a/src/util/src/tqueue.c b/src/util/src/tqueue.c index 475941dbdb9fb9dee3d656b841fc286cfd9043ef..428c40ef7afe254a032856833c6d1f4b7cd3f515 100644 --- a/src/util/src/tqueue.c +++ b/src/util/src/tqueue.c @@ -18,24 +18,24 @@ #include "taoserror.h" #include "tqueue.h" -typedef struct _taos_qnode { +typedef struct STaosQnode { int type; - struct _taos_qnode *next; + struct STaosQnode *next; char item[]; } STaosQnode; -typedef struct _taos_q { +typedef struct STaosQueue { int32_t itemSize; int32_t numOfItems; - struct _taos_qnode *head; - struct _taos_qnode *tail; - struct _taos_q *next; // for queue set - struct _taos_qset *qset; // for queue set + struct STaosQnode *head; + struct STaosQnode *tail; + struct STaosQueue *next; // for queue set + struct STaosQset *qset; // for queue set void *ahandle; // for queue set pthread_mutex_t mutex; } STaosQueue; -typedef struct _taos_qset { +typedef struct STaosQset { STaosQueue *head; STaosQueue *current; pthread_mutex_t mutex; @@ -44,7 +44,7 @@ typedef struct _taos_qset { tsem_t sem; } STaosQset; -typedef struct _taos_qall { +typedef struct STaosQall { STaosQnode *current; STaosQnode *start; int32_t itemSize; @@ -95,6 +95,7 @@ void *taosAllocateQitem(int size) { void taosFreeQitem(void *param) { if (param == NULL) return; + uTrace("item:%p is freed", param); char *temp = (char *)param; temp -= sizeof(STaosQnode); free(temp); @@ -104,6 +105,7 @@ int taosWriteQitem(taos_queue param, int type, void *item) { STaosQueue *queue = (STaosQueue *)param; STaosQnode *pNode = (STaosQnode *)(((char *)item) - sizeof(STaosQnode)); pNode->type = type; + pNode->next = NULL; pthread_mutex_lock(&queue->mutex); @@ -143,7 +145,7 @@ int taosReadQitem(taos_queue param, int *type, void **pitem) { queue->numOfItems--; if (queue->qset) atomic_sub_fetch_32(&queue->qset->numOfItems, 1); code = 1; - uTrace("item:%p is read out from queue, items:%d", *pitem, queue->numOfItems); + uTrace("item:%p is read out from queue:%p, type:%d items:%d", *pitem, *type, queue->numOfItems); } pthread_mutex_unlock(&queue->mutex); @@ -337,6 +339,7 @@ int taosReadQitemFromQset(taos_qset param, int *type, void **pitem, void **phand queue->numOfItems--; atomic_sub_fetch_32(&qset->numOfItems, 1); code = 1; + uTrace("item:%p is read out from queue:%p, type:%d items:%d", *pitem, queue, *type, queue->numOfItems); } pthread_mutex_unlock(&queue->mutex); diff --git a/src/util/src/ttimer.c b/src/util/src/ttimer.c index e6ef73ef57d3da8a8961c9673096e7f6e6a33e1a..68db574d82f3e668ced4b8895c3427640641079e 100644 --- a/src/util/src/ttimer.c +++ b/src/util/src/ttimer.c @@ -27,7 +27,7 @@ #define tmrWarn(...) \ do { if (tmrDebugFlag & DEBUG_WARN) { \ - taosPrintLog("WARN TMR ", tmrDebugFlag, __VA_ARGS__); \ + taosPrintLog("WARN TMR ", tmrDebugFlag, __VA_ARGS__); \ } } while(0) #define tmrTrace(...) \ diff --git a/src/util/src/tutil.c b/src/util/src/tutil.c index 989273e0515e7c1affdd3ff9afe747688923270f..2392560370e97a21a191f1c1a3fb4bab986a65e9 100644 --- a/src/util/src/tutil.c +++ b/src/util/src/tutil.c @@ -447,10 +447,10 @@ int tasoUcs4Compare(void* f1_ucs4, void *f2_ucs4, int bytes) { int32_t ucs4_max_len = bytes + 4; char *f1_mbs = calloc(bytes, 1); char *f2_mbs = calloc(bytes, 1); - if (!taosUcs4ToMbs(f1_ucs4, ucs4_max_len, f1_mbs)) { + if (taosUcs4ToMbs(f1_ucs4, ucs4_max_len, f1_mbs) < 0) { return -1; } - if (!taosUcs4ToMbs(f2_ucs4, ucs4_max_len, f2_mbs)) { + if (taosUcs4ToMbs(f2_ucs4, ucs4_max_len, f2_mbs) < 0) { return -1; } int32_t ret = strcmp(f1_mbs, f2_mbs); @@ -464,29 +464,29 @@ int tasoUcs4Compare(void* f1_ucs4, void *f2_ucs4, int bytes) { #endif } -bool taosUcs4ToMbs(void *ucs4, int32_t ucs4_max_len, char *mbs) { +int32_t taosUcs4ToMbs(void *ucs4, int32_t ucs4_max_len, char *mbs) { #ifdef USE_LIBICONV iconv_t cd = iconv_open(tsCharset, DEFAULT_UNICODE_ENCODEC); size_t ucs4_input_len = ucs4_max_len; size_t outLen = ucs4_max_len; if (iconv(cd, (char **)&ucs4, &ucs4_input_len, &mbs, &outLen) == -1) { iconv_close(cd); - return false; + return -1; } iconv_close(cd); - return true; + return (int32_t)(ucs4_max_len - outLen); #else mbstate_t state = {0}; int32_t len = (int32_t) wcsnrtombs(NULL, (const wchar_t **) &ucs4, ucs4_max_len / 4, 0, &state); if (len < 0) { - return false; + return -1; } memset(&state, 0, sizeof(state)); len = wcsnrtombs(mbs, (const wchar_t **) &ucs4, ucs4_max_len / 4, (size_t) len, &state); if (len < 0) { - return false; + return -1; } - return true; + return len; #endif } diff --git a/src/vnode/inc/vnodeInt.h b/src/vnode/inc/vnodeInt.h index dea9369dd8d08c6cc26dc6049b65fa6ec11e7682..0168304b5158a1a76bae06cce1b9b3dc0132360b 100644 --- a/src/vnode/inc/vnodeInt.h +++ b/src/vnode/inc/vnodeInt.h @@ -27,10 +27,10 @@ extern "C" { extern int32_t vDebugFlag; -#define vError(...) if (vDebugFlag & DEBUG_ERROR) {taosPrintLog("ERROR VND ", 255, __VA_ARGS__); } -#define vWarn(...) if (vDebugFlag & DEBUG_WARN) {taosPrintLog("WARN VND ", vDebugFlag, __VA_ARGS__); } -#define vTrace(...) if (vDebugFlag & DEBUG_TRACE) {taosPrintLog("VND ", vDebugFlag, __VA_ARGS__); } -#define vPrint(...) {taosPrintLog("VND ", 255, __VA_ARGS__); } +#define vError(...) { if (vDebugFlag & DEBUG_ERROR) { taosPrintLog("ERROR VND ", 255, __VA_ARGS__); }} +#define vWarn(...) { if (vDebugFlag & DEBUG_WARN) { taosPrintLog("WARN VND ", vDebugFlag, __VA_ARGS__); }} +#define vTrace(...) { if (vDebugFlag & DEBUG_TRACE) { taosPrintLog("VND ", vDebugFlag, __VA_ARGS__); }} +#define vPrint(...) { taosPrintLog("VND ", 255, __VA_ARGS__); } typedef struct { int32_t vgId; // global vnode group ID diff --git a/src/vnode/inc/vnodeLog.h b/src/vnode/inc/vnodeLog.h index bd8daae0b637c75188539e4cd7f9fb9579ede3c6..d6794202da416a72e8be6fd82ddceaa295076f2a 100644 --- a/src/vnode/inc/vnodeLog.h +++ b/src/vnode/inc/vnodeLog.h @@ -24,20 +24,10 @@ extern "C" { extern int32_t dDebugFlag; -#define dError(...) \ - if (dDebugFlag & DEBUG_ERROR) { \ - taosPrintLog("ERROR DND ", 255, __VA_ARGS__); \ - } -#define dWarn(...) \ - if (dDebugFlag & DEBUG_WARN) { \ - taosPrintLog("WARN DND ", dDebugFlag, __VA_ARGS__); \ - } -#define dTrace(...) \ - if (dDebugFlag & DEBUG_TRACE) { \ - taosPrintLog("DND ", dDebugFlag, __VA_ARGS__); \ - } -#define dPrint(...) \ - { taosPrintLog("DND ", 255, __VA_ARGS__); } +#define dError(...) { if (dDebugFlag & DEBUG_ERROR) { taosPrintLog("ERROR DND ", 255, __VA_ARGS__); }} +#define dWarn(...) { if (dDebugFlag & DEBUG_WARN) { taosPrintLog("WARN DND ", dDebugFlag, __VA_ARGS__); }} +#define dTrace(...) { if (dDebugFlag & DEBUG_TRACE) { taosPrintLog("DND ", dDebugFlag, __VA_ARGS__); }} +#define dPrint(...) { taosPrintLog("DND ", 255, __VA_ARGS__); } #ifdef __cplusplus } diff --git a/src/vnode/src/vnodeMain.c b/src/vnode/src/vnodeMain.c index b8bc29550e3ee810b850b12922debbe06dfbe104..cc92c0338989718f9ac134eda80b5c96c8255ba1 100644 --- a/src/vnode/src/vnodeMain.c +++ b/src/vnode/src/vnodeMain.c @@ -198,8 +198,8 @@ int32_t vnodeOpen(int32_t vnode, char *rootDir) { pVnode->fversion = pVnode->version; - pVnode->wqueue = dnodeAllocateWqueue(pVnode); - pVnode->rqueue = dnodeAllocateRqueue(pVnode); + pVnode->wqueue = dnodeAllocateVnodeWqueue(pVnode); + pVnode->rqueue = dnodeAllocateVnodeRqueue(pVnode); if (pVnode->wqueue == NULL || pVnode->rqueue == NULL) { vnodeCleanUp(pVnode); return terrno; @@ -220,6 +220,8 @@ int32_t vnodeOpen(int32_t vnode, char *rootDir) { appH.appH = (void *)pVnode; appH.notifyStatus = vnodeProcessTsdbStatus; appH.cqH = pVnode->cq; + appH.cqCreateFunc = cqCreate; + appH.cqDropFunc = cqDrop; sprintf(temp, "%s/tsdb", rootDir); pVnode->tsdb = tsdbOpenRepo(temp, &appH); if (pVnode->tsdb == NULL) { @@ -245,7 +247,7 @@ int32_t vnodeOpen(int32_t vnode, char *rootDir) { syncInfo.getWalInfo = vnodeGetWalInfo; syncInfo.getFileInfo = vnodeGetFileInfo; syncInfo.writeToCache = vnodeWriteToQueue; - syncInfo.confirmForward = dnodeSendRpcWriteRsp; + syncInfo.confirmForward = dnodeSendRpcVnodeWriteRsp; syncInfo.notifyRole = vnodeNotifyRole; syncInfo.notifyFileSynced = vnodeNotifyFileSynced; pVnode->sync = syncStart(&syncInfo); @@ -391,24 +393,24 @@ static void vnodeCleanUp(SVnodeObj *pVnode) { pVnode->sync = NULL; } - if (pVnode->wal) - walClose(pVnode->wal); - pVnode->wal = NULL; - if (pVnode->tsdb) tsdbCloseRepo(pVnode->tsdb, 1); pVnode->tsdb = NULL; + if (pVnode->wal) + walClose(pVnode->wal); + pVnode->wal = NULL; + if (pVnode->cq) cqClose(pVnode->cq); pVnode->cq = NULL; if (pVnode->wqueue) - dnodeFreeWqueue(pVnode->wqueue); + dnodeFreeVnodeWqueue(pVnode->wqueue); pVnode->wqueue = NULL; if (pVnode->rqueue) - dnodeFreeRqueue(pVnode->rqueue); + dnodeFreeVnodeRqueue(pVnode->rqueue); pVnode->rqueue = NULL; vnodeRelease(pVnode); @@ -467,6 +469,8 @@ static void vnodeNotifyFileSynced(void *ahandle, uint64_t fversion) { appH.appH = (void *)pVnode; appH.notifyStatus = vnodeProcessTsdbStatus; appH.cqH = pVnode->cq; + appH.cqCreateFunc = cqCreate; + appH.cqDropFunc = cqDrop; pVnode->tsdb = tsdbOpenRepo(rootDir, &appH); } diff --git a/src/vnode/src/vnodeRead.c b/src/vnode/src/vnodeRead.c index 03407a5aa217306c5480a50f30fe09f0aceec538..c41171a36e1fe4292ba059495dc01a39c1bc5782 100644 --- a/src/vnode/src/vnodeRead.c +++ b/src/vnode/src/vnodeRead.c @@ -43,7 +43,7 @@ int32_t vnodeProcessRead(void *param, int msgType, void *pCont, int32_t contLen, return TSDB_CODE_MSG_NOT_PROCESSED; if (pVnode->status == TAOS_VN_STATUS_DELETING || pVnode->status == TAOS_VN_STATUS_CLOSING) - return TSDB_CODE_NOT_ACTIVE_VNODE; + return TSDB_CODE_INVALID_VGROUP_ID; return (*vnodeProcessReadMsgFp[msgType])(pVnode, pCont, contLen, ret); } diff --git a/src/vnode/src/vnodeWrite.c b/src/vnode/src/vnodeWrite.c index 9c415d6af756d6c9f015ef50055a00b2c48a910b..09cb2d3fac89f634277ed9133c7723cf1f29cccc 100644 --- a/src/vnode/src/vnodeWrite.c +++ b/src/vnode/src/vnodeWrite.c @@ -53,7 +53,7 @@ int32_t vnodeProcessWrite(void *param1, int qtype, void *param2, void *item) { if (pHead->version == 0) { // from client or CQ if (pVnode->status != TAOS_VN_STATUS_READY) - return TSDB_CODE_NOT_ACTIVE_VNODE; + return TSDB_CODE_INVALID_VGROUP_ID; // it may be in deleting or closing state if (pVnode->syncCfg.replica > 1 && pVnode->role != TAOS_SYNC_ROLE_MASTER) return TSDB_CODE_NOT_READY; @@ -106,7 +106,7 @@ static int32_t vnodeProcessSubmitMsg(SVnodeObj *pVnode, void *pCont, SRspRet *pR static int32_t vnodeProcessCreateTableMsg(SVnodeObj *pVnode, void *pCont, SRspRet *pRet) { SMDCreateTableMsg *pTable = pCont; int32_t code = 0; - + vTrace("vgId:%d, table:%s, start to create", pVnode->vgId, pTable->tableId); int16_t numOfColumns = htons(pTable->numOfColumns); int16_t numOfTags = htons(pTable->numOfTags); @@ -139,16 +139,25 @@ static int32_t vnodeProcessCreateTableMsg(SVnodeObj *pVnode, void *pCont, SRspRe char *pTagData = pTable->data + totalCols * sizeof(SSchema); int accumBytes = 0; - dataRow = tdNewDataRowFromSchema(pDestTagSchema); + //dataRow = tdNewDataRowFromSchema(pDestTagSchema); + dataRow = tdNewTagRowFromSchema(pDestTagSchema, numOfTags); for (int i = 0; i < numOfTags; i++) { STColumn *pTCol = schemaColAt(pDestTagSchema, i); - tdAppendColVal(dataRow, pTagData + accumBytes, pTCol->type, pTCol->bytes, pTCol->offset); +// tdAppendColVal(dataRow, pTagData + accumBytes, pTCol->type, pTCol->bytes, pTCol->offset); + tdAppendTagColVal(dataRow, pTagData + accumBytes, pTCol->type, pTCol->bytes, pTCol->colId); accumBytes += htons(pSchema[i + numOfColumns].bytes); } tsdbTableSetTagValue(&tCfg, dataRow, false); } + // only normal has sql string + if (pTable->tableType == TSDB_STREAM_TABLE) { + char *sql = pTable->data + totalCols * sizeof(SSchema); + vTrace("vgId:%d, table:%s is creating, sql:%s", pVnode->vgId, pTable->tableId, sql); + tsdbTableSetStreamSql(&tCfg, sql, false); + } + code = tsdbCreateTable(pVnode->tsdb, &tCfg); tdFreeDataRow(dataRow); tfree(pDestTagSchema); diff --git a/tests/pytest/crash_gen.py b/tests/pytest/crash_gen.py index 44c978957e0ccf49805ded9e5c26ba0bfb3a656b..2e5d84d45f1cc7545c0af509087f930491d4bd0b 100755 --- a/tests/pytest/crash_gen.py +++ b/tests/pytest/crash_gen.py @@ -14,6 +14,7 @@ from __future__ import annotations # For type hinting before definition, ref: https://stackoverflow.com/questions/33533148/how-do-i-specify-that-the-return-type-of-a-method-is-the-same-as-the-class-itsel import sys +import traceback # Require Python 3 if sys.version_info[0] < 3: raise Exception("Must be using Python 3") @@ -24,11 +25,13 @@ import copy import threading import random +import time import logging import datetime import textwrap from typing import List +from typing import Dict from util.log import * from util.dnodes import * @@ -45,6 +48,14 @@ logger = None def runThread(wt: WorkerThread): wt.run() +class CrashGenError(Exception): + def __init__(self, msg=None, errno=None): + self.msg = msg + self.errno = errno + + def __str__(self): + return self.msg + class WorkerThread: def __init__(self, pool: ThreadPool, tid, tc: ThreadCoordinator, @@ -63,10 +74,10 @@ class WorkerThread: self._dbConn = DbConn() def logDebug(self, msg): - logger.info(" t[{}] {}".format(self._tid, msg)) + logger.debug(" TRD[{}] {}".format(self._tid, msg)) def logInfo(self, msg): - logger.info(" t[{}] {}".format(self._tid, msg)) + logger.info(" TRD[{}] {}".format(self._tid, msg)) def getTaskExecutor(self): @@ -95,15 +106,19 @@ class WorkerThread: while True: tc = self._tc # Thread Coordinator, the overall master tc.crossStepBarrier() # shared barrier first, INCLUDING the last one - logger.debug("Thread task loop exited barrier...") + logger.debug("[TRD] Worker thread [{}] exited barrier...".format(self._tid)) self.crossStepGate() # then per-thread gate, after being tapped - logger.debug("Thread task loop exited step gate...") + logger.debug("[TRD] Worker thread [{}] exited step gate...".format(self._tid)) if not self._tc.isRunning(): + logger.debug("[TRD] Thread Coordinator not running any more, worker thread now stopping...") break + logger.debug("[TRD] Worker thread [{}] about to fetch task".format(self._tid)) task = tc.fetchTask() + logger.debug("[TRD] Worker thread [{}] about to execute task: {}".format(self._tid, task.__class__.__name__)) task.execute(self) tc.saveExecutedTask(task) + logger.debug("[TRD] Worker thread [{}] finished executing task".format(self._tid)) def verifyThreadSelf(self): # ensure we are called by this own thread if ( threading.get_ident() != self._thread.ident ): @@ -123,7 +138,7 @@ class WorkerThread: self.verifyThreadSelf() # only allowed by ourselves # Wait again at the "gate", waiting to be "tapped" - # logger.debug("Worker thread {} about to cross the step gate".format(self._tid)) + logger.debug("[TRD] Worker thread {} about to cross the step gate".format(self._tid)) self._stepGate.wait() self._stepGate.clear() @@ -133,33 +148,40 @@ class WorkerThread: self.verifyThreadAlive() self.verifyThreadMain() # only allowed for main thread - logger.debug("Tapping worker thread {}".format(self._tid)) + logger.debug("[TRD] Tapping worker thread {}".format(self._tid)) self._stepGate.set() # wake up! time.sleep(0) # let the released thread run a bit - def execSql(self, sql): # not "execute", since we are out side the DB context + def execSql(self, sql): # TODO: expose DbConn directly if ( gConfig.per_thread_db_connection ): return self._dbConn.execute(sql) else: return self._tc.getDbState().getDbConn().execute(sql) - def querySql(self, sql): # not "execute", since we are out side the DB context + def getDbConn(self): if ( gConfig.per_thread_db_connection ): - return self._dbConn.query(sql) + return self._dbConn else: - return self._tc.getDbState().getDbConn().query(sql) - + return self._tc.getDbState().getDbConn() + + # def querySql(self, sql): # not "execute", since we are out side the DB context + # if ( gConfig.per_thread_db_connection ): + # return self._dbConn.query(sql) + # else: + # return self._tc.getDbState().getDbConn().query(sql) + class ThreadCoordinator: - def __init__(self, pool, wd: WorkDispatcher, dbState): + def __init__(self, pool, dbState): self._curStep = -1 # first step is 0 self._pool = pool - self._wd = wd + # self._wd = wd self._te = None # prepare for every new step self._dbState = dbState self._executedTasks: List[Task] = [] # in a given step self._lock = threading.RLock() # sync access for a few things self._stepBarrier = threading.Barrier(self._pool.numThreads + 1) # one barrier for all threads + self._execStats = ExecutionStats() def getTaskExecutor(self): return self._te @@ -176,41 +198,63 @@ class ThreadCoordinator: # Coordinate all threads step by step self._curStep = -1 # not started yet maxSteps = gConfig.max_steps # type: ignore - while(self._curStep < maxSteps): - print(".", end="", flush=True) - logger.debug("Main thread going to sleep") + self._execStats.startExec() # start the stop watch + failed = False + while(self._curStep < maxSteps-1 and not failed): # maxStep==10, last curStep should be 9 + if not gConfig.debug: + print(".", end="", flush=True) # print this only if we are not in debug mode + logger.debug("[TRD] Main thread going to sleep") # Now ready to enter a step self.crossStepBarrier() # let other threads go past the pool barrier, but wait at the thread gate self._stepBarrier.reset() # Other worker threads should now be at the "gate" # At this point, all threads should be pass the overall "barrier" and before the per-thread "gate" - self._dbState.transition(self._executedTasks) # at end of step, transiton the DB state + try: + self._dbState.transition(self._executedTasks) # at end of step, transiton the DB state + except taos.error.ProgrammingError as err: + if ( err.msg == 'network unavailable' ): # broken DB connection + logger.info("DB connection broken, execution failed") + traceback.print_stack() + failed = True + self._te = None # Not running any more + self._execStats.registerFailure("Broken DB Connection") + # continue # don't do that, need to tap all threads at end, and maybe signal them to stop + else: + raise + finally: + pass + self.resetExecutedTasks() # clear the tasks after we are done # Get ready for next step - logger.info("<-- Step {} finished".format(self._curStep)) + logger.debug("<-- Step {} finished".format(self._curStep)) self._curStep += 1 # we are about to get into next step. TODO: race condition here! logger.debug("\r\n--> Step {} starts with main thread waking up".format(self._curStep)) # Now not all threads had time to go to sleep # A new TE for the new step - self._te = TaskExecutor(self._curStep) + if not failed: # only if not failed + self._te = TaskExecutor(self._curStep) - logger.debug("Main thread waking up at step {}, tapping worker threads".format(self._curStep)) # Now not all threads had time to go to sleep + logger.debug("[TRD] Main thread waking up at step {}, tapping worker threads".format(self._curStep)) # Now not all threads had time to go to sleep self.tapAllThreads() logger.debug("Main thread ready to finish up...") - self.crossStepBarrier() # Cross it one last time, after all threads finish - self._stepBarrier.reset() - logger.debug("Main thread in exclusive zone...") - self._te = None # No more executor, time to end - logger.debug("Main thread tapping all threads one last time...") - self.tapAllThreads() # Let the threads run one last time + if not failed: # only in regular situations + self.crossStepBarrier() # Cross it one last time, after all threads finish + self._stepBarrier.reset() + logger.debug("Main thread in exclusive zone...") + self._te = None # No more executor, time to end + logger.debug("Main thread tapping all threads one last time...") + self.tapAllThreads() # Let the threads run one last time + logger.debug("Main thread joining all threads") self._pool.joinAll() # Get all threads to finish + logger.info("All worker thread finished") + self._execStats.endExec() - logger.info("All threads finished") - print("\r\nFinished") + def logStats(self): + self._execStats.logStats() def tapAllThreads(self): # in a deterministic manner wakeSeq = [] @@ -219,7 +263,7 @@ class ThreadCoordinator: wakeSeq.append(i) else: wakeSeq.insert(0, i) - logger.info("Waking up threads: {}".format(str(wakeSeq))) + logger.debug("[TRD] Main thread waking up worker thread: {}".format(str(wakeSeq))) # TODO: set dice seed to a deterministic value for i in wakeSeq: self._pool.threadList[i].tapStepGate() # TODO: maybe a bit too deep?! @@ -233,11 +277,15 @@ class ThreadCoordinator: raise RuntimeError("Cannot fetch task when not running") # return self._wd.pickTask() # Alternatively, let's ask the DbState for the appropriate task - dbState = self.getDbState() - tasks = dbState.getTasksAtState() - i = Dice.throw(len(tasks)) - # return copy.copy(tasks[i]) # Needs a fresh copy, to save execution results, etc. - return tasks[i].clone() + # dbState = self.getDbState() + # tasks = dbState.getTasksAtState() # TODO: create every time? + # nTasks = len(tasks) + # i = Dice.throw(nTasks) + # logger.debug(" (dice:{}/{}) ".format(i, nTasks)) + # # return copy.copy(tasks[i]) # Needs a fresh copy, to save execution results, etc. + # return tasks[i].clone() # TODO: still necessary? + taskType = self.getDbState().pickTaskType() # pick a task type for current state + return taskType(self.getDbState(), self._execStats) # create a task from it def resetExecutedTasks(self): self._executedTasks = [] # should be under single thread @@ -253,7 +301,7 @@ class ThreadPool: self.maxSteps = maxSteps self.funcSequencer = funcSequencer # Internal class variables - self.dispatcher = WorkDispatcher(dbState) + # self.dispatcher = WorkDispatcher(dbState) # Obsolete? self.curStep = 0 self.threadList = [] # self.stepGate = threading.Condition() # Gate to hold/sync all threads @@ -362,7 +410,7 @@ class DbConn: # Get the connection/cursor ready self._cursor.execute('reset query cache') - # self._cursor.execute('use db') + # self._cursor.execute('use db') # note we do this in _findCurrenState # Open connection self._tdSql = TDSql() @@ -390,28 +438,265 @@ class DbConn: def execute(self, sql): if ( not self.isOpen ): raise RuntimeError("Cannot execute database commands until connection is open") - return self._tdSql.execute(sql) + logger.debug("[SQL] Executing SQL: {}".format(sql)) + nRows = self._tdSql.execute(sql) + logger.debug("[SQL] Execution Result, nRows = {}, SQL = {}".format(nRows, sql)) + return nRows - def query(self, sql) -> int : # return number of rows retrieved + def query(self, sql) : # return rows affected if ( not self.isOpen ): raise RuntimeError("Cannot query database until connection is open") - return self._tdSql.query(sql) + logger.debug("[SQL] Executing SQL: {}".format(sql)) + nRows = self._tdSql.query(sql) + logger.debug("[SQL] Execution Result, nRows = {}, SQL = {}".format(nRows, sql)) + return nRows + # results are in: return self._tdSql.queryResult + def getQueryResult(self): + return self._tdSql.queryResult -# State of the database as we believe it to be -class DbState(): + def _queryAny(self, sql) : # actual query result as an int + if ( not self.isOpen ): + raise RuntimeError("Cannot query database until connection is open") + tSql = self._tdSql + nRows = tSql.query(sql) + if nRows != 1 : + raise RuntimeError("Unexpected result for query: {}, rows = {}".format(sql, nRows)) + if tSql.queryRows != 1 or tSql.queryCols != 1: + raise RuntimeError("Unexpected result set for query: {}".format(sql)) + return tSql.queryResult[0][0] + + def queryScalar(self, sql) -> int : + return self._queryAny(sql) + + def queryString(self, sql) -> str : + return self._queryAny(sql) + +class AnyState: STATE_INVALID = -1 - STATE_EMPTY = 1 # nothing there, no even a DB - STATE_DB_ONLY = 2 # we have a DB, but nothing else - STATE_TABLE_ONLY = 3 # we have a table, but totally empty - STATE_HAS_DATA = 4 # we have some data in the table + STATE_EMPTY = 0 # nothing there, no even a DB + STATE_DB_ONLY = 1 # we have a DB, but nothing else + STATE_TABLE_ONLY = 2 # we have a table, but totally empty + STATE_HAS_DATA = 3 # we have some data in the table + _stateNames = ["Invalid", "Empty", "DB_Only", "Table_Only", "Has_Data"] + + STATE_VAL_IDX = 0 + CAN_CREATE_DB = 1 + CAN_DROP_DB = 2 + CAN_CREATE_FIXED_SUPER_TABLE = 3 + CAN_DROP_FIXED_SUPER_TABLE = 4 + CAN_ADD_DATA = 5 + CAN_READ_DATA = 6 + + def __init__(self): + self._info = self.getInfo() + + def __str__(self): + return self._stateNames[self._info[self.STATE_VAL_IDX] + 1] # -1 hack to accomodate the STATE_INVALID case + + def getInfo(self): + raise RuntimeError("Must be overriden by child classes") + + def equals(self, other): + if isinstance(other, int): + return self.getValIndex() == other + elif isinstance(other, AnyState): + return self.getValIndex() == other.getValIndex() + else: + raise RuntimeError("Unexpected comparison, type = {}".format(type(other))) + + def verifyTasksToState(self, tasks, newState): + raise RuntimeError("Must be overriden by child classes") + + def getValIndex(self): + return self._info[self.STATE_VAL_IDX] + + def getValue(self): + return self._info[self.STATE_VAL_IDX] + def canCreateDb(self): + return self._info[self.CAN_CREATE_DB] + def canDropDb(self): + return self._info[self.CAN_DROP_DB] + def canCreateFixedSuperTable(self): + return self._info[self.CAN_CREATE_FIXED_SUPER_TABLE] + def canDropFixedSuperTable(self): + return self._info[self.CAN_DROP_FIXED_SUPER_TABLE] + def canAddData(self): + return self._info[self.CAN_ADD_DATA] + def canReadData(self): + return self._info[self.CAN_READ_DATA] + + def assertAtMostOneSuccess(self, tasks, cls): + sCnt = 0 + for task in tasks : + if not isinstance(task, cls): + continue + if task.isSuccess(): + # task.logDebug("Task success found") + sCnt += 1 + if ( sCnt >= 2 ): + raise RuntimeError("Unexpected more than 1 success with task: {}".format(cls)) + + def assertIfExistThenSuccess(self, tasks, cls): + sCnt = 0 + exists = False + for task in tasks : + if not isinstance(task, cls): + continue + exists = True # we have a valid instance + if task.isSuccess(): + sCnt += 1 + if ( exists and sCnt <= 0 ): + raise RuntimeError("Unexpected zero success for task: {}".format(cls)) + + def assertNoTask(self, tasks, cls): + for task in tasks : + if isinstance(task, cls): + raise CrashGenError("This task: {}, is not expected to be present, given the success/failure of others".format(cls.__name__)) + + def assertNoSuccess(self, tasks, cls): + for task in tasks : + if isinstance(task, cls): + if task.isSuccess(): + raise RuntimeError("Unexpected successful task: {}".format(cls)) + + def hasSuccess(self, tasks, cls): + for task in tasks : + if not isinstance(task, cls): + continue + if task.isSuccess(): + return True + return False + + def hasTask(self, tasks, cls): + for task in tasks : + if isinstance(task, cls): + return True + return False + +class StateInvalid(AnyState): + def getInfo(self): + return [ + self.STATE_INVALID, + False, False, # can create/drop Db + False, False, # can create/drop fixed table + False, False, # can insert/read data with fixed table + ] + + # def verifyTasksToState(self, tasks, newState): +class StateEmpty(AnyState): + def getInfo(self): + return [ + self.STATE_EMPTY, + True, False, # can create/drop Db + False, False, # can create/drop fixed table + False, False, # can insert/read data with fixed table + ] + + def verifyTasksToState(self, tasks, newState): + if ( self.hasSuccess(tasks, CreateDbTask) ): # at EMPTY, if there's succes in creating DB + if ( not self.hasTask(tasks, DropDbTask) ) : # and no drop_db tasks + self.assertAtMostOneSuccess(tasks, CreateDbTask) # we must have at most one. TODO: compare numbers + +class StateDbOnly(AnyState): + def getInfo(self): + return [ + self.STATE_DB_ONLY, + False, True, + True, False, + False, False, + ] + + def verifyTasksToState(self, tasks, newState): + self.assertAtMostOneSuccess(tasks, DropDbTask) # not true in massively parralel cases + self.assertIfExistThenSuccess(tasks, DropDbTask) + # self.assertAtMostOneSuccess(tasks, CreateFixedTableTask) # not true in massively parrallel cases + # Nothing to be said about adding data task + if ( self.hasSuccess(tasks, DropDbTask) ): # dropped the DB + # self.assertHasTask(tasks, DropDbTask) # implied by hasSuccess + self.assertAtMostOneSuccess(tasks, DropDbTask) + # self._state = self.STATE_EMPTY + elif ( self.hasSuccess(tasks, CreateFixedSuperTableTask) ): # did not drop db, create table success + # self.assertHasTask(tasks, CreateFixedTableTask) # tried to create table + self.assertAtMostOneSuccess(tasks, CreateFixedSuperTableTask) # at most 1 attempt is successful + self.assertNoTask(tasks, DropDbTask) # should have have tried + # if ( not self.hasSuccess(tasks, AddFixedDataTask) ): # just created table, no data yet + # # can't say there's add-data attempts, since they may all fail + # self._state = self.STATE_TABLE_ONLY + # else: + # self._state = self.STATE_HAS_DATA + # What about AddFixedData? + # elif ( self.hasSuccess(tasks, AddFixedDataTask) ): + # self._state = self.STATE_HAS_DATA + # else: # no success in dropping db tasks, no success in create fixed table? read data should also fail + # # raise RuntimeError("Unexpected no-success scenario") # We might just landed all failure tasks, + # self._state = self.STATE_DB_ONLY # no change + +class StateSuperTableOnly(AnyState): + def getInfo(self): + return [ + self.STATE_TABLE_ONLY, + False, True, + False, True, + True, True, + ] + + def verifyTasksToState(self, tasks, newState): + if ( self.hasSuccess(tasks, DropFixedSuperTableTask) ): # we are able to drop the table + self.assertAtMostOneSuccess(tasks, DropFixedSuperTableTask) + # self._state = self.STATE_DB_ONLY + # elif ( self.hasSuccess(tasks, AddFixedDataTask) ): # no success dropping the table, but added data + # self.assertNoTask(tasks, DropFixedTableTask) # not true in massively parrallel cases + # self._state = self.STATE_HAS_DATA + # elif ( self.hasSuccess(tasks, ReadFixedDataTask) ): # no success in prev cases, but was able to read data + # self.assertNoTask(tasks, DropFixedTableTask) + # self.assertNoTask(tasks, AddFixedDataTask) + # self._state = self.STATE_TABLE_ONLY # no change + # else: # did not drop table, did not insert data, did not read successfully, that is impossible + # raise RuntimeError("Unexpected no-success scenarios") + # TODO: need to revamp!! + +class StateHasData(AnyState): + def getInfo(self): + return [ + self.STATE_HAS_DATA, + False, True, + False, True, + True, True, + ] + + def verifyTasksToState(self, tasks, newState): + if ( newState.equals(AnyState.STATE_EMPTY) ): + self.hasSuccess(tasks, DropDbTask) + self.assertAtMostOneSuccess(tasks, DropDbTask) # TODO: dicy + elif ( newState.equals(AnyState.STATE_DB_ONLY) ): # in DB only + if ( not self.hasTask(tasks, CreateDbTask)): # without a create_db task + self.assertNoTask(tasks, DropDbTask) # we must have drop_db task + self.hasSuccess(tasks, DropFixedSuperTableTask) + self.assertAtMostOneSuccess(tasks, DropFixedSuperTableTask) # TODO: dicy + elif ( newState.equals(AnyState.STATE_TABLE_ONLY) ): # data deleted + self.assertNoTask(tasks, DropDbTask) + self.assertNoTask(tasks, DropFixedSuperTableTask) + self.assertNoTask(tasks, AddFixedDataTask) + # self.hasSuccess(tasks, DeleteDataTasks) + else: + self.assertNoTask(tasks, DropDbTask) + self.assertNoTask(tasks, DropFixedSuperTableTask) + self.assertIfExistThenSuccess(tasks, ReadFixedDataTask) + + +# State of the database as we believe it to be +class DbState(): + def __init__(self): self.tableNumQueue = LinearQueue() self._lastTick = datetime.datetime(2019, 1, 1) # initial date time tick self._lastInt = 0 # next one is initial integer self._lock = threading.RLock() - self._state = self.STATE_INVALID + + self._state = StateInvalid() # starting state + self._stateWeights = [1,3,5,10] # indexed with value of STATE_EMPTY, STATE_DB_ONLY, etc. # self.openDbServerConnection() self._dbConn = DbConn() @@ -425,10 +710,10 @@ class DbState(): else: raise except: - print("[=]Unexpected exception") + print("[=] Unexpected exception") raise self._dbConn.resetDb() # drop and recreate DB - self._state = self.STATE_EMPTY # initial state, the result of above + self._state = StateEmpty() # initial state, the result of above def getDbConn(self): return self._dbConn @@ -441,8 +726,8 @@ class DbState(): tIndex = self.tableNumQueue.push() return tIndex - def getFixedTableName(self): - return "fixed_table" + def getFixedSuperTableName(self): + return "fs_table" def releaseTable(self, i): # return the table back, so others can use it self.tableNumQueue.release(i) @@ -456,6 +741,12 @@ class DbState(): with self._lock: self._lastInt += 1 return self._lastInt + + def getNextBinary(self): + return "Los_Angeles_{}".format(self.getNextInt()) + + def getNextFloat(self): + return 0.9 + self.getNextInt() def getTableNameToDelete(self): tblNum = self.tableNumQueue.pop() # TODO: race condition! @@ -464,134 +755,124 @@ class DbState(): return "table_{}".format(tblNum) - def execSql(self, sql): # using the main DB connection - return self._dbConn.execute(sql) - def cleanUp(self): self._dbConn.close() - def getTasksAtState(self): - tasks = [] - tasks.append(ReadFixedDataTask(self)) # always - if ( self._state == self.STATE_EMPTY ): - tasks.append(CreateDbTask(self)) - tasks.append(CreateFixedTableTask(self)) - elif ( self._state == self.STATE_DB_ONLY ): - tasks.append(DropDbTask(self)) - tasks.append(CreateFixedTableTask(self)) - tasks.append(AddFixedDataTask(self)) - elif ( self._state == self.STATE_TABLE_ONLY ): - tasks.append(DropFixedTableTask(self)) - tasks.append(AddFixedDataTask(self)) - elif ( self._state == self.STATE_HAS_DATA ) : # same as above. TODO: adjust - tasks.append(DropFixedTableTask(self)) - tasks.append(AddFixedDataTask(self)) + # May be slow, use cautionsly... + def getTaskTypesAtState(self): + allTaskClasses = StateTransitionTask.__subclasses__() # all state transition tasks + firstTaskTypes = [] + for tc in allTaskClasses: + # t = tc(self) # create task object + if tc.canBeginFrom(self._state): + firstTaskTypes.append(tc) + # now we have all the tasks that can begin directly from the current state, let's figure out the INDIRECT ones + taskTypes = firstTaskTypes.copy() # have to have these + for task1 in firstTaskTypes: # each task type gathered so far + endState = task1.getEndState() # figure the end state + if endState == None: + continue + for tc in allTaskClasses: # what task can further begin from there? + if tc.canBeginFrom(endState) and (tc not in firstTaskTypes): + taskTypes.append(tc) # gather it + + if len(taskTypes) <= 0: + raise RuntimeError("No suitable task types found for state: {}".format(self._state)) + logger.debug("[OPS] Tasks found for state {}: {}".format(self._state, taskTypes)) + return taskTypes + + # tasks.append(ReadFixedDataTask(self)) # always for everybody + # if ( self._state == self.STATE_EMPTY ): + # tasks.append(CreateDbTask(self)) + # tasks.append(CreateFixedTableTask(self)) + # elif ( self._state == self.STATE_DB_ONLY ): + # tasks.append(DropDbTask(self)) + # tasks.append(CreateFixedTableTask(self)) + # tasks.append(AddFixedDataTask(self)) + # elif ( self._state == self.STATE_TABLE_ONLY ): + # tasks.append(DropFixedTableTask(self)) + # tasks.append(AddFixedDataTask(self)) + # elif ( self._state == self.STATE_HAS_DATA ) : # same as above. TODO: adjust + # tasks.append(DropFixedTableTask(self)) + # tasks.append(AddFixedDataTask(self)) + # else: + # raise RuntimeError("Unexpected DbState state: {}".format(self._state)) + # return tasks + + def pickTaskType(self): + taskTypes = self.getTaskTypesAtState() # all the task types we can choose from at curent state + weights = [] + for tt in taskTypes: + endState = tt.getEndState() + if endState != None : + weights.append(self._stateWeights[endState.getValIndex()]) # TODO: change to a method + else: + weights.append(10) # read data task, default to 10: TODO: change to a constant + i = self._weighted_choice_sub(weights) + # logger.debug(" (weighted random:{}/{}) ".format(i, len(taskTypes))) + return taskTypes[i] + + def _weighted_choice_sub(self, weights): # ref: https://eli.thegreenplace.net/2010/01/22/weighted-random-generation-in-python/ + rnd = random.random() * sum(weights) # TODO: use our dice to ensure it being determinstic? + for i, w in enumerate(weights): + rnd -= w + if rnd < 0: + return i + + def _findCurrentState(self): + dbc = self._dbConn + ts = time.time() + if dbc.query("show databases") == 0 : # no database?! + # logger.debug("Found EMPTY state") + logger.debug("[STT] empty database found, between {} and {}".format(ts, time.time())) + return StateEmpty() + dbc.execute("use db") # did not do this when openning connection + if dbc.query("show tables") == 0 : # no tables + # logger.debug("Found DB ONLY state") + logger.debug("[STT] DB_ONLY found, between {} and {}".format(ts, time.time())) + return StateDbOnly() + if dbc.query("SELECT * FROM db.{}".format(self.getFixedSuperTableName()) ) == 0 : # no data + # logger.debug("Found TABLE_ONLY state") + logger.debug("[STT] SUPER_TABLE_ONLY found, between {} and {}".format(ts, time.time())) + return StateSuperTableOnly() else: - raise RuntimeError("Unexpected DbState state: {}".format(self._state)) - return tasks - + # logger.debug("Found HAS_DATA state") + logger.debug("[STT] HAS_DATA found, between {} and {}".format(ts, time.time())) + return StateHasData() + def transition(self, tasks): if ( len(tasks) == 0 ): # before 1st step, or otherwise empty return # do nothing - if ( self._state == self.STATE_EMPTY ): - # self.assertNoSuccess(tasks, ReadFixedDataTask) # some read may be successful, since we might be creating a table - if ( self.hasSuccess(tasks, CreateDbTask) ): - self.assertAtMostOneSuccess(tasks, CreateDbTask) # param is class - self._state = self.STATE_DB_ONLY - if ( self.hasSuccess(tasks, CreateFixedTableTask )): - self._state = self.STATE_TABLE_ONLY - # else: # no successful table creation, not much we can say, as it is step 2 - else: # did not create db - self.assertNoTask(tasks, CreateDbTask) # because we did not have such task - # self.assertNoSuccess(tasks, CreateDbTask) # not necessary, since we just verified no such task - self.assertNoSuccess(tasks, CreateFixedTableTask) - - elif ( self._state == self.STATE_DB_ONLY ): - self.assertAtMostOneSuccess(tasks, DropDbTask) - self.assertIfExistThenSuccess(tasks, DropDbTask) - self.assertAtMostOneSuccess(tasks, CreateFixedTableTask) - # Nothing to be said about adding data task - if ( self.hasSuccess(tasks, DropDbTask) ): # dropped the DB - # self.assertHasTask(tasks, DropDbTask) # implied by hasSuccess - self.assertAtMostOneSuccess(tasks, DropDbTask) - self._state = self.STATE_EMPTY - elif ( self.hasSuccess(tasks, CreateFixedTableTask) ): # did not drop db, create table success - # self.assertHasTask(tasks, CreateFixedTableTask) # tried to create table - self.assertAtMostOneSuccess(tasks, CreateFixedTableTask) # at most 1 attempt is successful - self.assertNoTask(tasks, DropDbTask) # should have have tried - if ( not self.hasSuccess(tasks, AddFixedDataTask) ): # just created table, no data yet - # can't say there's add-data attempts, since they may all fail - self._state = self.STATE_TABLE_ONLY - else: - self._state = self.STATE_HAS_DATA - else: # no success in dropping db tasks, no success in create fixed table, not acceptable - raise RuntimeError("Unexpected no-success scenario") - - elif ( self._state == self.STATE_TABLE_ONLY ): - if ( self.hasSuccess(tasks, DropFixedTableTask) ): - self.assertAtMostOneSuccess(tasks, DropFixedTableTask) - self._state = self.STATE_DB_ONLY - elif ( self.hasSuccess(tasks, AddFixedDataTask) ): # no success dropping the table - self.assertNoTask(tasks, DropFixedTableTask) - self._state = self.STATE_HAS_DATA - else: # did not drop table, did not insert data, that is impossible - raise RuntimeError("Unexpected no-success scenarios") - - elif ( self._state == self.STATE_HAS_DATA ): # Same as above, TODO: adjust - if ( self.hasSuccess(tasks, DropFixedTableTask) ): - self.assertAtMostOneSuccess(tasks, DropFixedTableTask) - self._state = self.STATE_DB_ONLY - elif ( self.hasSuccess(tasks, AddFixedDataTask) ): # no success dropping the table - self.assertNoTask(tasks, DropFixedTableTask) - self._state = self.STATE_HAS_DATA - else: # did not drop table, did not insert data, that is impossible - raise RuntimeError("Unexpected no-success scenarios") - else: - raise RuntimeError("Unexpected DbState state: {}".format(self._state)) - logger.debug("New DB state is: {}".format(self._state)) + self._dbConn.execute("show dnodes") # this should show up in the server log, separating steps - def assertAtMostOneSuccess(self, tasks, cls): - sCnt = 0 - for task in tasks : - if not isinstance(task, cls): - continue - if task.isSuccess(): - task.logDebug("Task success found") - sCnt += 1 - if ( sCnt >= 2 ): - raise RuntimeError("Unexpected more than 1 success with task: {}".format(cls)) + # Generic Checks, first based on the start state + if self._state.canCreateDb(): + self._state.assertIfExistThenSuccess(tasks, CreateDbTask) + # self.assertAtMostOneSuccess(tasks, CreateDbTask) # not really, in case of multiple creation and drops - def assertIfExistThenSuccess(self, tasks, cls): - sCnt = 0 - exists = False - for task in tasks : - if not isinstance(task, cls): - continue - exists = True # we have a valid instance - if task.isSuccess(): - sCnt += 1 - if ( exists and sCnt <= 0 ): - raise RuntimeError("Unexpected zero success for task: {}".format(cls)) + if self._state.canDropDb(): + self._state.assertIfExistThenSuccess(tasks, DropDbTask) + # self.assertAtMostOneSuccess(tasks, DropDbTask) # not really in case of drop-create-drop - def assertNoTask(self, tasks, cls): - for task in tasks : - if isinstance(task, cls): - raise RuntimeError("Unexpected task: {}".format(cls)) + # if self._state.canCreateFixedTable(): + # self.assertIfExistThenSuccess(tasks, CreateFixedTableTask) # Not true, DB may be dropped + # self.assertAtMostOneSuccess(tasks, CreateFixedTableTask) # not really, in case of create-drop-create - def assertNoSuccess(self, tasks, cls): - for task in tasks : - if isinstance(task, cls): - if task.isSuccess(): - raise RuntimeError("Unexpected successful task: {}".format(cls)) + # if self._state.canDropFixedTable(): + # self.assertIfExistThenSuccess(tasks, DropFixedTableTask) # Not True, the whole DB may be dropped + # self.assertAtMostOneSuccess(tasks, DropFixedTableTask) # not really in case of drop-create-drop - def hasSuccess(self, tasks, cls): - for task in tasks : - if not isinstance(task, cls): - continue - if task.isSuccess(): - return True - return False + # if self._state.canAddData(): + # self.assertIfExistThenSuccess(tasks, AddFixedDataTask) # not true actually + + # if self._state.canReadData(): + # Nothing for sure + + newState = self._findCurrentState() + logger.debug("[STT] New DB state determined: {}".format(newState)) + self._state.verifyTasksToState(tasks, newState) # can old state move to new state through the tasks? + self._state = newState class TaskExecutor(): def __init__(self, curStep): @@ -614,10 +895,11 @@ class Task(): @classmethod def allocTaskNum(cls): - cls.taskSn += 1 - return cls.taskSn + Task.taskSn += 1 # IMPORTANT: cannot use cls.taskSn, since each sub class will have a copy + # logger.debug("Allocating taskSN: {}".format(Task.taskSn)) + return Task.taskSn - def __init__(self, dbState: DbState): + def __init__(self, dbState: DbState, execStats: ExecutionStats): self._dbState = dbState self._workerThread = None self._err = None @@ -626,19 +908,22 @@ class Task(): # Assign an incremental task serial number self._taskNum = self.allocTaskNum() + # logger.debug("Creating new task {}...".format(self._taskNum)) + + self._execStats = execStats def isSuccess(self): return self._err == None - def clone(self): - newTask = self.__class__(self._dbState) + def clone(self): # TODO: why do we need this again? + newTask = self.__class__(self._dbState, self._execStats) return newTask def logDebug(self, msg): - self._workerThread.logDebug("s[{}.{}] {}".format(self._curStep, self._taskNum, msg)) + self._workerThread.logDebug("Step[{}.{}] {}".format(self._curStep, self._taskNum, msg)) def logInfo(self, msg): - self._workerThread.logInfo("s[{}.{}] {}".format(self._curStep, self._taskNum, msg)) + self._workerThread.logInfo("Step[{}.{}] {}".format(self._curStep, self._taskNum, msg)) def _executeInternal(self, te: TaskExecutor, wt: WorkerThread): raise RuntimeError("To be implemeted by child classes, class name: {}".format(self.__class__.__name__)) @@ -652,29 +937,228 @@ class Task(): self.logDebug("[-] executing task {}...".format(self.__class__.__name__)) self._err = None + self._execStats.beginTaskType(self.__class__.__name__) # mark beginning try: self._executeInternal(te, wt) # TODO: no return value? except taos.error.ProgrammingError as err: - self.logDebug("[=]Taos Execution exception: {0}".format(err)) + self.logDebug("[=] Taos library exception: errno={}, msg: {}".format(err.errno, err)) self._err = err except: - self.logDebug("[=]Unexpected exception") + self.logDebug("[=] Unexpected exception") raise + self._execStats.endTaskType(self.__class__.__name__, self.isSuccess()) - self.logDebug("[X] task execution completed, {}, status: {}".format(self.__class__.__name__, "Success" if self.isSuccess() else "Failure")) + self.logDebug("[X] task execution completed, {}, status: {}".format(self.__class__.__name__, "Success" if self.isSuccess() else "Failure")) + self._execStats.incExecCount(self.__class__.__name__, self.isSuccess()) # TODO: merge with above. def execSql(self, sql): return self._dbState.execute(sql) -class CreateDbTask(Task): + +class ExecutionStats: + def __init__(self): + self._execTimes: Dict[str, [int, int]] = {} # total/success times for a task + self._tasksInProgress = 0 + self._lock = threading.Lock() + self._firstTaskStartTime = None + self._execStartTime = None + self._elapsedTime = 0.0 # total elapsed time + self._accRunTime = 0.0 # accumulated run time + + self._failed = False + self._failureReason = None + + def startExec(self): + self._execStartTime = time.time() + + def endExec(self): + self._elapsedTime = time.time() - self._execStartTime + + def incExecCount(self, klassName, isSuccess): # TODO: add a lock here + if klassName not in self._execTimes: + self._execTimes[klassName] = [0, 0] + t = self._execTimes[klassName] # tuple for the data + t[0] += 1 # index 0 has the "total" execution times + if isSuccess: + t[1] += 1 # index 1 has the "success" execution times + + def beginTaskType(self, klassName): + with self._lock: + if self._tasksInProgress == 0 : # starting a new round + self._firstTaskStartTime = time.time() # I am now the first task + self._tasksInProgress += 1 + + def endTaskType(self, klassName, isSuccess): + with self._lock: + self._tasksInProgress -= 1 + if self._tasksInProgress == 0 : # all tasks have stopped + self._accRunTime += (time.time() - self._firstTaskStartTime) + self._firstTaskStartTime = None + + def registerFailure(self, reason): + self._failed = True + self._failureReason = reason + + def logStats(self): + logger.info("----------------------------------------------------------------------") + logger.info("| Crash_Gen test {}, with the following stats:". + format("FAILED (reason: {})".format(self._failureReason) if self._failed else "SUCCEEDED")) + logger.info("| Task Execution Times (success/total):") + execTimesAny = 0 + for k, n in self._execTimes.items(): + execTimesAny += n[0] + logger.info("| {0:<24}: {1}/{2}".format(k,n[1],n[0])) + + logger.info("| Total Tasks Executed (success or not): {} ".format(execTimesAny)) + logger.info("| Total Tasks In Progress at End: {}".format(self._tasksInProgress)) + logger.info("| Total Task Busy Time (elapsed time when any task is in progress): {:.3f} seconds".format(self._accRunTime)) + logger.info("| Average Per-Task Execution Time: {:.3f} seconds".format(self._accRunTime/execTimesAny)) + logger.info("| Total Elapsed Time (from wall clock): {:.3f} seconds".format(self._elapsedTime)) + logger.info("----------------------------------------------------------------------") + + + +class StateTransitionTask(Task): + # @classmethod + # def getAllTaskClasses(cls): # static + # return cls.__subclasses__() + @classmethod + def getInfo(cls): # each sub class should supply their own information + raise RuntimeError("Overriding method expected") + + # @classmethod + # def getBeginStates(cls): + # return cls.getInfo()[0] + + @classmethod + def getEndState(cls): # returning the class name + return cls.getInfo()[0] + + @classmethod + def canBeginFrom(cls, state: AnyState): + # return state.getValue() in cls.getBeginStates() + raise RuntimeError("must be overriden") + + def execute(self, wt: WorkerThread): + super().execute(wt) + + + +class CreateDbTask(StateTransitionTask): + @classmethod + def getInfo(cls): + return [ + # [AnyState.STATE_EMPTY], # can begin from + StateDbOnly() # end state + ] + + @classmethod + def canBeginFrom(cls, state: AnyState): + return state.canCreateDb() + def _executeInternal(self, te: TaskExecutor, wt: WorkerThread): - wt.execSql("create database db") + wt.execSql("create database db") + +class DropDbTask(StateTransitionTask): + @classmethod + def getInfo(cls): + return [ + # [AnyState.STATE_DB_ONLY, AnyState.STATE_TABLE_ONLY, AnyState.STATE_HAS_DATA], + StateEmpty() + ] + + @classmethod + def canBeginFrom(cls, state: AnyState): + return state.canDropDb() -class DropDbTask(Task): def _executeInternal(self, te: TaskExecutor, wt: WorkerThread): wt.execSql("drop database db") + logger.debug("[OPS] database dropped at {}".format(time.time())) + +class CreateFixedSuperTableTask(StateTransitionTask): + @classmethod + def getInfo(cls): + return [ + # [AnyState.STATE_DB_ONLY], + StateSuperTableOnly() + ] + + @classmethod + def canBeginFrom(cls, state: AnyState): + return state.canCreateFixedSuperTable() + + def _executeInternal(self, te: TaskExecutor, wt: WorkerThread): + tblName = self._dbState.getFixedSuperTableName() + wt.execSql("create table db.{} (ts timestamp, speed int) tags (b binary(20), f float) ".format(tblName)) + # No need to create the regular tables, INSERT will do that automatically + + +class ReadFixedDataTask(StateTransitionTask): + @classmethod + def getInfo(cls): + return [ + # [AnyState.STATE_TABLE_ONLY, AnyState.STATE_HAS_DATA], + None # meaning doesn't affect state + ] + + @classmethod + def canBeginFrom(cls, state: AnyState): + return state.canReadData() -class CreateTableTask(Task): + def _executeInternal(self, te: TaskExecutor, wt: WorkerThread): + sTbName = self._dbState.getFixedSuperTableName() + dbc = wt.getDbConn() + dbc.query("select TBNAME from db.{}".format(sTbName)) # TODO: analyze result set later + rTables = dbc.getQueryResult() + # print("rTables[0] = {}, type = {}".format(rTables[0], type(rTables[0]))) + for rTbName in rTables : # regular tables + dbc.query("select * from db.{}".format(rTbName[0])) # TODO: check success failure + + # tdSql.query(" cars where tbname in ('carzero', 'carone')") + +class DropFixedSuperTableTask(StateTransitionTask): + @classmethod + def getInfo(cls): + return [ + # [AnyState.STATE_TABLE_ONLY, AnyState.STATE_HAS_DATA], + StateDbOnly() # meaning doesn't affect state + ] + + @classmethod + def canBeginFrom(cls, state: AnyState): + return state.canDropFixedSuperTable() + + def _executeInternal(self, te: TaskExecutor, wt: WorkerThread): + tblName = self._dbState.getFixedSuperTableName() + wt.execSql("drop table db.{}".format(tblName)) + +class AddFixedDataTask(StateTransitionTask): + @classmethod + def getInfo(cls): + return [ + # [AnyState.STATE_TABLE_ONLY, AnyState.STATE_HAS_DATA], + StateHasData() + ] + + @classmethod + def canBeginFrom(cls, state: AnyState): + return state.canAddData() + + def _executeInternal(self, te: TaskExecutor, wt: WorkerThread): + ds = self._dbState + wt.execSql("use db") # TODO: seems to be an INSERT bug to require this + for i in range(10): # 0 to 9 + sql = "insert into db.reg_table_{} using {} tags ('{}', {}) values ('{}', {});".format( + i, + ds.getFixedSuperTableName(), + ds.getNextBinary(), ds.getNextFloat(), + ds.getNextTick(), ds.getNextInt()) + wt.execSql(sql) + + +#---------- Non State-Transition Related Tasks ----------# + +class CreateTableTask(Task): def _executeInternal(self, te: TaskExecutor, wt: WorkerThread): tIndex = self._dbState.addTable() self.logDebug("Creating a table {} ...".format(tIndex)) @@ -682,17 +1166,6 @@ class CreateTableTask(Task): self.logDebug("Table {} created.".format(tIndex)) self._dbState.releaseTable(tIndex) -class CreateFixedTableTask(Task): - def _executeInternal(self, te: TaskExecutor, wt: WorkerThread): - tblName = self._dbState.getFixedTableName() - wt.execSql("create table db.{} (ts timestamp, speed int)".format(tblName)) - -class ReadFixedDataTask(Task): - def _executeInternal(self, te: TaskExecutor, wt: WorkerThread): - tblName = self._dbState.getFixedTableName() - self._numRows = wt.querySql("select * from db.{}".format(tblName)) # save the result for later - # tdSql.query(" cars where tbname in ('carzero', 'carone')") - class DropTableTask(Task): def _executeInternal(self, te: TaskExecutor, wt: WorkerThread): tableName = self._dbState.getTableNameToDelete() @@ -702,10 +1175,7 @@ class DropTableTask(Task): self.logInfo("Dropping a table db.{} ...".format(tableName)) wt.execSql("drop table db.{}".format(tableName)) -class DropFixedTableTask(Task): - def _executeInternal(self, te: TaskExecutor, wt: WorkerThread): - tblName = self._dbState.getFixedTableName() - wt.execSql("drop table db.{}".format(tblName)) + class AddDataTask(Task): def _executeInternal(self, te: TaskExecutor, wt: WorkerThread): @@ -716,16 +1186,11 @@ class AddDataTask(Task): self.logInfo("No table found to add data, skipping...") return sql = "insert into db.table_{} values ('{}', {});".format(tIndex, ds.getNextTick(), ds.getNextInt()) - self.logDebug("Executing SQL: {}".format(sql)) + self.logDebug("[SQL] Executing SQL: {}".format(sql)) wt.execSql(sql) ds.releaseTable(tIndex) - self.logDebug("Finished adding data") + self.logDebug("[OPS] Finished adding data") -class AddFixedDataTask(Task): - def _executeInternal(self, te: TaskExecutor, wt: WorkerThread): - ds = self._dbState - sql = "insert into db.table_{} values ('{}', {});".format(ds.getFixedTableName(), ds.getNextTick(), ds.getNextInt()) - wt.execSql(sql) # Deterministic random number generator class Dice(): @@ -760,28 +1225,45 @@ class Dice(): # Anyone needing to carry out work should simply come here -class WorkDispatcher(): - def __init__(self, dbState): - # self.totalNumMethods = 2 - self.tasks = [ - CreateTableTask(dbState), - DropTableTask(dbState), - AddDataTask(dbState), - ] +# class WorkDispatcher(): +# def __init__(self, dbState): +# # self.totalNumMethods = 2 +# self.tasks = [ +# # CreateTableTask(dbState), # Obsolete +# # DropTableTask(dbState), +# # AddDataTask(dbState), +# ] + +# def throwDice(self): +# max = len(self.tasks) - 1 +# dRes = random.randint(0, max) +# # logger.debug("Threw the dice in range [{},{}], and got: {}".format(0,max,dRes)) +# return dRes + +# def pickTask(self): +# dice = self.throwDice() +# return self.tasks[dice] + +# def doWork(self, workerThread): +# task = self.pickTask() +# task.execute(workerThread) + +class LoggingFilter(logging.Filter): + def filter(self, record: logging.LogRecord): + if ( record.levelno >= logging.INFO ) : + return True # info or above always log + + msg = record.msg + # print("type = {}, value={}".format(type(msg), msg)) + # sys.exit() + + # Commenting out below to adjust... + + # if msg.startswith("[TRD]"): + # return False + return True - def throwDice(self): - max = len(self.tasks) - 1 - dRes = random.randint(0, max) - # logger.debug("Threw the dice in range [{},{}], and got: {}".format(0,max,dRes)) - return dRes - - def pickTask(self): - dice = self.throwDice() - return self.tasks[dice] - - def doWork(self, workerThread): - task = self.pickTask() - task.execute(workerThread) + def main(): # Super cool Python argument library: https://docs.python.org/3/library/argparse.html @@ -810,9 +1292,12 @@ def main(): sys.exit() global logger - logger = logging.getLogger('myApp') + logger = logging.getLogger('CrashGen') + logger.addFilter(LoggingFilter()) if ( gConfig.debug ): logger.setLevel(logging.DEBUG) # default seems to be INFO + else: + logger.setLevel(logging.INFO) ch = logging.StreamHandler() logger.addHandler(ch) @@ -820,12 +1305,21 @@ def main(): Dice.seed(0) # initial seeding of dice tc = ThreadCoordinator( ThreadPool(dbState, gConfig.num_threads, gConfig.max_steps, 0), - WorkDispatcher(dbState), + # WorkDispatcher(dbState), # Obsolete? dbState ) + + # Sandbox testing code + # dbc = dbState.getDbConn() + # while True: + # rows = dbc.query("show databases") + # print("Rows: {}, time={}".format(rows, time.time())) + tc.run() - dbState.cleanUp() - logger.info("Finished running thread pool") + tc.logStats() + dbState.cleanUp() + + # logger.info("Crash_Gen execution finished") if __name__ == "__main__": main() diff --git a/tests/pytest/import_merge/importToCommit.py b/tests/pytest/import_merge/importToCommit.py index dd2c27918ab7f52d799715b2c14ac02c05fc7aca..9a17ae95faf039c4ceee29f949f6d0e47a2b2d28 100644 --- a/tests/pytest/import_merge/importToCommit.py +++ b/tests/pytest/import_merge/importToCommit.py @@ -46,7 +46,6 @@ class TDTestCase: tdLog.info('insert data until the first commit') dnodesDir = tdDnodes.getDnodesRootDir() dataDir = dnodesDir + '/dnode1/data/vnode' - tdLog.info('CBD: dataDir=%s' % dataDir) startTime = self.startTime rid0 = 1 while (True): diff --git a/tests/pytest/query/queryNormal.py b/tests/pytest/query/queryNormal.py index c4fa46c063f602548e358864a97e12f61d0851dc..442661f72a4510e9d311ee38bbc16a204bf71670 100644 --- a/tests/pytest/query/queryNormal.py +++ b/tests/pytest/query/queryNormal.py @@ -91,7 +91,7 @@ class TDTestCase: tdSql.query("select * from stb1 limit 2 offset 3") tdSql.checkRows(1) - # query ... alien for table ---- bug + # query ... alias for table ---- bug tdSql.query("select t.ts from tb1 t") tdSql.checkRows(2) diff --git a/tests/pytest/random-test/random-test-multi-threading.py b/tests/pytest/random-test/random-test-multi-threading.py new file mode 100644 index 0000000000000000000000000000000000000000..c6ccc44372eeb625f5fa01c962ebd97d92ee6a90 --- /dev/null +++ b/tests/pytest/random-test/random-test-multi-threading.py @@ -0,0 +1,215 @@ +################################################################### +# Copyright (c) 2016 by TAOS Technologies, Inc. +# All rights reserved. +# +# This file is proprietary and confidential to TAOS Technologies. +# No part of this file may be reproduced, stored, transmitted, +# disclosed or used in any form or by any means other than as +# expressly provided by the written permission from Jianhui Tao +# +################################################################### + +# -*- coding: utf-8 -*- + +import sys +import random +import threading + +from util.log import * +from util.cases import * +from util.sql import * +from util.dnodes import * + +current_tb = "" +last_tb = "" +written = 0 + + +class Test (threading.Thread): + def __init__(self, threadId, name, sleepTime): + threading.Thread.__init__(self) + self.threadId = threadId + self.name = name + self.sleepTime = sleepTime + + self.threadLock = threading.Lock() + + def create_table(self): + global current_tb + global last_tb + global written + + tdLog.info("create a table") + current_tb = "tb%d" % int(round(time.time() * 1000)) + tdLog.info("current table %s" % current_tb) + + if (current_tb == last_tb): + return + else: + tdSql.execute( + 'create table %s (ts timestamp, speed int)' % + current_tb) + last_tb = current_tb + written = 0 + + def insert_data(self): + global current_tb + global last_tb + global written + + tdLog.info("will insert data to table") + if (current_tb == ""): + tdLog.info("no table, create first") + self.create_table() + + tdLog.info("insert data to table") + for i in range(0, 10): + self.threadLock.acquire() + insertRows = 1000 + tdLog.info("insert %d rows to %s" % (insertRows, current_tb)) + + for j in range(0, insertRows): + ret = tdSql.execute( + 'insert into %s values (now + %dm, %d)' % + (current_tb, j, j)) + written = written + 1 + self.threadLock.release() + + def query_data(self): + global current_tb + global last_tb + global written + + if (written > 0): + tdLog.info("query data from table") + tdSql.query("select * from %s" % last_tb) + tdSql.checkRows(written) + + def create_stable(self): + global current_tb + global last_tb + global written + + tdLog.info("create a super table") + + def restart_database(self): + global current_tb + global last_tb + global written + + tdLog.info("restart databae") + tdDnodes.stop(1) + tdDnodes.start(1) + tdLog.sleep(5) + + def force_restart(self): + global current_tb + global last_tb + global written + + tdLog.info("force restart database") + tdDnodes.forcestop(1) + tdDnodes.start(1) + tdLog.sleep(5) + + def drop_table(self): + global current_tb + global last_tb + global written + + for i in range(0, 10): + self.threadLock.acquire() + + tdLog.info("current_tb %s" % current_tb) + + if (current_tb != ""): + tdLog.info("drop current tb %s" % current_tb) + tdSql.execute("drop table %s" % current_tb) + current_tb = "" + last_tb = "" + written = 0 + tdLog.sleep(self.sleepTime) + self.threadLock.release() + + def reset_query_cache(self): + global current_tb + global last_tb + global written + + tdLog.info("reset query cache") + tdSql.execute("reset query cache") + tdLog.sleep(1) + + def reset_database(self): + global current_tb + global last_tb + global written + + tdLog.info("reset database") + tdDnodes.forcestop(1) + tdDnodes.deploy(1) + current_tb = "" + last_tb = "" + written = 0 + tdDnodes.start(1) + tdSql.prepare() + + def delete_datafiles(self): + global current_tb + global last_tb + global written + + tdLog.info("delete data files") + dnodesDir = tdDnodes.getDnodesRootDir() + dataDir = dnodesDir + '/dnode1/*' + deleteCmd = 'rm -rf %s' % dataDir + os.system(deleteCmd) + + current_tb = "" + last_tb = "" + written = 0 + tdDnodes.start(1) + tdSql.prepare() + + def run(self): + switch = { + 1: self.create_table, + 2: self.insert_data, + 3: self.query_data, + 4: self.create_stable, + 5: self.restart_database, + 6: self.force_restart, + 7: self.drop_table, + 8: self.reset_query_cache, + 9: self.reset_database, + 10: self.delete_datafiles, + } + + switch.get(self.threadId, lambda: "ERROR")() + + +class TDTestCase: + def init(self, conn, logSql): + tdLog.debug("start to execute %s" % __file__) + tdSql.init(conn.cursor(), logSql) + + def run(self): + tdSql.prepare() + + test1 = Test(2, "insert_data", 1) + test2 = Test(7, "drop_table", 2) + + test1.start() + test2.start() + test1.join() + test2.join() + + tdLog.info("end of test") + + def stop(self): + tdSql.close() + tdLog.success("%s successfully executed" % __file__) + + +tdCases.addWindows(__file__, TDTestCase()) +tdCases.addLinux(__file__, TDTestCase()) diff --git a/tests/pytest/util/dnodes.py b/tests/pytest/util/dnodes.py index e9f0dad076d803a3533ee7656f014fb85b5d5550..13038f4ddaa33cec981fdb1ae13a7a8bd0ba6346 100644 --- a/tests/pytest/util/dnodes.py +++ b/tests/pytest/util/dnodes.py @@ -159,11 +159,11 @@ class TDDnode: self.cfg("dataDir", self.dataDir) self.cfg("logDir", self.logDir) self.cfg("numOfLogLines", "100000000") - self.cfg("mgmtEqualVnodeNum", "0") + self.cfg("mnodeEqualVnodeNum", "0") self.cfg("clog", "1") self.cfg("statusInterval", "1") self.cfg("numOfTotalVnodes", "64") - self.cfg("numOfMPeers", "3") + self.cfg("numOfMnodes", "3") self.cfg("numOfThreadsPerCore", "2.0") self.cfg("monitor", "0") self.cfg("maxVnodeConnections", "30000") diff --git a/tests/script/general/alter/count.sim b/tests/script/general/alter/count.sim index a42fde74b7586d43904bb915a36eae11f05cccde..4728433dd2af02de7597f33d5bb7b652d4ea5a8b 100644 --- a/tests/script/general/alter/count.sim +++ b/tests/script/general/alter/count.sim @@ -3,8 +3,8 @@ system sh/stop_dnodes.sh system sh/deploy.sh -n dnode1 -i 1 system sh/cfg.sh -n dnode1 -c wallevel -v 0 -system sh/cfg.sh -n dnode1 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode1 -c mgmtEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode1 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode1 -c mnodeEqualVnodeNum -v 4 print ========= start dnode1 as master system sh/exec.sh -n dnode1 -s start diff --git a/tests/script/general/alter/import.sim b/tests/script/general/alter/import.sim index 61297655133772ef45d6a28d9f140f0d7b60395f..0ad52373fc53d8b085c0112dbd2b1e713519a806 100644 --- a/tests/script/general/alter/import.sim +++ b/tests/script/general/alter/import.sim @@ -3,8 +3,8 @@ system sh/stop_dnodes.sh system sh/deploy.sh -n dnode1 -i 1 system sh/cfg.sh -n dnode1 -c wallevel -v 0 -system sh/cfg.sh -n dnode1 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode1 -c mgmtEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode1 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode1 -c mnodeEqualVnodeNum -v 4 print ========= start dnode1 as master system sh/exec.sh -n dnode1 -s start diff --git a/tests/script/general/db/backup/keep.sim b/tests/script/general/db/backup/keep.sim index 4899acdd213bd665fca5e23caf6b7883e638805b..156b32ba4c85764b2a53d77119c61aaff71baefc 100644 --- a/tests/script/general/db/backup/keep.sim +++ b/tests/script/general/db/backup/keep.sim @@ -9,12 +9,12 @@ system sh/deploy.sh -n dnode3 -i 3 system sh/cfg.sh -n dnode1 -c walLevel -v 0 system sh/cfg.sh -n dnode2 -c walLevel -v 0 system sh/cfg.sh -n dnode3 -c walLevel -v 0 -system sh/cfg.sh -n dnode1 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode2 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode3 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode1 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode2 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode3 -c mgmtEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode1 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode2 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode3 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode1 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode2 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode3 -c mnodeEqualVnodeNum -v 4 print ========= start dnode1 as master system sh/exec.sh -n dnode1 -s start diff --git a/tests/script/general/db/delete_reuse1.sim b/tests/script/general/db/delete_reuse1.sim index 1e4baeb57687fa13bc51401cd6f884498a9b9b77..7b8687d340a4079c74f3203e1e2c22e37a80a0f4 100644 --- a/tests/script/general/db/delete_reuse1.sim +++ b/tests/script/general/db/delete_reuse1.sim @@ -10,15 +10,15 @@ system sh/cfg.sh -n dnode2 -c walLevel -v 0 system sh/cfg.sh -n dnode3 -c walLevel -v 0 system sh/cfg.sh -n dnode4 -c walLevel -v 0 -system sh/cfg.sh -n dnode1 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode2 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode3 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode4 -c numOfMPeers -v 1 - -system sh/cfg.sh -n dnode1 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode2 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode3 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode4 -c mgmtEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode1 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode2 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode3 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode4 -c numOfMnodes -v 1 + +system sh/cfg.sh -n dnode1 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode2 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode3 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode4 -c mnodeEqualVnodeNum -v 4 system sh/cfg.sh -n dnode1 -c numOfTotalVnodes -v 4 system sh/cfg.sh -n dnode2 -c numOfTotalVnodes -v 4 diff --git a/tests/script/general/db/delete_reuse2.sim b/tests/script/general/db/delete_reuse2.sim index d2fcaad838346ffd0ca6d49113f078f20841066c..d27a54ad95c1b373150fd429bef412035fa505a9 100644 --- a/tests/script/general/db/delete_reuse2.sim +++ b/tests/script/general/db/delete_reuse2.sim @@ -10,15 +10,15 @@ system sh/cfg.sh -n dnode2 -c walLevel -v 0 system sh/cfg.sh -n dnode3 -c walLevel -v 0 system sh/cfg.sh -n dnode4 -c walLevel -v 0 -system sh/cfg.sh -n dnode1 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode2 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode3 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode4 -c numOfMPeers -v 1 - -system sh/cfg.sh -n dnode1 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode2 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode3 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode4 -c mgmtEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode1 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode2 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode3 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode4 -c numOfMnodes -v 1 + +system sh/cfg.sh -n dnode1 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode2 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode3 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode4 -c mnodeEqualVnodeNum -v 4 system sh/cfg.sh -n dnode1 -c numOfTotalVnodes -v 4 system sh/cfg.sh -n dnode2 -c numOfTotalVnodes -v 4 diff --git a/tests/script/general/db/delete_writing1.sim b/tests/script/general/db/delete_writing1.sim index 58fe68dd5f991098e0a6d33101f84effec0aa6c5..96783be030ecd68f1d984ec608af7d93ff34841d 100644 --- a/tests/script/general/db/delete_writing1.sim +++ b/tests/script/general/db/delete_writing1.sim @@ -10,15 +10,15 @@ system sh/cfg.sh -n dnode2 -c walLevel -v 0 system sh/cfg.sh -n dnode3 -c walLevel -v 0 system sh/cfg.sh -n dnode4 -c walLevel -v 0 -system sh/cfg.sh -n dnode1 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode2 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode3 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode4 -c numOfMPeers -v 1 - -system sh/cfg.sh -n dnode1 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode2 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode3 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode4 -c mgmtEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode1 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode2 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode3 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode4 -c numOfMnodes -v 1 + +system sh/cfg.sh -n dnode1 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode2 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode3 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode4 -c mnodeEqualVnodeNum -v 4 system sh/cfg.sh -n dnode1 -c numOfTotalVnodes -v 4 system sh/cfg.sh -n dnode2 -c numOfTotalVnodes -v 4 diff --git a/tests/script/general/import/basic.sim b/tests/script/general/import/basic.sim index 2b72d55cdaf76ed07cd3519053010c1186ccd28b..07febb2bd52155af1cec70f84a1fe60460815750 100644 --- a/tests/script/general/import/basic.sim +++ b/tests/script/general/import/basic.sim @@ -4,15 +4,15 @@ system sh/deploy.sh -n dnode2 -i 2 system sh/deploy.sh -n dnode3 -i 3 system sh/deploy.sh -n dnode4 -i 4 -system sh/cfg.sh -n dnode1 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode2 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode3 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode4 -c numOfMPeers -v 1 - -system sh/cfg.sh -n dnode1 -c mgmtEqualVnodeNum -v 10 -system sh/cfg.sh -n dnode2 -c mgmtEqualVnodeNum -v 10 -system sh/cfg.sh -n dnode3 -c mgmtEqualVnodeNum -v 10 -system sh/cfg.sh -n dnode4 -c mgmtEqualVnodeNum -v 10 +system sh/cfg.sh -n dnode1 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode2 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode3 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode4 -c numOfMnodes -v 1 + +system sh/cfg.sh -n dnode1 -c mnodeEqualVnodeNum -v 10 +system sh/cfg.sh -n dnode2 -c mnodeEqualVnodeNum -v 10 +system sh/cfg.sh -n dnode3 -c mnodeEqualVnodeNum -v 10 +system sh/cfg.sh -n dnode4 -c mnodeEqualVnodeNum -v 10 system sh/cfg.sh -n dnode1 -c maxtablesPerVnode -v 2000 system sh/cfg.sh -n dnode2 -c maxtablesPerVnode -v 2000 diff --git a/tests/script/general/import/commit.sim b/tests/script/general/import/commit.sim index 64833fcd61677fd3e6a2ac5ee4f2e46b662ce4a7..498bb4f2e6d6085ca62dd17060ff887ae8f75422 100644 --- a/tests/script/general/import/commit.sim +++ b/tests/script/general/import/commit.sim @@ -4,15 +4,15 @@ system sh/deploy.sh -n dnode2 -i 2 system sh/deploy.sh -n dnode3 -i 3 system sh/deploy.sh -n dnode4 -i 4 -system sh/cfg.sh -n dnode1 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode2 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode3 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode4 -c numOfMPeers -v 1 +system sh/cfg.sh -n dnode1 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode2 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode3 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode4 -c numOfMnodes -v 1 -system sh/cfg.sh -n dnode1 -c mgmtEqualVnodeNum -v 10 -system sh/cfg.sh -n dnode2 -c mgmtEqualVnodeNum -v 10 -system sh/cfg.sh -n dnode3 -c mgmtEqualVnodeNum -v 10 -system sh/cfg.sh -n dnode4 -c mgmtEqualVnodeNum -v 10 +system sh/cfg.sh -n dnode1 -c mnodeEqualVnodeNum -v 10 +system sh/cfg.sh -n dnode2 -c mnodeEqualVnodeNum -v 10 +system sh/cfg.sh -n dnode3 -c mnodeEqualVnodeNum -v 10 +system sh/cfg.sh -n dnode4 -c mnodeEqualVnodeNum -v 10 system sh/cfg.sh -n dnode1 -c maxtablesPerVnode -v 2000 system sh/cfg.sh -n dnode2 -c maxtablesPerVnode -v 2000 diff --git a/tests/script/general/import/large.sim b/tests/script/general/import/large.sim index 6f6889a226d8abdc9b2f1ffa026b57a0a55ae314..5bf05a57fbdb3bf3c8bb123215686ccd5bdd4a8e 100644 --- a/tests/script/general/import/large.sim +++ b/tests/script/general/import/large.sim @@ -4,15 +4,15 @@ system sh/deploy.sh -n dnode2 -i 2 system sh/deploy.sh -n dnode3 -i 3 system sh/deploy.sh -n dnode4 -i 4 -system sh/cfg.sh -n dnode1 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode2 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode3 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode4 -c numOfMPeers -v 1 +system sh/cfg.sh -n dnode1 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode2 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode3 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode4 -c numOfMnodes -v 1 -system sh/cfg.sh -n dnode1 -c mgmtEqualVnodeNum -v 10 -system sh/cfg.sh -n dnode2 -c mgmtEqualVnodeNum -v 10 -system sh/cfg.sh -n dnode3 -c mgmtEqualVnodeNum -v 10 -system sh/cfg.sh -n dnode4 -c mgmtEqualVnodeNum -v 10 +system sh/cfg.sh -n dnode1 -c mnodeEqualVnodeNum -v 10 +system sh/cfg.sh -n dnode2 -c mnodeEqualVnodeNum -v 10 +system sh/cfg.sh -n dnode3 -c mnodeEqualVnodeNum -v 10 +system sh/cfg.sh -n dnode4 -c mnodeEqualVnodeNum -v 10 system sh/cfg.sh -n dnode1 -c maxtablesPerVnode -v 2000 system sh/cfg.sh -n dnode2 -c maxtablesPerVnode -v 2000 diff --git a/tests/script/general/import/replica1.sim b/tests/script/general/import/replica1.sim index 61f563ba8e9489e2ad0ffe217b37603ffb975319..c212e8723e7430102217403c9a8901fb480010a4 100644 --- a/tests/script/general/import/replica1.sim +++ b/tests/script/general/import/replica1.sim @@ -5,15 +5,15 @@ system sh/deploy.sh -n dnode2 -i 2 system sh/deploy.sh -n dnode3 -i 3 system sh/deploy.sh -n dnode4 -i 4 -system sh/cfg.sh -n dnode1 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode2 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode3 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode4 -c numOfMPeers -v 1 - -system sh/cfg.sh -n dnode1 -c mgmtEqualVnodeNum -v 10 -system sh/cfg.sh -n dnode2 -c mgmtEqualVnodeNum -v 10 -system sh/cfg.sh -n dnode3 -c mgmtEqualVnodeNum -v 10 -system sh/cfg.sh -n dnode4 -c mgmtEqualVnodeNum -v 10 +system sh/cfg.sh -n dnode1 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode2 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode3 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode4 -c numOfMnodes -v 1 + +system sh/cfg.sh -n dnode1 -c mnodeEqualVnodeNum -v 10 +system sh/cfg.sh -n dnode2 -c mnodeEqualVnodeNum -v 10 +system sh/cfg.sh -n dnode3 -c mnodeEqualVnodeNum -v 10 +system sh/cfg.sh -n dnode4 -c mnodeEqualVnodeNum -v 10 system sh/cfg.sh -n dnode1 -c maxtablesPerVnode -v 2000 system sh/cfg.sh -n dnode2 -c maxtablesPerVnode -v 2000 diff --git a/tests/script/general/table/delete_reuse1.sim b/tests/script/general/table/delete_reuse1.sim index 94f5bdb2876e60b773003bf5584f2a235fd1b500..753d1e8a35429af0646dd6e8c818bbff473c0ab6 100644 --- a/tests/script/general/table/delete_reuse1.sim +++ b/tests/script/general/table/delete_reuse1.sim @@ -9,15 +9,15 @@ system sh/cfg.sh -n dnode2 -c walLevel -v 0 system sh/cfg.sh -n dnode3 -c walLevel -v 0 system sh/cfg.sh -n dnode4 -c walLevel -v 0 -system sh/cfg.sh -n dnode1 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode2 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode3 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode4 -c numOfMPeers -v 1 +system sh/cfg.sh -n dnode1 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode2 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode3 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode4 -c numOfMnodes -v 1 -system sh/cfg.sh -n dnode1 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode2 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode3 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode4 -c mgmtEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode1 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode2 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode3 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode4 -c mnodeEqualVnodeNum -v 4 system sh/cfg.sh -n dnode1 -c numOfTotalVnodes -v 4 system sh/cfg.sh -n dnode2 -c numOfTotalVnodes -v 4 diff --git a/tests/script/general/table/delete_reuse2.sim b/tests/script/general/table/delete_reuse2.sim index 1a9c80cff075bb06db5721bedca64ae4ba5bedc0..1c74399348d5a9e407fd2264d4a5f8a16124545c 100644 --- a/tests/script/general/table/delete_reuse2.sim +++ b/tests/script/general/table/delete_reuse2.sim @@ -9,15 +9,15 @@ system sh/cfg.sh -n dnode2 -c walLevel -v 0 system sh/cfg.sh -n dnode3 -c walLevel -v 0 system sh/cfg.sh -n dnode4 -c walLevel -v 0 -system sh/cfg.sh -n dnode1 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode2 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode3 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode4 -c numOfMPeers -v 1 +system sh/cfg.sh -n dnode1 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode2 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode3 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode4 -c numOfMnodes -v 1 -system sh/cfg.sh -n dnode1 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode2 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode3 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode4 -c mgmtEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode1 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode2 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode3 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode4 -c mnodeEqualVnodeNum -v 4 system sh/cfg.sh -n dnode1 -c numOfTotalVnodes -v 4 system sh/cfg.sh -n dnode2 -c numOfTotalVnodes -v 4 diff --git a/tests/script/general/table/delete_writing.sim b/tests/script/general/table/delete_writing.sim index 73d028928fca665e643ce052c9e7592508950ee0..853637967d086cf780ae7bec162a19833dc4074c 100644 --- a/tests/script/general/table/delete_writing.sim +++ b/tests/script/general/table/delete_writing.sim @@ -9,15 +9,15 @@ system sh/cfg.sh -n dnode2 -c walLevel -v 0 system sh/cfg.sh -n dnode3 -c walLevel -v 0 system sh/cfg.sh -n dnode4 -c walLevel -v 0 -system sh/cfg.sh -n dnode1 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode2 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode3 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode4 -c numOfMPeers -v 1 - -system sh/cfg.sh -n dnode1 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode2 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode3 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode4 -c mgmtEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode1 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode2 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode3 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode4 -c numOfMnodes -v 1 + +system sh/cfg.sh -n dnode1 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode2 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode3 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode4 -c mnodeEqualVnodeNum -v 4 system sh/cfg.sh -n dnode1 -c numOfTotalVnodes -v 4 system sh/cfg.sh -n dnode2 -c numOfTotalVnodes -v 4 diff --git a/tests/script/jenkins/basic.txt b/tests/script/jenkins/basic.txt index dc8c564fd95db99e5528e227f5098ea8317fc153..eb392bb00bef86b5228a725b715081efbf85e81e 100644 --- a/tests/script/jenkins/basic.txt +++ b/tests/script/jenkins/basic.txt @@ -269,6 +269,7 @@ cd ../../../debug; make ./test.sh -u -f unique/db/replica_reduce31.sim ./test.sh -u -f unique/db/replica_part.sim +./test.sh -u -f unique/dnode/alternativeRole.sim ./test.sh -u -f unique/dnode/balance1.sim ./test.sh -u -f unique/dnode/balance2.sim ./test.sh -u -f unique/dnode/balance3.sim diff --git a/tests/script/sh/clear.sh b/tests/script/sh/clear.sh index 714ff4358036c4aa432d9f0c24ba0a6ee17781fe..0d58b5aec475dce8d224b24f1dfbc3b9f45b5f58 100755 --- a/tests/script/sh/clear.sh +++ b/tests/script/sh/clear.sh @@ -106,12 +106,12 @@ echo "monitor 0" >> $TAOS_CFG echo "numOfThreadsPerCore 2.0" >> $TAOS_CFG echo "defaultPass taosdata" >> $TAOS_CFG echo "numOfLogLines 100000000" >> $TAOS_CFG -echo "mgmtEqualVnodeNum 0" >> $TAOS_CFG +echo "mnodeEqualVnodeNum 0" >> $TAOS_CFG echo "clog 0" >> $TAOS_CFG echo "statusInterval 1" >> $TAOS_CFG echo "numOfTotalVnodes 4" >> $TAOS_CFG echo "asyncLog 0" >> $TAOS_CFG -echo "numOfMPeers 1" >> $TAOS_CFG +echo "numOfMnodes 1" >> $TAOS_CFG echo "locale en_US.UTF-8" >> $TAOS_CFG diff --git a/tests/script/sh/deploy.sh b/tests/script/sh/deploy.sh index b3134938cbc063ed7c857f1839ff401a2b2bf904..47f9f7109d87f287fa1aae5c06af4fcc4af56890 100755 --- a/tests/script/sh/deploy.sh +++ b/tests/script/sh/deploy.sh @@ -96,7 +96,6 @@ echo "second ${HOSTNAME}:7200" >> $TAOS_CFG echo "serverPort ${NODE}" >> $TAOS_CFG echo "dataDir $DATA_DIR" >> $TAOS_CFG echo "logDir $LOG_DIR" >> $TAOS_CFG -echo "debugFlag 135" >> $TAOS_CFG echo "dDebugFlag 135" >> $TAOS_CFG echo "mDebugFlag 135" >> $TAOS_CFG echo "sdbDebugFlag 135" >> $TAOS_CFG @@ -105,8 +104,8 @@ echo "tmrDebugFlag 131" >> $TAOS_CFG echo "cDebugFlag 135" >> $TAOS_CFG echo "httpDebugFlag 135" >> $TAOS_CFG echo "monitorDebugFlag 131" >> $TAOS_CFG -echo "udebugFlag 131" >> $TAOS_CFG -echo "jnidebugFlag 131" >> $TAOS_CFG +echo "udebugFlag 135" >> $TAOS_CFG +echo "jnidebugFlag 135" >> $TAOS_CFG echo "sdebugFlag 135" >> $TAOS_CFG echo "qdebugFlag 135" >> $TAOS_CFG echo "monitor 0" >> $TAOS_CFG @@ -115,12 +114,12 @@ echo "http 0" >> $TAOS_CFG echo "numOfThreadsPerCore 2.0" >> $TAOS_CFG echo "defaultPass taosdata" >> $TAOS_CFG echo "numOfLogLines 100000000" >> $TAOS_CFG -echo "mgmtEqualVnodeNum 0" >> $TAOS_CFG +echo "mnodeEqualVnodeNum 0" >> $TAOS_CFG echo "clog 2" >> $TAOS_CFG echo "statusInterval 1" >> $TAOS_CFG echo "numOfTotalVnodes 4" >> $TAOS_CFG echo "asyncLog 0" >> $TAOS_CFG -echo "numOfMPeers 1" >> $TAOS_CFG +echo "numOfMnodes 1" >> $TAOS_CFG echo "locale en_US.UTF-8" >> $TAOS_CFG echo "anyIp 0" >> $TAOS_CFG diff --git a/tests/script/tmp/mnodes.sim b/tests/script/tmp/mnodes.sim index 38d99bd214c1bc39e66f70f2b0f3926c100f702d..67f3648f6429700468b00b0221db32ab9e851c42 100644 --- a/tests/script/tmp/mnodes.sim +++ b/tests/script/tmp/mnodes.sim @@ -2,6 +2,6 @@ system sh/stop_dnodes.sh system sh/deploy.sh -n dnode1 -i 1 system sh/deploy.sh -n dnode2 -i 2 system sh/deploy.sh -n dnode3 -i 3 -system sh/cfg.sh -n dnode1 -c numOfMPeers -v 3 -system sh/cfg.sh -n dnode2 -c numOfMPeers -v 3 -system sh/cfg.sh -n dnode3 -c numOfMPeers -v 3 +system sh/cfg.sh -n dnode1 -c numOfMnodes -v 3 +system sh/cfg.sh -n dnode2 -c numOfMnodes -v 3 +system sh/cfg.sh -n dnode3 -c numOfMnodes -v 3 diff --git a/tests/script/tmp/prepare.sim b/tests/script/tmp/prepare.sim index 1db643c5c9c198e103aa0429b06cf6b5848ea960..a71401d6e4709feec4bf60c3b1bb6eceba349179 100644 --- a/tests/script/tmp/prepare.sim +++ b/tests/script/tmp/prepare.sim @@ -10,15 +10,15 @@ system sh/cfg.sh -n dnode2 -c walLevel -v 2 system sh/cfg.sh -n dnode3 -c walLevel -v 2 system sh/cfg.sh -n dnode4 -c walLevel -v 2 -system sh/cfg.sh -n dnode1 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode2 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode3 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode4 -c numOfMPeers -v 1 +system sh/cfg.sh -n dnode1 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode2 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode3 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode4 -c numOfMnodes -v 1 -system sh/cfg.sh -n dnode1 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode2 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode3 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode4 -c mgmtEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode1 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode2 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode3 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode4 -c mnodeEqualVnodeNum -v 4 system sh/cfg.sh -n dnode1 -c numOfTotalVnodes -v 4 system sh/cfg.sh -n dnode2 -c numOfTotalVnodes -v 4 diff --git a/tests/script/unique/arbitrator/dn3_mn1_replica2_wal1_AddDelDnode.sim b/tests/script/unique/arbitrator/dn3_mn1_replica2_wal1_AddDelDnode.sim new file mode 100644 index 0000000000000000000000000000000000000000..b3b1e0fe2262fc0c50c74202c71da7333adcddca --- /dev/null +++ b/tests/script/unique/arbitrator/dn3_mn1_replica2_wal1_AddDelDnode.sim @@ -0,0 +1,266 @@ +system sh/stop_dnodes.sh +system sh/deploy.sh -n dnode1 -i 1 +system sh/deploy.sh -n dnode2 -i 2 +system sh/deploy.sh -n dnode3 -i 3 +system sh/deploy.sh -n dnode4 -i 4 +system sh/deploy.sh -n dnode5 -i 5 + +system sh/cfg.sh -n dnode1 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode2 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode3 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode4 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode5 -c numOfMnodes -v 1 + +system sh/cfg.sh -n dnode1 -c walLevel -v 1 +system sh/cfg.sh -n dnode2 -c walLevel -v 1 +system sh/cfg.sh -n dnode3 -c walLevel -v 1 +system sh/cfg.sh -n dnode4 -c walLevel -v 1 +system sh/cfg.sh -n dnode5 -c walLevel -v 1 + +system sh/cfg.sh -n dnode1 -c balanceInterval -v 10 +system sh/cfg.sh -n dnode2 -c balanceInterval -v 10 +system sh/cfg.sh -n dnode3 -c balanceInterval -v 10 +system sh/cfg.sh -n dnode4 -c balanceInterval -v 10 +system sh/cfg.sh -n dnode5 -c balanceInterval -v 10 + +system sh/cfg.sh -n dnode1 -c numOfTotalVnodes -v 200 +system sh/cfg.sh -n dnode2 -c numOfTotalVnodes -v 4 +system sh/cfg.sh -n dnode3 -c numOfTotalVnodes -v 4 +system sh/cfg.sh -n dnode4 -c numOfTotalVnodes -v 4 +system sh/cfg.sh -n dnode5 -c numOfTotalVnodes -v 4 + +system sh/cfg.sh -n dnode1 -c mnodeEqualVnodeNum -v 200 +#system sh/cfg.sh -n dnode2 -c mnodeEqualVnodeNum -v 200 +#system sh/cfg.sh -n dnode3 -c mnodeEqualVnodeNum -v 200 +#system sh/cfg.sh -n dnode4 -c mnodeEqualVnodeNum -v 200 + +system sh/cfg.sh -n dnode1 -c arbitrator -v $arbitrator +system sh/cfg.sh -n dnode2 -c arbitrator -v $arbitrator +system sh/cfg.sh -n dnode3 -c arbitrator -v $arbitrator +system sh/cfg.sh -n dnode4 -c arbitrator -v $arbitrator +system sh/cfg.sh -n dnode5 -c arbitrator -v $arbitrator + +system sh/cfg.sh -n dnode1 -c offlineThreshold -v 20 +system sh/cfg.sh -n dnode2 -c offlineThreshold -v 20 +system sh/cfg.sh -n dnode3 -c offlineThreshold -v 20 +system sh/cfg.sh -n dnode4 -c offlineThreshold -v 20 +system sh/cfg.sh -n dnode5 -c offlineThreshold -v 20 + +system sh/cfg.sh -n dnode1 -c enableCoreFile -v 1 +system sh/cfg.sh -n dnode2 -c enableCoreFile -v 1 +system sh/cfg.sh -n dnode3 -c enableCoreFile -v 1 +system sh/cfg.sh -n dnode4 -c enableCoreFile -v 1 +system sh/cfg.sh -n dnode5 -c enableCoreFile -v 1 + +print ============== step0: start tarbitrator +system sh/exec_tarbitrator.sh -s start + +print ============== step1: start dnode1, only deploy mnode +system sh/exec_up.sh -n dnode1 -s start +sleep 3000 +sql connect + +print ============== step2: start dnode2/dnode3 and add into cluster, then create database, create table , and insert data +system sh/exec_up.sh -n dnode2 -s start +system sh/exec_up.sh -n dnode3 -s start +sleep 1000 +sql create dnode $hostname2 +sql create dnode $hostname3 + +$rowNum = 100 +$tblNum = 16 +$totalRows = 0 +$tsStart = 1420041600000 + +$db = db +sql create database $db replica 2 maxTables 4 +sql use $db + +# create table , insert data +$stb = stb +sql create table $stb (ts timestamp, c1 int) tags(t1 int) + + +$i = 0 +while $i < $tblNum + $tb = tb . $i + sql create table $tb using $stb tags( $i ) + + $x = 0 + while $x < $rowNum + $ts = $tsStart + $x + sql insert into $tb values ( $ts + 0a , $x ) ( $ts + 1a , $x ) ( $ts + 2a , $x ) ( $ts + 3a , $x ) ( $ts + 4a , $x ) ( $ts + 5a , $x ) ( $ts + 6a , $x ) ( $ts + 7a , $x ) ( $ts + 8a , $x ) ( $ts + 9a , $x ) ( $ts + 10a , $x ) ( $ts + 11a , $x ) ( $ts + 12a , $x ) ( $ts + 13a , $x ) ( $ts + 14a , $x ) ( $ts + 15a , $x ) ( $ts + 16a , $x ) ( $ts + 17a , $x ) ( $ts + 18a , $x ) ( $ts + 19a , $x ) ( $ts + 20a , $x ) ( $ts + 21a , $x ) ( $ts + 22a , $x ) ( $ts + 23a , $x ) ( $ts + 24a , $x ) ( $ts + 25a , $x ) ( $ts + 26a , $x ) ( $ts + 27a , $x ) ( $ts + 28a , $x ) ( $ts + 29a , $x ) ( $ts + 30a , $x ) ( $ts + 31a , $x ) ( $ts + 32a , $x ) ( $ts + 33a , $x ) ( $ts + 34a , $x ) ( $ts + 25a , $x ) ( $ts + 26a , $x ) ( $ts + 27a , $x ) ( $ts + 28a , $x ) ( $ts + 29a , $x ) ( $ts + 30a , $x ) ( $ts + 31a , $x ) ( $ts + 32a , $x ) ( $ts + 33a , $x ) ( $ts + 34a , $x ) ( $ts + 35a , $x ) ( $ts + 36a , $x ) ( $ts + 37a , $x ) ( $ts + 38a , $x ) ( $ts + 39a , $x ) ( $ts + 40a , $x ) ( $ts + 41a , $x ) ( $ts + 42a , $x ) ( $ts + 43a , $x ) ( $ts + 44a , $x ) ( $ts + 45a , $x ) ( $ts + 46a , $x ) ( $ts + 47a , $x ) ( $ts + 48a , $x ) ( $ts + 49a , $x ) ( $ts + 50a , $x ) ( $ts + 51a , $x ) ( $ts + 52a , $x ) ( $ts + 53a , $x ) ( $ts + 54a , $x ) ( $ts + 55a , $x ) ( $ts + 56a , $x ) ( $ts + 57a , $x ) ( $ts + 58a , $x ) ( $ts + 59a , $x ) + $x = $x + 60 + endw + $totalRows = $totalRows + $x + print info: inserted $x rows into $tb and totalRows: $totalRows + $i = $i + 1 +endw + +print info: select count(*) from $stb +sleep 1000 +sql reset query cache +sleep 1000 +sql select count(*) from $stb +print data00 $data00 +if $data00 != $totalRows then + return -1 +endi + +print ============== step3: add one new dnode4 expect auto balancing +system sh/exec_up.sh -n dnode4 -s start +sql create dnode $hostname4 +sleep 10000 +sql select count(*) from $stb +print data00 $data00 +if $data00 != $totalRows then + return -1 +endi + + +print ============== step4: stop dnode3, after offlineThreshold, dnode3 will be dropped for cluster +system sh/exec_up.sh -n dnode3 -s stop +sql select count(*) from $stb +print data00 $data00 +if $data00 != $totalRows then + return -1 +endi + +#sleep 3000 +#sql show dnodes +#print $data0_1 $data1_1 $data2_1 $data3_1 $data4_1 +#print $data0_2 $data1_2 $data2_2 $data3_2 $data4_2 +#print $data0_3 $data1_3 $data2_3 $data3_3 $data4_3 +#print $data0_4 $data1_4 $data2_4 $data3_4 $data4_4 +# +#$dnode1Status = $data4_1 +#$dnode2Status = $data4_2 +#$dnode3Status = $data4_3 +#$dnode4Status = $data4_4 +# +#if $dnode1Status != ready then +# return -1 +#endi +#if $dnode2Status != ready then +# return -1 +#endi +#if $dnode3Status != offline then +# return -1 +#endi +#if $dnode4Status != ready then +# return -1 +#endi + +sleep 30000 + +wait_drop: +sql show dnodes +if $rows != 3 then + sleep 3000 + goto wait_drop +endi +print $data0_1 $data1_1 $data2_1 $data3_1 $data4_1 +print $data0_2 $data1_2 $data2_2 $data3_2 $data4_2 +print $data0_3 $data1_3 $data2_3 $data3_3 $data4_3 +print $data0_4 $data1_4 $data2_4 $data3_4 $data4_4 +$dnode1Status = $data4_1 +$dnode2Status = $data4_2 +$dnode3Status = $data4_3 +$dnode4Status = $data4_4 + +if $dnode1Status != ready then + return -1 +endi +if $dnode2Status != ready then + return -1 +endi +if $dnode4Status != ready then + return -1 +endi + + +print ============== step5: start dnode5 and add into cluster , drop database +sql drop database $db +sleep 1000 +system sh/exec_up.sh -n dnode5 -s start +sql create dnode $hostname5 +sleep 3000 +wait_dnode5: +sql show dnodes +if $rows != 4 then + sleep 3000 + goto wait_dnode5 +endi +print $data0_1 $data1_1 $data2_1 $data3_1 $data4_1 +print $data0_2 $data1_2 $data2_2 $data3_2 $data4_2 +print $data0_3 $data1_3 $data2_3 $data3_3 $data4_3 +print $data0_4 $data1_4 $data2_4 $data3_4 $data4_4 +print $data0_5 $data1_5 $data2_5 $data3_5 $data4_5 +$dnode1Status = $data4_1 +$dnode2Status = $data4_2 +$dnode4Status = $data4_4 +$dnode5Status = $data4_5 + +if $dnode1Status != ready then + return -1 +endi +if $dnode2Status != ready then + return -1 +endi +if $dnode4Status != ready then + return -1 +endi +if $dnode5Status != ready then + return -1 +endi + + +print ============== step6: create database and table until not free vnodes +$rowNum = 100 +$tblNum = 24 +$totalRows = 0 +$tsStart = 1420041600000 + +$db = db1 +sql create database $db replica 2 maxTables 4 +sql use $db +$stb = stb +sql create table $stb (ts timestamp, c1 int) tags(t1 int) + + +# create table , insert data +$stb = stb +sql create table $stb (ts timestamp, c1 int) tags(t1 int) + + +$i = 0 +while $i < $tblNum + $tb = tb . $i + sql create table $tb using $stb tags( $i ) + + $x = 0 + while $x < $rowNum + $ts = $tsStart + $x + sql insert into $tb values ( $ts + 0a , $x ) ( $ts + 1a , $x ) ( $ts + 2a , $x ) ( $ts + 3a , $x ) ( $ts + 4a , $x ) ( $ts + 5a , $x ) ( $ts + 6a , $x ) ( $ts + 7a , $x ) ( $ts + 8a , $x ) ( $ts + 9a , $x ) ( $ts + 10a , $x ) ( $ts + 11a , $x ) ( $ts + 12a , $x ) ( $ts + 13a , $x ) ( $ts + 14a , $x ) ( $ts + 15a , $x ) ( $ts + 16a , $x ) ( $ts + 17a , $x ) ( $ts + 18a , $x ) ( $ts + 19a , $x ) ( $ts + 20a , $x ) ( $ts + 21a , $x ) ( $ts + 22a , $x ) ( $ts + 23a , $x ) ( $ts + 24a , $x ) ( $ts + 25a , $x ) ( $ts + 26a , $x ) ( $ts + 27a , $x ) ( $ts + 28a , $x ) ( $ts + 29a , $x ) ( $ts + 30a , $x ) ( $ts + 31a , $x ) ( $ts + 32a , $x ) ( $ts + 33a , $x ) ( $ts + 34a , $x ) ( $ts + 25a , $x ) ( $ts + 26a , $x ) ( $ts + 27a , $x ) ( $ts + 28a , $x ) ( $ts + 29a , $x ) ( $ts + 30a , $x ) ( $ts + 31a , $x ) ( $ts + 32a , $x ) ( $ts + 33a , $x ) ( $ts + 34a , $x ) ( $ts + 35a , $x ) ( $ts + 36a , $x ) ( $ts + 37a , $x ) ( $ts + 38a , $x ) ( $ts + 39a , $x ) ( $ts + 40a , $x ) ( $ts + 41a , $x ) ( $ts + 42a , $x ) ( $ts + 43a , $x ) ( $ts + 44a , $x ) ( $ts + 45a , $x ) ( $ts + 46a , $x ) ( $ts + 47a , $x ) ( $ts + 48a , $x ) ( $ts + 49a , $x ) ( $ts + 50a , $x ) ( $ts + 51a , $x ) ( $ts + 52a , $x ) ( $ts + 53a , $x ) ( $ts + 54a , $x ) ( $ts + 55a , $x ) ( $ts + 56a , $x ) ( $ts + 57a , $x ) ( $ts + 58a , $x ) ( $ts + 59a , $x ) + $x = $x + 60 + endw + $totalRows = $totalRows + $x + print info: inserted $x rows into $tb and totalRows: $totalRows + $i = $i + 1 +endw + +print info: select count(*) from $stb +sleep 2000 +sql reset query cache +sleep 3000 +sql select count(*) from $stb +print data00 $data00 +if $data00 != $totalRows then + return -1 +endi + +print ============== step7: drop dnode3, and system should prompt cannot drop dnodes +sql_error drop dnode $hostname3 +print ============== step8: add one new table, and system should prompt 'need more dnode' +sql_error create table tb_more using $stb tags( 1000 ) + diff --git a/tests/script/unique/arbitrator/replica_changeWithArbitrator.sim b/tests/script/unique/arbitrator/replica_changeWithArbitrator.sim index d1b4a2ec72e9253025c37c1a9c9e2cd868ceb93f..fd69d955c3a33073ef1391c499bd7ea66f716839 100644 --- a/tests/script/unique/arbitrator/replica_changeWithArbitrator.sim +++ b/tests/script/unique/arbitrator/replica_changeWithArbitrator.sim @@ -3,9 +3,9 @@ system sh/deploy.sh -n dnode1 -i 1 system sh/deploy.sh -n dnode2 -i 2 system sh/deploy.sh -n dnode3 -i 3 -system sh/cfg.sh -n dnode1 -c numOfMPeers -v 2 -system sh/cfg.sh -n dnode2 -c numOfMPeers -v 2 -system sh/cfg.sh -n dnode3 -c numOfMPeers -v 2 +system sh/cfg.sh -n dnode1 -c numOfMnodes -v 2 +system sh/cfg.sh -n dnode2 -c numOfMnodes -v 2 +system sh/cfg.sh -n dnode3 -c numOfMnodes -v 2 system sh/cfg.sh -n dnode1 -c walLevel -v 1 system sh/cfg.sh -n dnode2 -c walLevel -v 1 diff --git a/tests/script/unique/cluster/balance1.sim b/tests/script/unique/cluster/balance1.sim index 424a80d25aa397837fd4598ccd0a48c77e27f140..47b23d16c996a483de9fc23027d4e5b3b93c541a 100644 --- a/tests/script/unique/cluster/balance1.sim +++ b/tests/script/unique/cluster/balance1.sim @@ -17,23 +17,23 @@ system sh/cfg.sh -n dnode6 -c numOfTotalVnodes -v 4 system sh/cfg.sh -n dnode7 -c numOfTotalVnodes -v 4 system sh/cfg.sh -n dnode8 -c numOfTotalVnodes -v 4 -system sh/cfg.sh -n dnode1 -c numOfMpeers -v 3 -system sh/cfg.sh -n dnode2 -c numOfMpeers -v 3 -system sh/cfg.sh -n dnode3 -c numOfMpeers -v 3 -system sh/cfg.sh -n dnode4 -c numOfMpeers -v 3 -system sh/cfg.sh -n dnode5 -c numOfMpeers -v 3 -system sh/cfg.sh -n dnode6 -c numOfMpeers -v 3 -system sh/cfg.sh -n dnode7 -c numOfMpeers -v 3 -system sh/cfg.sh -n dnode8 -c numOfMpeers -v 3 - -system sh/cfg.sh -n dnode1 -c mgmtEqualVnodeNum -v 0 -system sh/cfg.sh -n dnode2 -c mgmtEqualVnodeNum -v 0 -system sh/cfg.sh -n dnode3 -c mgmtEqualVnodeNum -v 0 -system sh/cfg.sh -n dnode4 -c mgmtEqualVnodeNum -v 0 -system sh/cfg.sh -n dnode5 -c mgmtEqualVnodeNum -v 0 -system sh/cfg.sh -n dnode6 -c mgmtEqualVnodeNum -v 0 -system sh/cfg.sh -n dnode7 -c mgmtEqualVnodeNum -v 0 -system sh/cfg.sh -n dnode8 -c mgmtEqualVnodeNum -v 0 +system sh/cfg.sh -n dnode1 -c numOfMnodes -v 3 +system sh/cfg.sh -n dnode2 -c numOfMnodes -v 3 +system sh/cfg.sh -n dnode3 -c numOfMnodes -v 3 +system sh/cfg.sh -n dnode4 -c numOfMnodes -v 3 +system sh/cfg.sh -n dnode5 -c numOfMnodes -v 3 +system sh/cfg.sh -n dnode6 -c numOfMnodes -v 3 +system sh/cfg.sh -n dnode7 -c numOfMnodes -v 3 +system sh/cfg.sh -n dnode8 -c numOfMnodes -v 3 + +system sh/cfg.sh -n dnode1 -c mnodeEqualVnodeNum -v 0 +system sh/cfg.sh -n dnode2 -c mnodeEqualVnodeNum -v 0 +system sh/cfg.sh -n dnode3 -c mnodeEqualVnodeNum -v 0 +system sh/cfg.sh -n dnode4 -c mnodeEqualVnodeNum -v 0 +system sh/cfg.sh -n dnode5 -c mnodeEqualVnodeNum -v 0 +system sh/cfg.sh -n dnode6 -c mnodeEqualVnodeNum -v 0 +system sh/cfg.sh -n dnode7 -c mnodeEqualVnodeNum -v 0 +system sh/cfg.sh -n dnode8 -c mnodeEqualVnodeNum -v 0 system sh/cfg.sh -n dnode1 -c wallevel -v 1 system sh/cfg.sh -n dnode2 -c wallevel -v 1 diff --git a/tests/script/unique/cluster/balance2.sim b/tests/script/unique/cluster/balance2.sim index 08fdd233e012f89a29bcaca938fb34f73aeb224a..48670ee9b1feb58b849c2743ef6e153583c2658f 100644 --- a/tests/script/unique/cluster/balance2.sim +++ b/tests/script/unique/cluster/balance2.sim @@ -17,14 +17,14 @@ system sh/cfg.sh -n dnode6 -c numOfTotalVnodes -v 4 system sh/cfg.sh -n dnode7 -c numOfTotalVnodes -v 4 system sh/cfg.sh -n dnode8 -c numOfTotalVnodes -v 4 -system sh/cfg.sh -n dnode1 -c numOfMPeers -v 3 -system sh/cfg.sh -n dnode2 -c numOfMPeers -v 3 -system sh/cfg.sh -n dnode3 -c numOfMPeers -v 3 -system sh/cfg.sh -n dnode4 -c numOfMPeers -v 3 -system sh/cfg.sh -n dnode5 -c numOfMPeers -v 3 -system sh/cfg.sh -n dnode6 -c numOfMPeers -v 3 -system sh/cfg.sh -n dnode7 -c numOfMPeers -v 3 -system sh/cfg.sh -n dnode8 -c numOfMPeers -v 3 +system sh/cfg.sh -n dnode1 -c numOfMnodes -v 3 +system sh/cfg.sh -n dnode2 -c numOfMnodes -v 3 +system sh/cfg.sh -n dnode3 -c numOfMnodes -v 3 +system sh/cfg.sh -n dnode4 -c numOfMnodes -v 3 +system sh/cfg.sh -n dnode5 -c numOfMnodes -v 3 +system sh/cfg.sh -n dnode6 -c numOfMnodes -v 3 +system sh/cfg.sh -n dnode7 -c numOfMnodes -v 3 +system sh/cfg.sh -n dnode8 -c numOfMnodes -v 3 system sh/cfg.sh -n dnode1 -c wallevel -v 1 system sh/cfg.sh -n dnode2 -c wallevel -v 1 @@ -35,14 +35,14 @@ system sh/cfg.sh -n dnode6 -c wallevel -v 1 system sh/cfg.sh -n dnode7 -c wallevel -v 1 system sh/cfg.sh -n dnode8 -c wallevel -v 1 -system sh/cfg.sh -n dnode1 -c mgmtEqualVnodeNum -v 0 -system sh/cfg.sh -n dnode2 -c mgmtEqualVnodeNum -v 0 -system sh/cfg.sh -n dnode3 -c mgmtEqualVnodeNum -v 0 -system sh/cfg.sh -n dnode4 -c mgmtEqualVnodeNum -v 0 -system sh/cfg.sh -n dnode5 -c mgmtEqualVnodeNum -v 0 -system sh/cfg.sh -n dnode6 -c mgmtEqualVnodeNum -v 0 -system sh/cfg.sh -n dnode7 -c mgmtEqualVnodeNum -v 0 -system sh/cfg.sh -n dnode8 -c mgmtEqualVnodeNum -v 0 +system sh/cfg.sh -n dnode1 -c mnodeEqualVnodeNum -v 0 +system sh/cfg.sh -n dnode2 -c mnodeEqualVnodeNum -v 0 +system sh/cfg.sh -n dnode3 -c mnodeEqualVnodeNum -v 0 +system sh/cfg.sh -n dnode4 -c mnodeEqualVnodeNum -v 0 +system sh/cfg.sh -n dnode5 -c mnodeEqualVnodeNum -v 0 +system sh/cfg.sh -n dnode6 -c mnodeEqualVnodeNum -v 0 +system sh/cfg.sh -n dnode7 -c mnodeEqualVnodeNum -v 0 +system sh/cfg.sh -n dnode8 -c mnodeEqualVnodeNum -v 0 print ============== step1 print ========= start dnode1 diff --git a/tests/script/unique/cluster/balance3.sim b/tests/script/unique/cluster/balance3.sim index 407adc7f3b61301d6a5d08047160ae0b4d3fac27..4f53a04daf1a7b90a684183fe1e68f0d1a008343 100644 --- a/tests/script/unique/cluster/balance3.sim +++ b/tests/script/unique/cluster/balance3.sim @@ -18,23 +18,23 @@ system sh/cfg.sh -n dnode6 -c numOfTotalVnodes -v 4 system sh/cfg.sh -n dnode7 -c numOfTotalVnodes -v 4 system sh/cfg.sh -n dnode8 -c numOfTotalVnodes -v 4 -system sh/cfg.sh -n dnode1 -c numOfMpeers -v 3 -system sh/cfg.sh -n dnode2 -c numOfMpeers -v 3 -system sh/cfg.sh -n dnode3 -c numOfMpeers -v 3 -system sh/cfg.sh -n dnode4 -c numOfMpeers -v 3 -system sh/cfg.sh -n dnode5 -c numOfMpeers -v 3 -system sh/cfg.sh -n dnode6 -c numOfMpeers -v 3 -system sh/cfg.sh -n dnode7 -c numOfMpeers -v 3 -system sh/cfg.sh -n dnode8 -c numOfMpeers -v 3 - -system sh/cfg.sh -n dnode1 -c mgmtEqualVnodeNum -v 0 -system sh/cfg.sh -n dnode2 -c mgmtEqualVnodeNum -v 0 -system sh/cfg.sh -n dnode3 -c mgmtEqualVnodeNum -v 0 -system sh/cfg.sh -n dnode4 -c mgmtEqualVnodeNum -v 0 -system sh/cfg.sh -n dnode5 -c mgmtEqualVnodeNum -v 0 -system sh/cfg.sh -n dnode6 -c mgmtEqualVnodeNum -v 0 -system sh/cfg.sh -n dnode7 -c mgmtEqualVnodeNum -v 0 -system sh/cfg.sh -n dnode8 -c mgmtEqualVnodeNum -v 0 +system sh/cfg.sh -n dnode1 -c numOfMnodes -v 3 +system sh/cfg.sh -n dnode2 -c numOfMnodes -v 3 +system sh/cfg.sh -n dnode3 -c numOfMnodes -v 3 +system sh/cfg.sh -n dnode4 -c numOfMnodes -v 3 +system sh/cfg.sh -n dnode5 -c numOfMnodes -v 3 +system sh/cfg.sh -n dnode6 -c numOfMnodes -v 3 +system sh/cfg.sh -n dnode7 -c numOfMnodes -v 3 +system sh/cfg.sh -n dnode8 -c numOfMnodes -v 3 + +system sh/cfg.sh -n dnode1 -c mnodeEqualVnodeNum -v 0 +system sh/cfg.sh -n dnode2 -c mnodeEqualVnodeNum -v 0 +system sh/cfg.sh -n dnode3 -c mnodeEqualVnodeNum -v 0 +system sh/cfg.sh -n dnode4 -c mnodeEqualVnodeNum -v 0 +system sh/cfg.sh -n dnode5 -c mnodeEqualVnodeNum -v 0 +system sh/cfg.sh -n dnode6 -c mnodeEqualVnodeNum -v 0 +system sh/cfg.sh -n dnode7 -c mnodeEqualVnodeNum -v 0 +system sh/cfg.sh -n dnode8 -c mnodeEqualVnodeNum -v 0 system sh/cfg.sh -n dnode1 -c wallevel -v 1 system sh/cfg.sh -n dnode2 -c wallevel -v 1 diff --git a/tests/script/unique/column/replica3.sim b/tests/script/unique/column/replica3.sim index 4e45edc02d142f056d2f1593c78831d94fddf071..d294bf0a57719b08e741fbcea07297b58200bab3 100644 --- a/tests/script/unique/column/replica3.sim +++ b/tests/script/unique/column/replica3.sim @@ -6,9 +6,9 @@ system sh/deploy.sh -n dnode3 -i 3 system sh/cfg.sh -n dnode1 -c wallevel -v 2 system sh/cfg.sh -n dnode2 -c wallevel -v 2 system sh/cfg.sh -n dnode3 -c wallevel -v 2 -system sh/cfg.sh -n dnode1 -c numofMpeers -v 3 -system sh/cfg.sh -n dnode2 -c numofMpeers -v 3 -system sh/cfg.sh -n dnode3 -c numofMpeers -v 3 +system sh/cfg.sh -n dnode1 -c numOfMnodes -v 3 +system sh/cfg.sh -n dnode2 -c numOfMnodes -v 3 +system sh/cfg.sh -n dnode3 -c numOfMnodes -v 3 system sh/exec_up.sh -n dnode1 -s start sql connect diff --git a/tests/script/unique/db/commit.sim b/tests/script/unique/db/commit.sim index 5bf6ea6f10bf7e52581785459aa03de321f91c1c..4cef0c56380cb630f2c7ee8fbcc93627620d136e 100644 --- a/tests/script/unique/db/commit.sim +++ b/tests/script/unique/db/commit.sim @@ -6,12 +6,12 @@ system sh/deploy.sh -n dnode3 -i 3 system sh/cfg.sh -n dnode1 -c walLevel -v 2 system sh/cfg.sh -n dnode2 -c walLevel -v 2 system sh/cfg.sh -n dnode3 -c walLevel -v 2 -system sh/cfg.sh -n dnode1 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode2 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode3 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode1 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode2 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode3 -c mgmtEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode1 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode2 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode3 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode1 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode2 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode3 -c mnodeEqualVnodeNum -v 4 print ========= start dnode1 as master system sh/exec_up.sh -n dnode1 -s start diff --git a/tests/script/unique/db/delete.sim b/tests/script/unique/db/delete.sim index 5688333d20fc8867de16dbe5cc4705183d12d8df..1f9d27218321134d4d92979fc45a2414b13df287 100644 --- a/tests/script/unique/db/delete.sim +++ b/tests/script/unique/db/delete.sim @@ -6,9 +6,9 @@ system sh/deploy.sh -n dnode3 -i 3 system sh/cfg.sh -n dnode1 -c wallevel -v 2 system sh/cfg.sh -n dnode2 -c wallevel -v 2 system sh/cfg.sh -n dnode3 -c wallevel -v 2 -system sh/cfg.sh -n dnode1 -c numOfMPeers -v 3 -system sh/cfg.sh -n dnode2 -c numOfMPeers -v 3 -system sh/cfg.sh -n dnode3 -c numOfMPeers -v 3 +system sh/cfg.sh -n dnode1 -c numOfMnodes -v 3 +system sh/cfg.sh -n dnode2 -c numOfMnodes -v 3 +system sh/cfg.sh -n dnode3 -c numOfMnodes -v 3 system sh/cfg.sh -n dnode1 -c numOfTotalVnodes -v 10 system sh/cfg.sh -n dnode2 -c numOfTotalVnodes -v 10 system sh/cfg.sh -n dnode3 -c numOfTotalVnodes -v 10 diff --git a/tests/script/unique/db/delete_part.sim b/tests/script/unique/db/delete_part.sim index 179d729d8dae2b7b0a8bf890fdfb43e5be76a05a..a4ad615f60234c6cb928e95ffe60888389130097 100644 --- a/tests/script/unique/db/delete_part.sim +++ b/tests/script/unique/db/delete_part.sim @@ -9,15 +9,15 @@ system sh/cfg.sh -n dnode2 -c wallevel -v 2 system sh/cfg.sh -n dnode3 -c wallevel -v 2 system sh/cfg.sh -n dnode4 -c wallevel -v 2 -system sh/cfg.sh -n dnode1 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode2 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode3 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode4 -c numOfMPeers -v 1 - -system sh/cfg.sh -n dnode1 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode2 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode3 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode4 -c mgmtEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode1 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode2 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode3 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode4 -c numOfMnodes -v 1 + +system sh/cfg.sh -n dnode1 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode2 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode3 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode4 -c mnodeEqualVnodeNum -v 4 system sh/cfg.sh -n dnode1 -c numOfTotalVnodes -v 4 system sh/cfg.sh -n dnode2 -c numOfTotalVnodes -v 4 diff --git a/tests/script/unique/db/replica_add12.sim b/tests/script/unique/db/replica_add12.sim index b0de807fca7d1b293918092606a5a0730c790556..ec376d4e15fbc928787bf31fa4377dd0746e7ed9 100644 --- a/tests/script/unique/db/replica_add12.sim +++ b/tests/script/unique/db/replica_add12.sim @@ -10,15 +10,15 @@ system sh/cfg.sh -n dnode2 -c wallevel -v 2 system sh/cfg.sh -n dnode3 -c wallevel -v 2 system sh/cfg.sh -n dnode4 -c wallevel -v 2 -system sh/cfg.sh -n dnode1 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode2 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode3 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode4 -c numOfMPeers -v 1 - -system sh/cfg.sh -n dnode1 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode2 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode3 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode4 -c mgmtEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode1 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode2 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode3 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode4 -c numOfMnodes -v 1 + +system sh/cfg.sh -n dnode1 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode2 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode3 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode4 -c mnodeEqualVnodeNum -v 4 system sh/cfg.sh -n dnode1 -c numOfTotalVnodes -v 4 system sh/cfg.sh -n dnode2 -c numOfTotalVnodes -v 4 diff --git a/tests/script/unique/db/replica_add13.sim b/tests/script/unique/db/replica_add13.sim index 9f66faab0aa60e8493234d5ff2fb933f130a672e..f6d1c9012ba346c54e52d8765a3d3db2d70cd114 100644 --- a/tests/script/unique/db/replica_add13.sim +++ b/tests/script/unique/db/replica_add13.sim @@ -10,15 +10,15 @@ system sh/cfg.sh -n dnode2 -c wallevel -v 2 system sh/cfg.sh -n dnode3 -c wallevel -v 2 system sh/cfg.sh -n dnode4 -c wallevel -v 2 -system sh/cfg.sh -n dnode1 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode2 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode3 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode4 -c numOfMPeers -v 1 - -system sh/cfg.sh -n dnode1 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode2 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode3 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode4 -c mgmtEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode1 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode2 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode3 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode4 -c numOfMnodes -v 1 + +system sh/cfg.sh -n dnode1 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode2 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode3 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode4 -c mnodeEqualVnodeNum -v 4 system sh/cfg.sh -n dnode1 -c numOfTotalVnodes -v 4 system sh/cfg.sh -n dnode2 -c numOfTotalVnodes -v 4 diff --git a/tests/script/unique/db/replica_add23.sim b/tests/script/unique/db/replica_add23.sim index 2d293183af197379f2a541fc0a2662dc26c8d1cb..4defaef153326a078eb46775b2bacf27c458cbc2 100644 --- a/tests/script/unique/db/replica_add23.sim +++ b/tests/script/unique/db/replica_add23.sim @@ -10,15 +10,15 @@ system sh/cfg.sh -n dnode2 -c wallevel -v 2 system sh/cfg.sh -n dnode3 -c wallevel -v 2 system sh/cfg.sh -n dnode4 -c wallevel -v 2 -system sh/cfg.sh -n dnode1 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode2 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode3 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode4 -c numOfMPeers -v 1 - -system sh/cfg.sh -n dnode1 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode2 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode3 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode4 -c mgmtEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode1 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode2 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode3 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode4 -c numOfMnodes -v 1 + +system sh/cfg.sh -n dnode1 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode2 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode3 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode4 -c mnodeEqualVnodeNum -v 4 system sh/cfg.sh -n dnode1 -c numOfTotalVnodes -v 4 system sh/cfg.sh -n dnode2 -c numOfTotalVnodes -v 4 diff --git a/tests/script/unique/db/replica_part.sim b/tests/script/unique/db/replica_part.sim index 76e3eaabbe1ad7cfa94177186df6e66327f248a7..3011fac02c406e90e6acd0c673d21d8fc25fe8d5 100644 --- a/tests/script/unique/db/replica_part.sim +++ b/tests/script/unique/db/replica_part.sim @@ -7,12 +7,12 @@ system sh/deploy.sh -n dnode3 -i 3 system sh/cfg.sh -n dnode1 -c wallevel -v 2 system sh/cfg.sh -n dnode2 -c wallevel -v 2 system sh/cfg.sh -n dnode3 -c wallevel -v 2 -system sh/cfg.sh -n dnode1 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode2 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode3 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode1 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode2 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode3 -c mgmtEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode1 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode2 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode3 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode1 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode2 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode3 -c mnodeEqualVnodeNum -v 4 system sh/cfg.sh -n dnode1 -c numOfTotalVnodes -v 4 system sh/cfg.sh -n dnode2 -c numOfTotalVnodes -v 4 system sh/cfg.sh -n dnode3 -c numOfTotalVnodes -v 4 diff --git a/tests/script/unique/db/replica_reduce21.sim b/tests/script/unique/db/replica_reduce21.sim index 3a324c604a55f0be44bb1be94991f17007ba933c..efaa4fa95c201569ea7c3469723713a18bb2e2f7 100644 --- a/tests/script/unique/db/replica_reduce21.sim +++ b/tests/script/unique/db/replica_reduce21.sim @@ -7,12 +7,12 @@ system sh/deploy.sh -n dnode3 -i 3 system sh/cfg.sh -n dnode1 -c wallevel -v 2 system sh/cfg.sh -n dnode2 -c wallevel -v 2 system sh/cfg.sh -n dnode3 -c wallevel -v 2 -system sh/cfg.sh -n dnode1 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode2 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode3 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode1 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode2 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode3 -c mgmtEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode1 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode2 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode3 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode1 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode2 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode3 -c mnodeEqualVnodeNum -v 4 system sh/cfg.sh -n dnode1 -c numOfTotalVnodes -v 4 system sh/cfg.sh -n dnode2 -c numOfTotalVnodes -v 4 system sh/cfg.sh -n dnode3 -c numOfTotalVnodes -v 4 diff --git a/tests/script/unique/db/replica_reduce31.sim b/tests/script/unique/db/replica_reduce31.sim index fdb5e3c1dc813d88cbbca7682ee4812b5168fb46..98aec8dd19b644e0e4a58ddb9098879da1fdb1ce 100644 --- a/tests/script/unique/db/replica_reduce31.sim +++ b/tests/script/unique/db/replica_reduce31.sim @@ -7,12 +7,12 @@ system sh/deploy.sh -n dnode3 -i 3 system sh/cfg.sh -n dnode1 -c wallevel -v 2 system sh/cfg.sh -n dnode2 -c wallevel -v 2 system sh/cfg.sh -n dnode3 -c wallevel -v 2 -system sh/cfg.sh -n dnode1 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode2 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode3 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode1 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode2 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode3 -c mgmtEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode1 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode2 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode3 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode1 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode2 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode3 -c mnodeEqualVnodeNum -v 4 system sh/cfg.sh -n dnode1 -c numOfTotalVnodes -v 4 system sh/cfg.sh -n dnode2 -c numOfTotalVnodes -v 4 system sh/cfg.sh -n dnode3 -c numOfTotalVnodes -v 4 diff --git a/tests/script/unique/db/replica_reduce32.sim b/tests/script/unique/db/replica_reduce32.sim index 83b5cb7a5bc20fa2f19963db1b5f1cd881449de4..735f9abb820731a06afa13e9c9925fe767b7ac6e 100644 --- a/tests/script/unique/db/replica_reduce32.sim +++ b/tests/script/unique/db/replica_reduce32.sim @@ -7,12 +7,12 @@ system sh/deploy.sh -n dnode3 -i 3 system sh/cfg.sh -n dnode1 -c wallevel -v 2 system sh/cfg.sh -n dnode2 -c wallevel -v 2 system sh/cfg.sh -n dnode3 -c wallevel -v 2 -system sh/cfg.sh -n dnode1 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode2 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode3 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode1 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode2 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode3 -c mgmtEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode1 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode2 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode3 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode1 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode2 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode3 -c mnodeEqualVnodeNum -v 4 system sh/cfg.sh -n dnode1 -c numOfTotalVnodes -v 4 system sh/cfg.sh -n dnode2 -c numOfTotalVnodes -v 4 system sh/cfg.sh -n dnode3 -c numOfTotalVnodes -v 4 diff --git a/tests/script/unique/dnode/alternativeRole.sim b/tests/script/unique/dnode/alternativeRole.sim new file mode 100644 index 0000000000000000000000000000000000000000..7c33cac0dc1a62bb6f56d5a5744c07327cb74fc5 --- /dev/null +++ b/tests/script/unique/dnode/alternativeRole.sim @@ -0,0 +1,91 @@ +system sh/stop_dnodes.sh + +system sh/deploy.sh -n dnode1 -i 1 +system sh/deploy.sh -n dnode2 -i 2 +system sh/deploy.sh -n dnode3 -i 3 + +system sh/cfg.sh -n dnode1 -c alternativeRole -v 1 +system sh/cfg.sh -n dnode2 -c alternativeRole -v 2 +system sh/cfg.sh -n dnode3 -c alternativeRole -v 0 + +system sh/cfg.sh -n dnode1 -c wallevel -v 1 +system sh/cfg.sh -n dnode2 -c wallevel -v 1 +system sh/cfg.sh -n dnode3 -c wallevel -v 1 + +system sh/cfg.sh -n dnode1 -c numOfMnodes -v 3 +system sh/cfg.sh -n dnode2 -c numOfMnodes -v 3 +system sh/cfg.sh -n dnode3 -c numOfMnodes -v 3 + +print ========== step1 +system sh/exec_up.sh -n dnode1 -s start +sql connect +sql create dnode $hostname2 +system sh/exec_up.sh -n dnode2 -s start +sleep 3000 +sql create dnode $hostname3 +system sh/exec_up.sh -n dnode3 -s start +sleep 3000 + +sql show dnodes +print dnode1 $data5_1 +print dnode1 $data5_2 +print dnode1 $data5_3 + +if $data5_1 != mnode then + return -1 +endi +if $data5_2 != vnode then + return -1 +endi +if $data5_3 != any then + return -1 +endi + +sql show mnodes +print dnode1 ==> $data2_1 +print dnode2 ==> $data2_2 +print dnode3 ==> $data2_3 +if $data2_1 != master then + return -1 +endi +if $data2_2 != null then + return -1 +endi +if $data2_3 != slave then + return -1 +endi + +print ========== step2 +sql create database d1 maxTables 4 +sql create table d1.t1 (ts timestamp, i int) +sql create table d1.t2 (ts timestamp, i int) +sql create table d1.t3 (ts timestamp, i int) +sql create table d1.t4 (ts timestamp, i int) +sql create table d1.t5 (ts timestamp, i int) +sql create table d1.t6 (ts timestamp, i int) +sql create table d1.t7 (ts timestamp, i int) +sql create table d1.t8 (ts timestamp, i int) + +sql show dnodes +print dnode1 $data2_1 +print dnode2 $data2_2 +print dnode3 $data2_3 + +if $data2_1 != 0 then + return -1 +endi +if $data2_2 != 1 then + return -1 +endi +if $data2_3 != 1 then + return -1 +endi + +system sh/exec_up.sh -n dnode1 -s stop -x SIGINT +system sh/exec_up.sh -n dnode2 -s stop -x SIGINT +system sh/exec_up.sh -n dnode3 -s stop -x SIGINT +system sh/exec_up.sh -n dnode4 -s stop -x SIGINT +system sh/exec_up.sh -n dnode5 -s stop -x SIGINT +system sh/exec_up.sh -n dnode6 -s stop -x SIGINT +system sh/exec_up.sh -n dnode7 -s stop -x SIGINT +system sh/exec_up.sh -n dnode8 -s stop -x SIGINT \ No newline at end of file diff --git a/tests/script/unique/dnode/balance1.sim b/tests/script/unique/dnode/balance1.sim index 9a598e1704815f35929dd123aad7b53594e4a5ee..a6be723d7c16ce39f0fd76f560b5b107a065130f 100644 --- a/tests/script/unique/dnode/balance1.sim +++ b/tests/script/unique/dnode/balance1.sim @@ -10,10 +10,10 @@ system sh/cfg.sh -n dnode2 -c balanceInterval -v 10 system sh/cfg.sh -n dnode3 -c balanceInterval -v 10 system sh/cfg.sh -n dnode4 -c balanceInterval -v 10 -system sh/cfg.sh -n dnode1 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode2 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode3 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode4 -c mgmtEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode1 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode2 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode3 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode4 -c mnodeEqualVnodeNum -v 4 system sh/cfg.sh -n dnode1 -c wallevel -v 2 system sh/cfg.sh -n dnode2 -c wallevel -v 2 diff --git a/tests/script/unique/dnode/balance2.sim b/tests/script/unique/dnode/balance2.sim index f039579012798fb338e1b742f1c76d1a8ccfa6bd..c60ac145171b71e66a87a63d06925661c2feddb2 100644 --- a/tests/script/unique/dnode/balance2.sim +++ b/tests/script/unique/dnode/balance2.sim @@ -6,11 +6,11 @@ system sh/deploy.sh -n dnode3 -i 3 system sh/deploy.sh -n dnode4 -i 4 system sh/deploy.sh -n dnode5 -i 5 -system sh/cfg.sh -n dnode1 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode2 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode3 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode4 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode5 -c mgmtEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode1 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode2 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode3 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode4 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode5 -c mnodeEqualVnodeNum -v 4 system sh/cfg.sh -n dnode1 -c wallevel -v 1 system sh/cfg.sh -n dnode2 -c wallevel -v 1 diff --git a/tests/script/unique/dnode/balance3.sim b/tests/script/unique/dnode/balance3.sim index acb0d033d4a21ed7bb262b9437cdb4d404fe437f..f0bf0c7c2b7e124cb98d3e1fcbf23f2785bbda49 100644 --- a/tests/script/unique/dnode/balance3.sim +++ b/tests/script/unique/dnode/balance3.sim @@ -7,12 +7,12 @@ system sh/deploy.sh -n dnode4 -i 4 system sh/deploy.sh -n dnode5 -i 5 system sh/deploy.sh -n dnode6 -i 6 -system sh/cfg.sh -n dnode1 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode2 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode3 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode4 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode5 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode6 -c mgmtEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode1 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode2 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode3 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode4 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode5 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode6 -c mnodeEqualVnodeNum -v 4 system sh/cfg.sh -n dnode1 -c wallevel -v 1 system sh/cfg.sh -n dnode2 -c wallevel -v 1 diff --git a/tests/script/unique/dnode/balancex.sim b/tests/script/unique/dnode/balancex.sim index 202c9b5396206815e8bf4480505542fff6ea78eb..14b09826784a7d709a7d977347fc5a3116556353 100644 --- a/tests/script/unique/dnode/balancex.sim +++ b/tests/script/unique/dnode/balancex.sim @@ -5,10 +5,10 @@ system sh/deploy.sh -n dnode2 -i 2 system sh/deploy.sh -n dnode3 -i 3 system sh/deploy.sh -n dnode4 -i 4 -system sh/cfg.sh -n dnode1 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode2 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode3 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode4 -c mgmtEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode1 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode2 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode3 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode4 -c mnodeEqualVnodeNum -v 4 system sh/cfg.sh -n dnode1 -c wallevel -v 1 system sh/cfg.sh -n dnode2 -c wallevel -v 1 diff --git a/tests/script/unique/dnode/monitor_bug.sim b/tests/script/unique/dnode/monitor_bug.sim index 8726caf1db8d43f4418c6873fbd96cf46dd46579..a20185240f351457dab37d47aa3066e063408976 100644 --- a/tests/script/unique/dnode/monitor_bug.sim +++ b/tests/script/unique/dnode/monitor_bug.sim @@ -3,8 +3,8 @@ system sh/stop_dnodes.sh system sh/deploy.sh -n dnode1 -i 1 system sh/deploy.sh -n dnode2 -i 2 -system sh/cfg.sh -n dnode1 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode2 -c mgmtEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode1 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode2 -c mnodeEqualVnodeNum -v 4 system sh/cfg.sh -n dnode1 -c wallevel -v 1 system sh/cfg.sh -n dnode2 -c wallevel -v 1 diff --git a/tests/script/unique/dnode/offline1.sim b/tests/script/unique/dnode/offline1.sim index 5e4ab65be37e90a4cf6737d6fa7b485c16dd61ae..9d068e835875d2f6cb5448d5b7947bde1420ffe6 100644 --- a/tests/script/unique/dnode/offline1.sim +++ b/tests/script/unique/dnode/offline1.sim @@ -12,9 +12,9 @@ system sh/cfg.sh -n dnode1 -c balanceInterval -v 5 system sh/cfg.sh -n dnode2 -c balanceInterval -v 5 system sh/cfg.sh -n dnode3 -c balanceInterval -v 5 -system sh/cfg.sh -n dnode1 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode2 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode3 -c mgmtEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode1 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode2 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode3 -c mnodeEqualVnodeNum -v 4 system sh/cfg.sh -n dnode1 -c wallevel -v 1 system sh/cfg.sh -n dnode2 -c wallevel -v 1 diff --git a/tests/script/unique/dnode/offline2.sim b/tests/script/unique/dnode/offline2.sim index 9d8ba8bf9d15442a271f9d2755fce6f3f28a274e..e9c1218e1f784974147934ebef43d0b2d22f3da8 100644 --- a/tests/script/unique/dnode/offline2.sim +++ b/tests/script/unique/dnode/offline2.sim @@ -12,9 +12,9 @@ system sh/cfg.sh -n dnode1 -c balanceInterval -v 5 system sh/cfg.sh -n dnode2 -c balanceInterval -v 5 system sh/cfg.sh -n dnode3 -c balanceInterval -v 5 -system sh/cfg.sh -n dnode1 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode2 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode3 -c mgmtEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode1 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode2 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode3 -c mnodeEqualVnodeNum -v 4 system sh/cfg.sh -n dnode1 -c wallevel -v 1 system sh/cfg.sh -n dnode2 -c wallevel -v 1 diff --git a/tests/script/unique/dnode/remove1.sim b/tests/script/unique/dnode/remove1.sim index 6b23014b0348c9149bd301cf97dd1b7e7210c6c3..2eb1fe4ae73f5af2b6398caba462f1cd06efdf01 100644 --- a/tests/script/unique/dnode/remove1.sim +++ b/tests/script/unique/dnode/remove1.sim @@ -5,10 +5,10 @@ system sh/deploy.sh -n dnode2 -i 2 system sh/deploy.sh -n dnode3 -i 3 system sh/deploy.sh -n dnode4 -i 4 -system sh/cfg.sh -n dnode1 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode2 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode3 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode4 -c mgmtEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode1 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode2 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode3 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode4 -c mnodeEqualVnodeNum -v 4 system sh/cfg.sh -n dnode1 -c wallevel -v 1 system sh/cfg.sh -n dnode2 -c wallevel -v 1 diff --git a/tests/script/unique/dnode/remove2.sim b/tests/script/unique/dnode/remove2.sim index 42cb9e5485b2a65b67b29bccff99f73055056ed0..db77a8e05dfd41e96267c4f6a9b97233cdcf728f 100644 --- a/tests/script/unique/dnode/remove2.sim +++ b/tests/script/unique/dnode/remove2.sim @@ -5,10 +5,10 @@ system sh/deploy.sh -n dnode2 -i 2 system sh/deploy.sh -n dnode3 -i 3 system sh/deploy.sh -n dnode4 -i 4 -system sh/cfg.sh -n dnode1 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode2 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode3 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode4 -c mgmtEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode1 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode2 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode3 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode4 -c mnodeEqualVnodeNum -v 4 system sh/cfg.sh -n dnode1 -c wallevel -v 2 system sh/cfg.sh -n dnode2 -c wallevel -v 2 diff --git a/tests/script/unique/dnode/vnode_clean.sim b/tests/script/unique/dnode/vnode_clean.sim index d46e1a751ed788249f158e519ef901cdf46b886c..d42fbf4e2d594791234dad1df333d4d9567f7432 100644 --- a/tests/script/unique/dnode/vnode_clean.sim +++ b/tests/script/unique/dnode/vnode_clean.sim @@ -5,10 +5,10 @@ system sh/deploy.sh -n dnode2 -i 2 system sh/deploy.sh -n dnode3 -i 3 system sh/deploy.sh -n dnode4 -i 4 -system sh/cfg.sh -n dnode1 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode2 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode3 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode4 -c mgmtEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode1 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode2 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode3 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode4 -c mnodeEqualVnodeNum -v 4 system sh/cfg.sh -n dnode1 -c wallevel -v 1 system sh/cfg.sh -n dnode2 -c wallevel -v 1 diff --git a/tests/script/unique/import/replica2.sim b/tests/script/unique/import/replica2.sim index 70097a50221195b4f7bee6d7e5f54572faa509a3..fab26a31cd715394277fb5e7dd4baf93eb793d13 100644 --- a/tests/script/unique/import/replica2.sim +++ b/tests/script/unique/import/replica2.sim @@ -5,15 +5,15 @@ system sh/deploy.sh -n dnode2 -i 2 system sh/deploy.sh -n dnode3 -i 3 system sh/deploy.sh -n dnode4 -i 4 -system sh/cfg.sh -n dnode1 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode2 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode3 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode4 -c numOfMPeers -v 1 - -system sh/cfg.sh -n dnode1 -c mgmtEqualVnodeNum -v 10 -system sh/cfg.sh -n dnode2 -c mgmtEqualVnodeNum -v 10 -system sh/cfg.sh -n dnode3 -c mgmtEqualVnodeNum -v 10 -system sh/cfg.sh -n dnode4 -c mgmtEqualVnodeNum -v 10 +system sh/cfg.sh -n dnode1 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode2 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode3 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode4 -c numOfMnodes -v 1 + +system sh/cfg.sh -n dnode1 -c mnodeEqualVnodeNum -v 10 +system sh/cfg.sh -n dnode2 -c mnodeEqualVnodeNum -v 10 +system sh/cfg.sh -n dnode3 -c mnodeEqualVnodeNum -v 10 +system sh/cfg.sh -n dnode4 -c mnodeEqualVnodeNum -v 10 system sh/cfg.sh -n dnode1 -c maxtablesPerVnode -v 2000 system sh/cfg.sh -n dnode2 -c maxtablesPerVnode -v 2000 diff --git a/tests/script/unique/import/replica3.sim b/tests/script/unique/import/replica3.sim index ad236bfb0cae4a821b757d93d0f04285439f0f7b..3939bc2e1be9c3c996e7f1fb2f5ed48e5ab9d27e 100644 --- a/tests/script/unique/import/replica3.sim +++ b/tests/script/unique/import/replica3.sim @@ -5,15 +5,15 @@ system sh/deploy.sh -n dnode2 -i 2 system sh/deploy.sh -n dnode3 -i 3 system sh/deploy.sh -n dnode4 -i 4 -system sh/cfg.sh -n dnode1 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode2 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode3 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode4 -c numOfMPeers -v 1 - -system sh/cfg.sh -n dnode1 -c mgmtEqualVnodeNum -v 10 -system sh/cfg.sh -n dnode2 -c mgmtEqualVnodeNum -v 10 -system sh/cfg.sh -n dnode3 -c mgmtEqualVnodeNum -v 10 -system sh/cfg.sh -n dnode4 -c mgmtEqualVnodeNum -v 10 +system sh/cfg.sh -n dnode1 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode2 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode3 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode4 -c numOfMnodes -v 1 + +system sh/cfg.sh -n dnode1 -c mnodeEqualVnodeNum -v 10 +system sh/cfg.sh -n dnode2 -c mnodeEqualVnodeNum -v 10 +system sh/cfg.sh -n dnode3 -c mnodeEqualVnodeNum -v 10 +system sh/cfg.sh -n dnode4 -c mnodeEqualVnodeNum -v 10 system sh/cfg.sh -n dnode1 -c maxtablesPerVnode -v 2000 system sh/cfg.sh -n dnode2 -c maxtablesPerVnode -v 2000 diff --git a/tests/script/unique/mnode/mgmt22.sim b/tests/script/unique/mnode/mgmt22.sim index c6ea026af9a522c29fb85701fe5c76bb547d87cc..e306b1b02a9f2b21abbaad6ae2ec2db7e0f38b6d 100644 --- a/tests/script/unique/mnode/mgmt22.sim +++ b/tests/script/unique/mnode/mgmt22.sim @@ -3,9 +3,9 @@ system sh/deploy.sh -n dnode1 -i 1 system sh/deploy.sh -n dnode2 -i 2 system sh/deploy.sh -n dnode3 -i 3 -system sh/cfg.sh -n dnode1 -c numOfMPeers -v 2 -system sh/cfg.sh -n dnode2 -c numOfMPeers -v 2 -system sh/cfg.sh -n dnode3 -c numOfMPeers -v 2 +system sh/cfg.sh -n dnode1 -c numOfMnodes -v 2 +system sh/cfg.sh -n dnode2 -c numOfMnodes -v 2 +system sh/cfg.sh -n dnode3 -c numOfMnodes -v 2 print ============== step1 system sh/exec_up.sh -n dnode1 -s start diff --git a/tests/script/unique/mnode/mgmt23.sim b/tests/script/unique/mnode/mgmt23.sim index 8c144dd0c44e23dc381ead7aa3ff32815965b6e1..2a28f8daf98bd5ffa601858c31f250d8b2881094 100644 --- a/tests/script/unique/mnode/mgmt23.sim +++ b/tests/script/unique/mnode/mgmt23.sim @@ -3,9 +3,9 @@ system sh/deploy.sh -n dnode1 -i 1 system sh/deploy.sh -n dnode2 -i 2 system sh/deploy.sh -n dnode3 -i 3 -system sh/cfg.sh -n dnode1 -c numOfMPeers -v 2 -system sh/cfg.sh -n dnode2 -c numOfMPeers -v 2 -system sh/cfg.sh -n dnode3 -c numOfMPeers -v 2 +system sh/cfg.sh -n dnode1 -c numOfMnodes -v 2 +system sh/cfg.sh -n dnode2 -c numOfMnodes -v 2 +system sh/cfg.sh -n dnode3 -c numOfMnodes -v 2 print ============== step1 system sh/exec_up.sh -n dnode1 -s start @@ -91,7 +91,7 @@ print ============== step5 sleep 2000 sql create dnode $hostname2 system sh/deploy.sh -n dnode2 -i 2 -system sh/cfg.sh -n dnode2 -c numOfMPeers -v 2 +system sh/cfg.sh -n dnode2 -c numOfMnodes -v 2 system sh/exec_up.sh -n dnode2 -s start sleep 8000 diff --git a/tests/script/unique/mnode/mgmt24.sim b/tests/script/unique/mnode/mgmt24.sim index 4813d96f5f36b584952a8f8025d0a7ac9297eb07..c989faca818d6b67ca44fd294737edb830447ad1 100644 --- a/tests/script/unique/mnode/mgmt24.sim +++ b/tests/script/unique/mnode/mgmt24.sim @@ -3,9 +3,9 @@ system sh/deploy.sh -n dnode1 -i 1 system sh/deploy.sh -n dnode2 -i 2 system sh/deploy.sh -n dnode3 -i 3 -system sh/cfg.sh -n dnode1 -c numOfMPeers -v 2 -system sh/cfg.sh -n dnode2 -c numOfMPeers -v 2 -system sh/cfg.sh -n dnode3 -c numOfMPeers -v 2 +system sh/cfg.sh -n dnode1 -c numOfMnodes -v 2 +system sh/cfg.sh -n dnode2 -c numOfMnodes -v 2 +system sh/cfg.sh -n dnode3 -c numOfMnodes -v 2 print ============== step1 system sh/exec_up.sh -n dnode1 -s start diff --git a/tests/script/unique/mnode/mgmt25.sim b/tests/script/unique/mnode/mgmt25.sim index 8c8eeba466c31ab6efabd00e129a78d13c13ecd3..f955ac70c39a3ee185c94a7542d6166bcaacd0c9 100644 --- a/tests/script/unique/mnode/mgmt25.sim +++ b/tests/script/unique/mnode/mgmt25.sim @@ -3,9 +3,9 @@ system sh/deploy.sh -n dnode1 -i 1 system sh/deploy.sh -n dnode2 -i 2 system sh/deploy.sh -n dnode3 -i 3 -system sh/cfg.sh -n dnode1 -c numOfMPeers -v 2 -system sh/cfg.sh -n dnode2 -c numOfMPeers -v 2 -system sh/cfg.sh -n dnode3 -c numOfMPeers -v 2 +system sh/cfg.sh -n dnode1 -c numOfMnodes -v 2 +system sh/cfg.sh -n dnode2 -c numOfMnodes -v 2 +system sh/cfg.sh -n dnode3 -c numOfMnodes -v 2 print ============== step1 system sh/exec_up.sh -n dnode1 -s start diff --git a/tests/script/unique/mnode/mgmt26.sim b/tests/script/unique/mnode/mgmt26.sim index 373a0abaed191075666085574dc013c3f1cb0721..170905195becf44f1e5029e7d54aa1e3f4532bf5 100644 --- a/tests/script/unique/mnode/mgmt26.sim +++ b/tests/script/unique/mnode/mgmt26.sim @@ -3,9 +3,9 @@ system sh/deploy.sh -n dnode1 -i 1 system sh/deploy.sh -n dnode2 -i 2 system sh/deploy.sh -n dnode3 -i 3 -system sh/cfg.sh -n dnode1 -c numOfMPeers -v 2 -system sh/cfg.sh -n dnode2 -c numOfMPeers -v 2 -system sh/cfg.sh -n dnode3 -c numOfMPeers -v 2 +system sh/cfg.sh -n dnode1 -c numOfMnodes -v 2 +system sh/cfg.sh -n dnode2 -c numOfMnodes -v 2 +system sh/cfg.sh -n dnode3 -c numOfMnodes -v 2 print ============== step1 system sh/exec_up.sh -n dnode1 -s start @@ -89,7 +89,7 @@ endi print ============== step5 system sh/exec_up.sh -n dnode2 -s stop system sh/deploy.sh -n dnode2 -i 2 -system sh/cfg.sh -n dnode2 -c numOfMPeers -v 2 +system sh/cfg.sh -n dnode2 -c numOfMnodes -v 2 sleep 5000 system sh/exec_up.sh -n dnode2 -s start sql create dnode $hostname2 diff --git a/tests/script/unique/mnode/mgmt33.sim b/tests/script/unique/mnode/mgmt33.sim index e3a62a2d22cae72ef7a12d9f85dc5da1046a88a3..4a1fbf00b672151fdb5e8f7055d947549c15c402 100644 --- a/tests/script/unique/mnode/mgmt33.sim +++ b/tests/script/unique/mnode/mgmt33.sim @@ -3,9 +3,9 @@ system sh/deploy.sh -n dnode1 -i 1 system sh/deploy.sh -n dnode2 -i 2 system sh/deploy.sh -n dnode3 -i 3 -system sh/cfg.sh -n dnode1 -c numOfMPeers -v 3 -system sh/cfg.sh -n dnode2 -c numOfMPeers -v 3 -system sh/cfg.sh -n dnode3 -c numOfMPeers -v 3 +system sh/cfg.sh -n dnode1 -c numOfMnodes -v 3 +system sh/cfg.sh -n dnode2 -c numOfMnodes -v 3 +system sh/cfg.sh -n dnode3 -c numOfMnodes -v 3 print ============== step1 system sh/exec_up.sh -n dnode1 -s start @@ -97,7 +97,7 @@ system sh/exec_up.sh -n dnode2 -s stop sleep 3000 system sh/deploy.sh -n dnode2 -i 2 -system sh/cfg.sh -n dnode2 -c numOfMPeers -v 3 +system sh/cfg.sh -n dnode2 -c numOfMnodes -v 3 system sh/exec_up.sh -n dnode2 -s start print ============== step5 diff --git a/tests/script/unique/mnode/mgmt34.sim b/tests/script/unique/mnode/mgmt34.sim index 4409c825c8138420ab853a787b8d8041bf544584..ada5321f1b298fca33e36b674566c1794d8cb1c7 100644 --- a/tests/script/unique/mnode/mgmt34.sim +++ b/tests/script/unique/mnode/mgmt34.sim @@ -5,10 +5,10 @@ system sh/deploy.sh -n dnode2 -i 2 system sh/deploy.sh -n dnode3 -i 3 system sh/deploy.sh -n dnode4 -i 4 -system sh/cfg.sh -n dnode1 -c numOfMPeers -v 3 -system sh/cfg.sh -n dnode2 -c numOfMPeers -v 3 -system sh/cfg.sh -n dnode3 -c numOfMPeers -v 3 -system sh/cfg.sh -n dnode4 -c numOfMPeers -v 3 +system sh/cfg.sh -n dnode1 -c numOfMnodes -v 3 +system sh/cfg.sh -n dnode2 -c numOfMnodes -v 3 +system sh/cfg.sh -n dnode3 -c numOfMnodes -v 3 +system sh/cfg.sh -n dnode4 -c numOfMnodes -v 3 print ============== step1 system sh/exec_up.sh -n dnode1 -s start @@ -145,7 +145,7 @@ system sh/exec_up.sh -n dnode2 -s stop sleep 3000 system sh/deploy.sh -n dnode2 -i 2 -system sh/cfg.sh -n dnode2 -c numOfMPeers -v 3 +system sh/cfg.sh -n dnode2 -c numOfMnodes -v 3 system sh/exec_up.sh -n dnode2 -s start print ============== step6 diff --git a/tests/script/unique/mnode/mgmtr2.sim b/tests/script/unique/mnode/mgmtr2.sim index 625e42a334dc9de7cb85111e17257878e8d2552c..77e5e601bb6493688a42307c34df52f8906a151f 100644 --- a/tests/script/unique/mnode/mgmtr2.sim +++ b/tests/script/unique/mnode/mgmtr2.sim @@ -3,9 +3,9 @@ system sh/deploy.sh -n dnode1 -i 1 system sh/deploy.sh -n dnode2 -i 2 system sh/deploy.sh -n dnode3 -i 3 -system sh/cfg.sh -n dnode1 -c numOfMPeers -v 2 -system sh/cfg.sh -n dnode2 -c numOfMPeers -v 2 -system sh/cfg.sh -n dnode3 -c numOfMPeers -v 2 +system sh/cfg.sh -n dnode1 -c numOfMnodes -v 2 +system sh/cfg.sh -n dnode2 -c numOfMnodes -v 2 +system sh/cfg.sh -n dnode3 -c numOfMnodes -v 2 print ============== step1 system sh/exec_up.sh -n dnode1 -s start diff --git a/tests/script/unique/stable/balance_replica1.sim b/tests/script/unique/stable/balance_replica1.sim index 9257ba79beeea5060ed8b4b7dc38e0ad7721c59b..4269c2fc7f1d5b50bd423435c71363101830f105 100644 --- a/tests/script/unique/stable/balance_replica1.sim +++ b/tests/script/unique/stable/balance_replica1.sim @@ -9,8 +9,8 @@ system sh/cfg.sh -n dnode1 -c balanceInterval -v 10 system sh/cfg.sh -n dnode2 -c balanceInterval -v 10 system sh/cfg.sh -n dnode1 -c walLevel -v 2 system sh/cfg.sh -n dnode2 -c walLevel -v 2 -system sh/cfg.sh -n dnode1 -c mgmtEqualVnodeNum -v 0 -system sh/cfg.sh -n dnode2 -c mgmtEqualVnodeNum -v 0 +system sh/cfg.sh -n dnode1 -c mnodeEqualVnodeNum -v 0 +system sh/cfg.sh -n dnode2 -c mnodeEqualVnodeNum -v 0 system sh/cfg.sh -n dnode1 -c maxtablesPerVnode -v 4 system sh/cfg.sh -n dnode2 -c maxtablesPerVnode -v 4 diff --git a/tests/script/unique/stream/metrics_balance.sim b/tests/script/unique/stream/metrics_balance.sim index e711baf2066f39a9a6614da36058e0d2fbfbb87b..e306da3cce90237088542e7be637f7e4aee28097 100644 --- a/tests/script/unique/stream/metrics_balance.sim +++ b/tests/script/unique/stream/metrics_balance.sim @@ -12,14 +12,14 @@ system sh/cfg.sh -n dnode1 -c balanceInterval -v 10 system sh/cfg.sh -n dnode2 -c balanceInterval -v 10 system sh/cfg.sh -n dnode1 -c walLevel -v 0 system sh/cfg.sh -n dnode2 -c walLevel -v 0 -system sh/cfg.sh -n dnode1 -c mgmtEqualVnodeNum -v 0 -system sh/cfg.sh -n dnode2 -c mgmtEqualVnodeNum -v 0 +system sh/cfg.sh -n dnode1 -c mnodeEqualVnodeNum -v 0 +system sh/cfg.sh -n dnode2 -c mnodeEqualVnodeNum -v 0 system sh/cfg.sh -n dnode1 -c maxTablesPerVnode -v 4 system sh/cfg.sh -n dnode2 -c maxTablesPerVnode -v 4 system sh/cfg.sh -n dnode1 -c tableMetaKeepTimer -v 5 system sh/cfg.sh -n dnode2 -c tableMetaKeepTimer -v 5 -system sh/cfg.sh -n dnode1 -c numOfMPeers -v 2 -system sh/cfg.sh -n dnode2 -c numOfMPeers -v 2 +system sh/cfg.sh -n dnode1 -c numOfMnodes -v 2 +system sh/cfg.sh -n dnode2 -c numOfMnodes -v 2 $dbPrefix = db $tbPrefix = tb diff --git a/tests/script/unique/stream/metrics_vnode_stop.sim b/tests/script/unique/stream/metrics_vnode_stop.sim index 4f85d75e7f9fb3561c20e635be2eaca385fcaf02..1f9cab48e5f738dabe1e07fa5b1017eb23282ac0 100644 --- a/tests/script/unique/stream/metrics_vnode_stop.sim +++ b/tests/script/unique/stream/metrics_vnode_stop.sim @@ -6,8 +6,8 @@ system sh/deploy.sh -n dnode1 -i 1 system sh/deploy.sh -n dnode2 -i 2 system sh/cfg.sh -n dnode1 -c walLevel -v 0 system sh/cfg.sh -n dnode2 -c walLevel -v 0 -system sh/cfg.sh -n dnode1 -c numOfMPeers -v 2 -system sh/cfg.sh -n dnode2 -c numOfMPeers -v 2 +system sh/cfg.sh -n dnode1 -c numOfMnodes -v 2 +system sh/cfg.sh -n dnode2 -c numOfMnodes -v 2 system sh/exec.sh -n dnode1 -s start sql connect diff --git a/tests/script/unique/stream/table_balance.sim b/tests/script/unique/stream/table_balance.sim index 7e33005b9b963b26f428ac6ee52a0fa83f243e5b..992b4c08eebfb5a81979ad8b71815d0503175049 100644 --- a/tests/script/unique/stream/table_balance.sim +++ b/tests/script/unique/stream/table_balance.sim @@ -12,12 +12,12 @@ system sh/cfg.sh -n dnode1 -c balanceInterval -v 10 system sh/cfg.sh -n dnode2 -c balanceInterval -v 10 system sh/cfg.sh -n dnode1 -c walLevel -v 0 system sh/cfg.sh -n dnode2 -c walLevel -v 0 -system sh/cfg.sh -n dnode1 -c mgmtEqualVnodeNum -v 0 -system sh/cfg.sh -n dnode2 -c mgmtEqualVnodeNum -v 0 +system sh/cfg.sh -n dnode1 -c mnodeEqualVnodeNum -v 0 +system sh/cfg.sh -n dnode2 -c mnodeEqualVnodeNum -v 0 system sh/cfg.sh -n dnode1 -c maxtablesPerVnode -v 4 system sh/cfg.sh -n dnode2 -c maxtablesPerVnode -v 4 -system sh/cfg.sh -n dnode1 -c numOfMPeers -v 2 -system sh/cfg.sh -n dnode2 -c numOfMPeers -v 2 +system sh/cfg.sh -n dnode1 -c numOfMnodes -v 2 +system sh/cfg.sh -n dnode2 -c numOfMnodes -v 2 $dbPrefix = tb_db $tbPrefix = tb_tb diff --git a/tests/script/unique/stream/table_move.sim b/tests/script/unique/stream/table_move.sim index 54cd3195dbba171c62aadad2beb9b66d1d75e2e8..0f264ade33f62a443a938e88470a9109f634057b 100644 --- a/tests/script/unique/stream/table_move.sim +++ b/tests/script/unique/stream/table_move.sim @@ -30,20 +30,20 @@ system sh/cfg.sh -n dnode2 -c wallevel -v 1 system sh/cfg.sh -n dnode3 -c wallevel -v 1 system sh/cfg.sh -n dnode4 -c wallevel -v 1 -system sh/cfg.sh -n dnode1 -c mgmtEqualVnodeNum -v 0 -system sh/cfg.sh -n dnode2 -c mgmtEqualVnodeNum -v 0 -system sh/cfg.sh -n dnode3 -c mgmtEqualVnodeNum -v 0 -system sh/cfg.sh -n dnode4 -c mgmtEqualVnodeNum -v 0 +system sh/cfg.sh -n dnode1 -c mnodeEqualVnodeNum -v 0 +system sh/cfg.sh -n dnode2 -c mnodeEqualVnodeNum -v 0 +system sh/cfg.sh -n dnode3 -c mnodeEqualVnodeNum -v 0 +system sh/cfg.sh -n dnode4 -c mnodeEqualVnodeNum -v 0 system sh/cfg.sh -n dnode1 -c maxtablesPerVnode -v 4 system sh/cfg.sh -n dnode2 -c maxtablesPerVnode -v 4 system sh/cfg.sh -n dnode3 -c maxtablesPerVnode -v 4 system sh/cfg.sh -n dnode4 -c maxtablesPerVnode -v 4 -system sh/cfg.sh -n dnode1 -c numOfMPeers -v 3 -system sh/cfg.sh -n dnode2 -c numOfMPeers -v 3 -system sh/cfg.sh -n dnode3 -c numOfMPeers -v 3 -system sh/cfg.sh -n dnode4 -c numOfMPeers -v 3 +system sh/cfg.sh -n dnode1 -c numOfMnodes -v 3 +system sh/cfg.sh -n dnode2 -c numOfMnodes -v 3 +system sh/cfg.sh -n dnode3 -c numOfMnodes -v 3 +system sh/cfg.sh -n dnode4 -c numOfMnodes -v 3 system sh/cfg.sh -n dnode1 -c maxVnodeConnections -v 1000 system sh/cfg.sh -n dnode1 -c maxMeterConnections -v 1000 diff --git a/tests/script/unique/stream/table_vnode_stop.sim b/tests/script/unique/stream/table_vnode_stop.sim index 8528ee675586941d99b0e3d5b16571b3cf5bbf3e..16350949b12e53dd452e1fba46524352a1953408 100644 --- a/tests/script/unique/stream/table_vnode_stop.sim +++ b/tests/script/unique/stream/table_vnode_stop.sim @@ -6,8 +6,8 @@ system sh/deploy.sh -n dnode1 -i 1 system sh/deploy.sh -n dnode2 -i 2 system sh/cfg.sh -n dnode1 -c walLevel -v 0 system sh/cfg.sh -n dnode2 -c walLevel -v 0 -system sh/cfg.sh -n dnode1 -c numOfMPeers -v 2 -system sh/cfg.sh -n dnode2 -c numOfMPeers -v 2 +system sh/cfg.sh -n dnode1 -c numOfMnodes -v 2 +system sh/cfg.sh -n dnode2 -c numOfMnodes -v 2 system sh/exec.sh -n dnode1 -s start sql connect diff --git a/tests/script/unique/vnode/many.sim b/tests/script/unique/vnode/many.sim index bb3e8813bd7666fa9fda2bb8495cf28a3648fe22..4b7fb8ffc47e9a8b669f6506458b21ad5267e3bf 100644 --- a/tests/script/unique/vnode/many.sim +++ b/tests/script/unique/vnode/many.sim @@ -8,14 +8,14 @@ system sh/cfg.sh -n dnode1 -c wallevel -v 2 system sh/cfg.sh -n dnode2 -c wallevel -v 2 system sh/cfg.sh -n dnode3 -c wallevel -v 2 system sh/cfg.sh -n dnode4 -c wallevel -v 2 -system sh/cfg.sh -n dnode1 -c numofMpeers -v 1 -system sh/cfg.sh -n dnode2 -c numofMpeers -v 1 -system sh/cfg.sh -n dnode3 -c numofMpeers -v 1 -system sh/cfg.sh -n dnode4 -c numofMpeers -v 1 -system sh/cfg.sh -n dnode1 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode2 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode3 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode4 -c mgmtEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode1 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode2 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode3 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode4 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode1 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode2 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode3 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode4 -c mnodeEqualVnodeNum -v 4 system sh/cfg.sh -n dnode1 -c numOfTotalVnodes -v 4 system sh/cfg.sh -n dnode2 -c numOfTotalVnodes -v 4 system sh/cfg.sh -n dnode3 -c numOfTotalVnodes -v 4 diff --git a/tests/script/unique/vnode/replica2_basic2.sim b/tests/script/unique/vnode/replica2_basic2.sim index 041db8249ac829c7ff355c6a76be74a3bddf44b8..6468538ec629152223580ac9a9f350e1a1165ea2 100644 --- a/tests/script/unique/vnode/replica2_basic2.sim +++ b/tests/script/unique/vnode/replica2_basic2.sim @@ -10,15 +10,15 @@ system sh/cfg.sh -n dnode2 -c wallevel -v 2 system sh/cfg.sh -n dnode3 -c wallevel -v 2 system sh/cfg.sh -n dnode4 -c wallevel -v 2 -system sh/cfg.sh -n dnode1 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode2 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode3 -c numOfMPeers -v 1 -system sh/cfg.sh -n dnode4 -c numOfMPeers -v 1 - -system sh/cfg.sh -n dnode1 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode2 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode3 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode4 -c mgmtEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode1 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode2 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode3 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode4 -c numOfMnodes -v 1 + +system sh/cfg.sh -n dnode1 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode2 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode3 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode4 -c mnodeEqualVnodeNum -v 4 system sh/cfg.sh -n dnode1 -c numOfTotalVnodes -v 4 system sh/cfg.sh -n dnode2 -c numOfTotalVnodes -v 4 diff --git a/tests/script/unique/vnode/replica2_repeat.sim b/tests/script/unique/vnode/replica2_repeat.sim index a6bd226484cf10dcbc2bcd8d4687e69b0fed63a7..4b024bfc9cb0ea8e81cc9cb4afc3ada5de805390 100644 --- a/tests/script/unique/vnode/replica2_repeat.sim +++ b/tests/script/unique/vnode/replica2_repeat.sim @@ -8,14 +8,14 @@ system sh/cfg.sh -n dnode1 -c wallevel -v 2 system sh/cfg.sh -n dnode2 -c wallevel -v 2 system sh/cfg.sh -n dnode3 -c wallevel -v 2 system sh/cfg.sh -n dnode4 -c wallevel -v 2 -system sh/cfg.sh -n dnode1 -c numofMpeers -v 1 -system sh/cfg.sh -n dnode2 -c numofMpeers -v 1 -system sh/cfg.sh -n dnode3 -c numofMpeers -v 1 -system sh/cfg.sh -n dnode4 -c numofMpeers -v 1 -system sh/cfg.sh -n dnode1 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode2 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode3 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode4 -c mgmtEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode1 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode2 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode3 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode4 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode1 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode2 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode3 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode4 -c mnodeEqualVnodeNum -v 4 system sh/exec_up.sh -n dnode1 -s start sql connect diff --git a/tests/script/unique/vnode/replica3_basic.sim b/tests/script/unique/vnode/replica3_basic.sim index 62910b947afdd5f3483c6d8e3ced05098ba3ac7e..288996548f851598f5451858980e2ea3bd0beac2 100644 --- a/tests/script/unique/vnode/replica3_basic.sim +++ b/tests/script/unique/vnode/replica3_basic.sim @@ -6,9 +6,9 @@ system sh/deploy.sh -n dnode3 -i 3 system sh/cfg.sh -n dnode1 -c wallevel -v 2 system sh/cfg.sh -n dnode2 -c wallevel -v 2 system sh/cfg.sh -n dnode3 -c wallevel -v 2 -system sh/cfg.sh -n dnode1 -c numofMpeers -v 3 -system sh/cfg.sh -n dnode2 -c numofMpeers -v 3 -system sh/cfg.sh -n dnode3 -c numofMpeers -v 3 +system sh/cfg.sh -n dnode1 -c numOfMnodes -v 3 +system sh/cfg.sh -n dnode2 -c numOfMnodes -v 3 +system sh/cfg.sh -n dnode3 -c numOfMnodes -v 3 system sh/exec_up.sh -n dnode1 -s start sql connect diff --git a/tests/script/unique/vnode/replica3_repeat.sim b/tests/script/unique/vnode/replica3_repeat.sim index 2f311a5d7a5fbba7eb754ac65b281947038fdaa0..b184bf194caffe893aac06485ffcf8d57bf101de 100644 --- a/tests/script/unique/vnode/replica3_repeat.sim +++ b/tests/script/unique/vnode/replica3_repeat.sim @@ -8,14 +8,14 @@ system sh/cfg.sh -n dnode1 -c wallevel -v 2 system sh/cfg.sh -n dnode2 -c wallevel -v 2 system sh/cfg.sh -n dnode3 -c wallevel -v 2 system sh/cfg.sh -n dnode4 -c wallevel -v 2 -system sh/cfg.sh -n dnode1 -c numofMpeers -v 1 -system sh/cfg.sh -n dnode2 -c numofMpeers -v 1 -system sh/cfg.sh -n dnode3 -c numofMpeers -v 1 -system sh/cfg.sh -n dnode4 -c numofMpeers -v 1 -system sh/cfg.sh -n dnode1 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode2 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode3 -c mgmtEqualVnodeNum -v 4 -system sh/cfg.sh -n dnode4 -c mgmtEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode1 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode2 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode3 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode4 -c numOfMnodes -v 1 +system sh/cfg.sh -n dnode1 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode2 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode3 -c mnodeEqualVnodeNum -v 4 +system sh/cfg.sh -n dnode4 -c mnodeEqualVnodeNum -v 4 system sh/exec_up.sh -n dnode1 -s start diff --git a/tests/script/unique/vnode/replica3_vgroup.sim b/tests/script/unique/vnode/replica3_vgroup.sim index 6315a4335cf47abe6f3b42ca7db4a2406fc90c5f..beb0667d08a8399fc9870650faaa57ccaafb2a6e 100644 --- a/tests/script/unique/vnode/replica3_vgroup.sim +++ b/tests/script/unique/vnode/replica3_vgroup.sim @@ -6,9 +6,9 @@ system sh/deploy.sh -n dnode3 -i 3 system sh/cfg.sh -n dnode1 -c wallevel -v 2 system sh/cfg.sh -n dnode2 -c wallevel -v 2 system sh/cfg.sh -n dnode3 -c wallevel -v 2 -system sh/cfg.sh -n dnode1 -c numofMpeers -v 3 -system sh/cfg.sh -n dnode2 -c numofMpeers -v 3 -system sh/cfg.sh -n dnode3 -c numofMpeers -v 3 +system sh/cfg.sh -n dnode1 -c numOfMnodes -v 3 +system sh/cfg.sh -n dnode2 -c numOfMnodes -v 3 +system sh/cfg.sh -n dnode3 -c numOfMnodes -v 3 system sh/exec_up.sh -n dnode1 -s start sql connect diff --git a/tests/test/c/CMakeLists.txt b/tests/test/c/CMakeLists.txt index 893abb8e821cdf2771add3d87ea01f4398ea4dd2..d40db5ee40fad13e2232dbe4ea62ec86c547b7af 100644 --- a/tests/test/c/CMakeLists.txt +++ b/tests/test/c/CMakeLists.txt @@ -16,6 +16,6 @@ IF ((TD_LINUX_64) OR (TD_LINUX_32 AND TD_ARM)) add_executable(importOneRow importOneRow.c) target_link_libraries(importOneRow taos_static pthread) - add_executable(importPerTabe importPerTabe.c) - target_link_libraries(importPerTabe taos_static pthread) + add_executable(importPerTable importPerTable.c) + target_link_libraries(importPerTable taos_static pthread) ENDIF() diff --git a/tests/test/c/importPerTabe.c b/tests/test/c/importPerTable.c similarity index 100% rename from tests/test/c/importPerTabe.c rename to tests/test/c/importPerTable.c