diff --git a/documentation20/cn/02.getting-started/docs.md b/documentation20/cn/02.getting-started/docs.md index a98159d8c408c7140b1a24f2534e575b38d1c56a..6eb58a1433ed0d43b313a9dc979ae5873ba00e8f 100644 --- a/documentation20/cn/02.getting-started/docs.md +++ b/documentation20/cn/02.getting-started/docs.md @@ -24,7 +24,7 @@ TDengine的安装非常简单，从下载到安装成功仅仅只要几秒钟。 ## 轻松启动 -安装成功后，用户可使用`systemctl`命令来启动TDengine的服务进程。 +安装成功后，用户可使用 `systemctl` 命令来启动 TDengine 的服务进程。 ```bash $ systemctl start taosd @@ -35,21 +35,22 @@ $ systemctl start taosd $ systemctl status taosd ``` -如果TDengine服务正常工作，那么您可以通过TDengine的命令行程序`taos`来访问并体验TDengine。 +如果 TDengine 服务正常工作，那么您可以通过 TDengine 的命令行程序 `taos` 来访问并体验 TDengine。 **注意：** -- systemctl命令需要 _root_ 权限来运行，如果您非 _root_ 用户，请在命令前添加 sudo -- 为更好的获得产品反馈，改善产品，TDengine会采集基本的使用信息，但您可以修改系统配置文件taos.cfg里的配置参数telemetryReporting, 将其设为0，就可将其关闭。 -- TDengine采用FQDN(一般就是hostname)作为节点的ID，为保证正常运行，需要给运行taosd的服务器配置好hostname，在客户端应用运行的机器配置好DNS服务或hosts文件，保证FQDN能够解析。 +- systemctl 命令需要 _root_ 权限来运行，如果您非 _root_ 用户，请在命令前添加 sudo 。 +- 为更好的获得产品反馈，改善产品，TDengine 会采集基本的使用信息，但您可以修改系统配置文件 taos.cfg 里的配置参数 telemetryReporting, 将其设为 0，就可将其关闭。 +- TDengine 采用 FQDN (一般就是 hostname )作为节点的 ID，为保证正常运行，需要给运行 taosd 的服务器配置好 hostname，在客户端应用运行的机器配置好 DNS 服务或 hosts 文件，保证 FQDN 能够解析。 +- `systemctl stop taosd` 指令在执行后并不会马上停止 TDengine 服务，而是会等待系统中必要的落盘工作正常完成。在数据量很大的情况下，这可能会消耗较长时间。 -* TDengine 支持在使用[`systemd`](https://en.wikipedia.org/wiki/Systemd)做进程服务管理的linux系统上安装，用`which systemctl`命令来检测系统中是否存在`systemd`包: +* TDengine 支持在使用 [`systemd`](https://en.wikipedia.org/wiki/Systemd) 做进程服务管理的 linux 系统上安装，用 `which systemctl` 命令来检测系统中是否存在 `systemd` 包: ```bash $ which systemctl ``` - 如果系统中不支持systemd，也可以用手动运行 /usr/local/taos/bin/taosd 方式启动 TDengine 服务。 + 如果系统中不支持 systemd，也可以用手动运行 /usr/local/taos/bin/taosd 方式启动 TDengine 服务。 ## TDengine命令行程序 diff --git a/documentation20/cn/11.administrator/docs.md b/documentation20/cn/11.administrator/docs.md index bfa0456c7d4e80d1fd9336d7c4b7b9ca829b278e..ee2f34ff11ef5e69195c01d0b605383c3257f48c 100644 --- a/documentation20/cn/11.administrator/docs.md +++ b/documentation20/cn/11.administrator/docs.md @@ -129,7 +129,7 @@ taosd -C - blocks：每个VNODE（TSDB）中有多少cache大小的内存块。因此一个VNODE的用的内存大小粗略为（cache * blocks）。单位为块，默认值：4。（可通过 alter database 修改） - replica：副本个数，取值范围：1-3。单位为个，默认值：1。（可通过 alter database 修改） - precision：时间戳精度标识，ms表示毫秒，us表示微秒。默认值：ms。 -- cacheLast：是否在内存中缓存子表 last_row，0：关闭；1：开启。默认值：0。（可通过 alter database 修改）（从 2.0.11 版本开始支持此参数） +- cacheLast：是否在内存中缓存子表的最近数据，0：关闭；1：缓存子表最近一行数据；2：缓存子表每一列的最近的非NULL值，3：同时打开缓存最近行和列功能，默认值：0。（可通过 alter database 修改）（从 2.0.11 版本开始支持此参数） 对于一个应用场景，可能有多种数据特征的数据并存，最佳的设计是将具有相同数据特征的表放在一个库里，这样一个应用有多个库，而每个库可以配置不同的存储参数，从而保证系统有最优的性能。TDengine允许应用在创建库时指定上述存储参数，如果指定，该参数就将覆盖对应的系统配置参数。举例，有下述SQL： diff --git a/documentation20/cn/12.taos-sql/01.error-code/docs.md b/documentation20/cn/12.taos-sql/01.error-code/docs.md index 0a0d694ee3da417ec0162cd9e4f8ddda9ab044f1..867aa18715f87a1dfc9ea36203d32382bb726e30 100644 --- a/documentation20/cn/12.taos-sql/01.error-code/docs.md +++ b/documentation20/cn/12.taos-sql/01.error-code/docs.md @@ -26,7 +26,7 @@ | TSDB_CODE_COM_OUT_OF_MEMORY | 0 | 0x0102 | "Out of memory" | -2147483390 | | TSDB_CODE_COM_INVALID_CFG_MSG | 0 | 0x0103 | "Invalid config message" | -2147483389 | | TSDB_CODE_COM_FILE_CORRUPTED | 0 | 0x0104 | "Data file corrupted" | -2147483388 | -| TSDB_CODE_TSC_INVALID_SQL | 0 | 0x0200 | "Invalid SQL statement" | -2147483136 | +| TSDB_CODE_TSC_INVALID_OPERATION | 0 | 0x0200 | "Invalid SQL statement" | -2147483136 | | TSDB_CODE_TSC_INVALID_QHANDLE | 0 | 0x0201 | "Invalid qhandle" | -2147483135 | | TSDB_CODE_TSC_INVALID_TIME_STAMP | 0 | 0x0202 | "Invalid combination of client/service time" | -2147483134 | | TSDB_CODE_TSC_INVALID_VALUE | 0 | 0x0203 | "Invalid value in client" | -2147483133 | diff --git a/documentation20/cn/12.taos-sql/docs.md b/documentation20/cn/12.taos-sql/docs.md index bcf80d8fa20e613ff8955afa13036687da9d8a59..af78074eb25391d605bfca9067d24e1626296891 100644 --- a/documentation20/cn/12.taos-sql/docs.md +++ b/documentation20/cn/12.taos-sql/docs.md @@ -76,7 +76,7 @@ TDengine 缺省的时间戳是毫秒精度，但通过修改配置参数 enableM 4) 一条SQL 语句的最大长度为65480个字符； - 5) 数据库还有更多与存储相关的配置参数，请参见 [服务端配置](https://www.taosdata.com/cn/documentation/taos-sql#management) 章节。 + 5) 数据库还有更多与存储相关的配置参数，请参见 [服务端配置](https://www.taosdata.com/cn/documentation/administrator#config) 章节。 - **显示系统当前参数** @@ -126,7 +126,7 @@ TDengine 缺省的时间戳是毫秒精度，但通过修改配置参数 enableM ```mysql ALTER DATABASE db_name CACHELAST 0; ``` - CACHELAST 参数控制是否在内存中缓存数据子表的 last_row。缺省值为 0，取值范围 [0, 1]。其中 0 表示不启用、1 表示启用。（从 2.0.11 版本开始支持，修改后需要重启服务器生效。） + CACHELAST 参数控制是否在内存中缓存数据子表的 last_row。缺省值为 0，取值范围 [0, 1]。其中 0 表示不启用、1 表示启用。（从 2.0.11.0 版本开始支持。从 2.1.1.0 版本开始，修改此参数后无需重启服务器即可生效。） **Tips**: 以上所有参数修改后都可以用show databases来确认是否修改成功。 @@ -399,7 +399,12 @@ TDengine 缺省的时间戳是毫秒精度，但通过修改配置参数 enableM INSERT INTO tb1_name (tb1_field1_name, ...) [USING stb1_name TAGS (tag_value1, ...)] VALUES (field1_value1, ...) (field1_value2, ...) ... tb2_name (tb2_field1_name, ...) [USING stb2_name TAGS (tag_value2, ...)] VALUES (field1_value1, ...) (field1_value2, ...) ...; ``` - 以自动建表的方式，同时向表tb1_name和tb2_name中按列分别插入多条记录。 + 以自动建表的方式，同时向表tb1_name和tb2_name中按列分别插入多条记录。 + 从 2.0.20.5 版本开始，子表的列名可以不跟在子表名称后面，而是可以放在 TAGS 和 VALUES 之间，例如像下面这样写： + ```mysql + INSERT INTO tb1_name [USING stb1_name TAGS (tag_value1, ...)] (tb1_field1_name, ...) VALUES (field1_value1, ...) (field1_value2, ...) ...; + ``` + 注意：虽然两种写法都可以，但并不能在一条 SQL 语句中混用，否则会报语法错误。 **历史记录写入**：可使用IMPORT或者INSERT命令，IMPORT的语法，功能与INSERT完全一样。 diff --git a/packaging/tools/makepkg.sh b/packaging/tools/makepkg.sh index d114d5eef810c0590e0d2ae90cfd27e4952f5ddc..e4d2d71b014293e1e328112e0d5e6e5677b772c0 100755 --- a/packaging/tools/makepkg.sh +++ b/packaging/tools/makepkg.sh @@ -114,6 +114,25 @@ mkdir -p ${install_dir}/examples examples_dir="${top_dir}/tests/examples" cp -r ${examples_dir}/c ${install_dir}/examples if [[ "$pagMode" != "lite" ]] && [[ "$cpuType" != "aarch32" ]]; then + if [ -d ${examples_dir}/JDBC/connectionPools/target ]; then + rm -rf ${examples_dir}/JDBC/connectionPools/target + fi + if [ -d ${examples_dir}/JDBC/JDBCDemo/target ]; then + rm -rf ${examples_dir}/JDBC/JDBCDemo/target + fi + if [ -d ${examples_dir}/JDBC/mybatisplus-demo/target ]; then + rm -rf ${examples_dir}/JDBC/mybatisplus-demo/target + fi + if [ -d ${examples_dir}/JDBC/springbootdemo/target ]; then + rm -rf ${examples_dir}/JDBC/springbootdemo/target + fi + if [ -d ${examples_dir}/JDBC/SpringJdbcTemplate/target ]; then + rm -rf ${examples_dir}/JDBC/SpringJdbcTemplate/target + fi + if [ -d ${examples_dir}/JDBC/taosdemo/target ]; then + rm -rf ${examples_dir}/JDBC/taosdemo/target + fi + cp -r ${examples_dir}/JDBC ${install_dir}/examples cp -r ${examples_dir}/matlab ${install_dir}/examples cp -r ${examples_dir}/python ${install_dir}/examples diff --git a/snap/snapcraft.yaml b/snap/snapcraft.yaml index d85c8e5871ba070afc74418b056420134a44f627..a63225ab324cb1eefc9f221423c1018e8e1528eb 100644 --- a/snap/snapcraft.yaml +++ b/snap/snapcraft.yaml @@ -1,6 +1,5 @@ name: tdengine base: core18 - version: '2.1.1.0' icon: snap/gui/t-dengine.svg summary: an open-source big data platform designed and optimized for IoT. diff --git a/src/client/inc/tscLocalMerge.h b/src/client/inc/tscLocalMerge.h index 143922bb1fb6d6e10a157de5d90af2da5e221f76..3c0bde000030000330b33212c7e0c942d50d6a90 100644 --- a/src/client/inc/tscLocalMerge.h +++ b/src/client/inc/tscLocalMerge.h @@ -39,39 +39,29 @@ typedef struct SLocalDataSource { } SLocalDataSource; typedef struct SLocalMerger { - SLocalDataSource ** pLocalDataSrc; + SLocalDataSource **pLocalDataSrc; int32_t numOfBuffer; int32_t numOfCompleted; int32_t numOfVnode; - SLoserTreeInfo * pLoserTree; - tFilePage * pResultBuf; - int32_t nResultBufSize; - tFilePage * pTempBuffer; - struct SQLFunctionCtx *pCtx; - int32_t rowSize; // size of each intermediate result. - tOrderDescriptor * pDesc; - SColumnModel * resColModel; - SColumnModel* finalModel; - tExtMemBuffer ** pExtMemBuffer; // disk-based buffer - bool orderPrjOnSTable; // projection query on stable + SLoserTreeInfo *pLoserTree; + int32_t rowSize; // size of each intermediate result. + tOrderDescriptor *pDesc; + tExtMemBuffer **pExtMemBuffer; // disk-based buffer + char *buf; // temp buffer } SLocalMerger; typedef struct SRetrieveSupport { tExtMemBuffer ** pExtMemBuffer; // for build loser tree tOrderDescriptor *pOrderDescriptor; - SColumnModel* pFinalColModel; // colModel for final result - SColumnModel* pFFColModel; int32_t subqueryIndex; // index of current vnode in vnode list SSqlObj * pParentSql; tFilePage * localBuffer; // temp buffer, there is a buffer for each vnode to uint32_t numOfRetry; // record the number of retry times } SRetrieveSupport; -int32_t tscLocalReducerEnvCreate(SSqlObj *pSql, tExtMemBuffer ***pMemBuffer, tOrderDescriptor **pDesc, - SColumnModel **pFinalModel, SColumnModel** pFFModel, uint32_t nBufferSize); +int32_t tscLocalReducerEnvCreate(SQueryInfo* pQueryInfo, tExtMemBuffer ***pMemBuffer, int32_t numOfSub, tOrderDescriptor **pDesc, uint32_t nBufferSize, int64_t id); -void tscLocalReducerEnvDestroy(tExtMemBuffer **pMemBuffer, tOrderDescriptor *pDesc, SColumnModel *pFinalModel, SColumnModel* pFFModel, - int32_t numOfVnodes); +void tscLocalReducerEnvDestroy(tExtMemBuffer **pMemBuffer, tOrderDescriptor *pDesc, int32_t numOfVnodes); int32_t saveToBuffer(tExtMemBuffer *pMemoryBuf, tOrderDescriptor *pDesc, tFilePage *pPage, void *data, int32_t numOfRows, int32_t orderType); @@ -81,12 +71,10 @@ int32_t tscFlushTmpBuffer(tExtMemBuffer *pMemoryBuf, tOrderDescriptor *pDesc, tF /* * create local reducer to launch the second-stage reduce process at client site */ -void tscCreateLocalMerger(tExtMemBuffer **pMemBuffer, int32_t numOfBuffer, tOrderDescriptor *pDesc, - SColumnModel *finalModel, SColumnModel *pFFModel, SSqlObj* pSql); +int32_t tscCreateLocalMerger(tExtMemBuffer **pMemBuffer, int32_t numOfBuffer, tOrderDescriptor *pDesc, + SQueryInfo *pQueryInfo, SLocalMerger **pMerger, int64_t id); -void tscDestroyLocalMerger(SSqlObj *pSql); - -//int32_t tscDoLocalMerge(SSqlObj *pSql); +void tscDestroyLocalMerger(SLocalMerger* pLocalMerger); #ifdef __cplusplus } diff --git a/src/client/inc/tscUtil.h b/src/client/inc/tscUtil.h index c444456337f3d65dcd4c7c9bf39093eb404e2a3f..34dbd9b3379ab7e3ddd63c78ec889d0d84458979 100644 --- a/src/client/inc/tscUtil.h +++ b/src/client/inc/tscUtil.h @@ -20,6 +20,7 @@ extern "C" { #endif +#include "tsched.h" #include "exception.h" #include "os.h" #include "qExtbuffer.h" @@ -36,6 +37,9 @@ extern "C" { #define UTIL_TABLE_IS_NORMAL_TABLE(metaInfo)\ (!(UTIL_TABLE_IS_SUPER_TABLE(metaInfo) || UTIL_TABLE_IS_CHILD_TABLE(metaInfo))) +#define UTIL_TABLE_IS_TMP_TABLE(metaInfo) \ + (((metaInfo)->pTableMeta != NULL) && ((metaInfo)->pTableMeta->tableType == TSDB_TEMP_TABLE)) + #pragma pack(push,1) // this struct is transfered as binary, padding two bytes to avoid // an 'uid' whose low bytes is 0xff being recoginized as NULL, @@ -59,7 +63,7 @@ typedef struct SJoinSupporter { SArray* exprList; SFieldInfo fieldsInfo; STagCond tagCond; - SSqlGroupbyExpr groupInfo; // group by info + SGroupbyExpr groupInfo; // group by info struct STSBuf* pTSBuf; // the TSBuf struct that holds the compressed timestamp array FILE* f; // temporary file in order to create TSBuf char path[PATH_MAX]; // temporary file path, todo dynamic allocate memory @@ -90,24 +94,14 @@ typedef struct SVgroupTableInfo { SArray *itemList; // SArray } SVgroupTableInfo; -static FORCE_INLINE SQueryInfo* tscGetQueryInfo(SSqlCmd* pCmd, int32_t subClauseIndex) { - assert(pCmd != NULL && subClauseIndex >= 0); - if (pCmd->pQueryInfo == NULL || subClauseIndex >= pCmd->numOfClause) { - return NULL; - } - - return pCmd->pQueryInfo[subClauseIndex]; -} - int32_t converToStr(char *str, int type, void *buf, int32_t bufSize, int32_t *len); -SQueryInfo* tscGetActiveQueryInfo(SSqlCmd* pCmd); - int32_t tscCreateDataBlock(size_t initialSize, int32_t rowSize, int32_t startOffset, SName* name, STableMeta* pTableMeta, STableDataBlocks** dataBlocks); void tscDestroyDataBlock(STableDataBlocks* pDataBlock, bool removeMeta); void tscSortRemoveDataBlockDupRows(STableDataBlocks* dataBuf); void tscDestroyBoundColumnInfo(SParsedDataColInfo* pColInfo); +void doRetrieveSubqueryData(SSchedMsg *pMsg); SParamInfo* tscAddParamToDataBlock(STableDataBlocks* pDataBlock, char type, uint8_t timePrec, int16_t bytes, uint32_t offset); @@ -129,7 +123,7 @@ int32_t tscGetDataBlockFromList(SHashObj* pHashList, int64_t id, int32_t size, i */ bool tscIsPointInterpQuery(SQueryInfo* pQueryInfo); bool tscIsTWAQuery(SQueryInfo* pQueryInfo); -bool tscIsSecondStageQuery(SQueryInfo* pQueryInfo); +bool tsIsArithmeticQueryOnAggResult(SQueryInfo* pQueryInfo); bool tscGroupbyColumn(SQueryInfo* pQueryInfo); bool tscIsTopBotQuery(SQueryInfo* pQueryInfo); bool hasTagValOutput(SQueryInfo* pQueryInfo); @@ -138,13 +132,14 @@ bool isStabledev(SQueryInfo* pQueryInfo); bool isTsCompQuery(SQueryInfo* pQueryInfo); bool isSimpleAggregate(SQueryInfo* pQueryInfo); bool isBlockDistQuery(SQueryInfo* pQueryInfo); -int32_t tscGetTopbotQueryParam(SQueryInfo* pQueryInfo); +bool isSimpleAggregateRv(SQueryInfo* pQueryInfo); bool tscNonOrderedProjectionQueryOnSTable(SQueryInfo *pQueryInfo, int32_t tableIndex); bool tscOrderedProjectionQueryOnSTable(SQueryInfo* pQueryInfo, int32_t tableIndex); bool tscIsProjectionQueryOnSTable(SQueryInfo* pQueryInfo, int32_t tableIndex); bool tscIsProjectionQuery(SQueryInfo* pQueryInfo); +bool tscHasColumnFilter(SQueryInfo* pQueryInfo); bool tscIsTwoStageSTableQuery(SQueryInfo* pQueryInfo, int32_t tableIndex); bool tscQueryTags(SQueryInfo* pQueryInfo); @@ -152,9 +147,9 @@ bool tscMultiRoundQuery(SQueryInfo* pQueryInfo, int32_t tableIndex); bool tscQueryBlockInfo(SQueryInfo* pQueryInfo); SExprInfo* tscAddFuncInSelectClause(SQueryInfo* pQueryInfo, int32_t outputColIndex, int16_t functionId, - SColumnIndex* pIndex, SSchema* pColSchema, int16_t colType); + SColumnIndex* pIndex, SSchema* pColSchema, int16_t colType, int16_t colId); -int32_t tscSetTableFullName(STableMetaInfo* pTableMetaInfo, SStrToken* pzTableName, SSqlObj* pSql); +int32_t tscSetTableFullName(SName* pName, SStrToken* pzTableName, SSqlObj* pSql); void tscClearInterpInfo(SQueryInfo* pQueryInfo); bool tscIsInsertData(char* sqlstr); @@ -173,36 +168,49 @@ void tscFieldInfoUpdateOffset(SQueryInfo* pQueryInfo); int16_t tscFieldInfoGetOffset(SQueryInfo* pQueryInfo, int32_t index); void tscFieldInfoClear(SFieldInfo* pFieldInfo); +void tscFieldInfoCopy(SFieldInfo* pFieldInfo, const SFieldInfo* pSrc, const SArray* pExprList); static FORCE_INLINE int32_t tscNumOfFields(SQueryInfo* pQueryInfo) { return pQueryInfo->fieldsInfo.numOfOutput; } int32_t tscFieldInfoCompare(const SFieldInfo* pFieldInfo1, const SFieldInfo* pFieldInfo2, int32_t *diffSize); -int32_t tscFieldInfoSetSize(const SFieldInfo* pFieldInfo1, const SFieldInfo* pFieldInfo2); +void tscInsertPrimaryTsSourceColumn(SQueryInfo* pQueryInfo, uint64_t uid); +int32_t tscFieldInfoSetSize(const SFieldInfo* pFieldInfo1, const SFieldInfo* pFieldInfo2); void addExprParams(SSqlExpr* pExpr, char* argument, int32_t type, int32_t bytes); int32_t tscGetResRowLength(SArray* pExprList); -SExprInfo* tscSqlExprInsert(SQueryInfo* pQueryInfo, int32_t index, int16_t functionId, SColumnIndex* pColIndex, int16_t type, +SExprInfo* tscExprInsert(SQueryInfo* pQueryInfo, int32_t index, int16_t functionId, SColumnIndex* pColIndex, int16_t type, int16_t size, int16_t resColId, int16_t interSize, bool isTagCol); -SExprInfo* tscSqlExprAppend(SQueryInfo* pQueryInfo, int16_t functionId, SColumnIndex* pColIndex, int16_t type, +SExprInfo* tscExprCreate(SQueryInfo* pQueryInfo, int16_t functionId, SColumnIndex* pColIndex, int16_t type, + int16_t size, int16_t resColId, int16_t interSize, int32_t colType); + +void tscExprAddParams(SSqlExpr* pExpr, char* argument, int32_t type, int32_t bytes); + +SExprInfo* tscExprAppend(SQueryInfo* pQueryInfo, int16_t functionId, SColumnIndex* pColIndex, int16_t type, int16_t size, int16_t resColId, int16_t interSize, bool isTagCol); -SExprInfo* tscSqlExprUpdate(SQueryInfo* pQueryInfo, int32_t index, int16_t functionId, int16_t srcColumnIndex, int16_t type, +SExprInfo* tscExprUpdate(SQueryInfo* pQueryInfo, int32_t index, int16_t functionId, int16_t srcColumnIndex, int16_t type, int16_t size); -size_t tscSqlExprNumOfExprs(SQueryInfo* pQueryInfo); -void tscInsertPrimaryTsSourceColumn(SQueryInfo* pQueryInfo, uint64_t uid); -SExprInfo* tscSqlExprGet(SQueryInfo* pQueryInfo, int32_t index); -int32_t tscSqlExprCopy(SArray* dst, const SArray* src, uint64_t uid, bool deepcopy); -void tscSqlExprAssign(SExprInfo* dst, const SExprInfo* src); -void tscSqlExprInfoDestroy(SArray* pExprInfo); +size_t tscNumOfExprs(SQueryInfo* pQueryInfo); +SExprInfo *tscExprGet(SQueryInfo* pQueryInfo, int32_t index); +int32_t tscExprCopy(SArray* dst, const SArray* src, uint64_t uid, bool deepcopy); +int32_t tscExprCopyAll(SArray* dst, const SArray* src, bool deepcopy); +void tscExprAssign(SExprInfo* dst, const SExprInfo* src); +void tscExprDestroy(SArray* pExprInfo); + +int32_t createProjectionExpr(SQueryInfo* pQueryInfo, STableMetaInfo* pTableMetaInfo, SExprInfo*** pExpr, int32_t* num); + +void clearAllTableMetaInfo(SQueryInfo* pQueryInfo, bool removeMeta); SColumn* tscColumnClone(const SColumn* src); bool tscColumnExists(SArray* pColumnList, int32_t columnIndex, uint64_t uid); SColumn* tscColumnListInsert(SArray* pColumnList, int32_t columnIndex, uint64_t uid, SSchema* pSchema); void tscColumnListDestroy(SArray* pColList); +void tscColumnListCopy(SArray* dst, const SArray* src, uint64_t tableUid); +void tscColumnListCopyAll(SArray* dst, const SArray* src); void convertQueryResult(SSqlRes* pRes, SQueryInfo* pQueryInfo); @@ -224,14 +232,14 @@ void tscGetSrcColumnInfo(SSrcColumnInfo* pColInfo, SQueryInfo* pQueryInfo); bool tscShouldBeFreed(SSqlObj* pSql); -STableMetaInfo* tscGetTableMetaInfoFromCmd(SSqlCmd *pCmd, int32_t subClauseIndex, int32_t tableIndex); +STableMetaInfo* tscGetTableMetaInfoFromCmd(SSqlCmd *pCmd, int32_t tableIndex); STableMetaInfo* tscGetMetaInfo(SQueryInfo *pQueryInfo, int32_t tableIndex); void tscInitQueryInfo(SQueryInfo* pQueryInfo); void tscClearSubqueryInfo(SSqlCmd* pCmd); int32_t tscAddQueryInfo(SSqlCmd *pCmd); -SQueryInfo *tscGetQueryInfo(SSqlCmd* pCmd, int32_t subClauseIndex); -SQueryInfo *tscGetQueryInfoS(SSqlCmd *pCmd, int32_t subClauseIndex); +SQueryInfo *tscGetQueryInfo(SSqlCmd* pCmd); +SQueryInfo *tscGetQueryInfoS(SSqlCmd *pCmd); void tscClearTableMetaInfo(STableMetaInfo* pTableMetaInfo); @@ -245,12 +253,11 @@ SArray* tscVgroupTableInfoDup(SArray* pVgroupTables); void tscRemoveVgroupTableGroup(SArray* pVgroupTable, int32_t index); void tscVgroupTableCopy(SVgroupTableInfo* info, SVgroupTableInfo* pInfo); -int tscGetSTableVgroupInfo(SSqlObj* pSql, int32_t clauseIndex); +int tscGetSTableVgroupInfo(SSqlObj* pSql, SQueryInfo* pQueryInfo); int tscGetTableMeta(SSqlObj* pSql, STableMetaInfo* pTableMetaInfo); int tscGetTableMetaEx(SSqlObj* pSql, STableMetaInfo* pTableMetaInfo, bool createIfNotExists); void tscResetForNextRetrieve(SSqlRes* pRes); -void tscDoQuery(SSqlObj* pSql); void executeQuery(SSqlObj* pSql, SQueryInfo* pQueryInfo); void doExecuteQuery(SSqlObj* pSql, SQueryInfo* pQueryInfo); @@ -281,7 +288,7 @@ void registerSqlObj(SSqlObj* pSql); SSqlObj* createSubqueryObj(SSqlObj* pSql, int16_t tableIndex, __async_cb_func_t fp, void* param, int32_t cmd, SSqlObj* pPrevSql); void addGroupInfoForSubquery(SSqlObj* pParentObj, SSqlObj* pSql, int32_t subClauseIndex, int32_t tableIndex); -void doAddGroupColumnForSubquery(SQueryInfo* pQueryInfo, int32_t tagIndex); +void doAddGroupColumnForSubquery(SQueryInfo* pQueryInfo, int32_t tagIndex, SSqlCmd* pCmd); int16_t tscGetJoinTagColIdByUid(STagCond* pTagCond, uint64_t uid); int16_t tscGetTagColIndexById(STableMeta* pTableMeta, int16_t colId); @@ -297,6 +304,11 @@ void tscTryQueryNextVnode(SSqlObj *pSql, __async_cb_func_t fp); void tscAsyncQuerySingleRowForNextVnode(void *param, TAOS_RES *tres, int numOfRows); void tscTryQueryNextClause(SSqlObj* pSql, __async_cb_func_t fp); int tscSetMgmtEpSetFromCfg(const char *first, const char *second, SRpcCorEpSet *corEpSet); +int32_t getMultiTableMetaFromMnode(SSqlObj *pSql, SArray* pNameList, SArray* pVgroupNameList, __async_cb_func_t fp); + +int tscTransferTableNameList(SSqlObj *pSql, const char *pNameList, int32_t length, SArray* pNameArray); + +bool subAndCheckDone(SSqlObj *pSql, SSqlObj *pParentSql, int idx); bool tscSetSqlOwner(SSqlObj* pSql); void tscClearSqlOwner(SSqlObj* pSql); @@ -311,10 +323,12 @@ CChildTableMeta* tscCreateChildMeta(STableMeta* pTableMeta); uint32_t tscGetTableMetaMaxSize(); int32_t tscCreateTableMetaFromSTableMeta(STableMeta* pChild, const char* name, void* buf); STableMeta* tscTableMetaDup(STableMeta* pTableMeta); +SVgroupsInfo* tscVgroupsInfoDup(SVgroupsInfo* pVgroupsInfo); + int32_t tscCreateQueryFromQueryInfo(SQueryInfo* pQueryInfo, SQueryAttr* pQueryAttr, void* addr); void tsCreateSQLFunctionCtx(SQueryInfo* pQueryInfo, SQLFunctionCtx* pCtx, SSchema* pSchema); -void* createQueryInfoFromQueryNode(SQueryInfo* pQueryInfo, SExprInfo* pExprs, STableGroupInfo* pTableGroupInfo, SOperatorInfo* pOperator, char* sql, void* addr, int32_t stage); +void* createQInfoFromQueryNode(SQueryInfo* pQueryInfo, SExprInfo* pExprs, STableGroupInfo* pTableGroupInfo, SOperatorInfo* pOperator, char* sql, void* addr, int32_t stage); void* malloc_throw(size_t size); void* calloc_throw(size_t nmemb, size_t size); diff --git a/src/client/inc/tsclient.h b/src/client/inc/tsclient.h index 81443360f4a5bbd4f2917cff136fb926eafd009d..c1a85b6d27aebbc34da29e8ea207b2b62c50e728 100644 --- a/src/client/inc/tsclient.h +++ b/src/client/inc/tsclient.h @@ -42,12 +42,6 @@ extern "C" { struct SSqlInfo; struct SLocalMerger; -// data source from sql string or from file -enum { - DATA_FROM_SQL_STRING = 1, - DATA_FROM_DATA_FILE = 2, -}; - typedef void (*__async_cb_func_t)(void *param, TAOS_RES *tres, int32_t numOfRows); typedef struct STableComInfo { @@ -85,10 +79,10 @@ typedef struct STableMeta { } STableMeta; typedef struct STableMetaInfo { - STableMeta *pTableMeta; // table meta, cached in client side and acquired by name + STableMeta *pTableMeta; // table meta, cached in client side and acquired by name uint32_t tableMetaSize; - SVgroupsInfo *vgroupList; - SArray *pVgroupTables; // SArray + SVgroupsInfo *vgroupList; + SArray *pVgroupTables; // SArray /* * 1. keep the vgroup index during the multi-vnode super table projection query @@ -137,8 +131,8 @@ typedef struct SJoinNode { } SJoinNode; typedef struct SJoinInfo { - bool hasJoin; - SJoinNode* joinTables[TSDB_MAX_JOIN_TABLE_NUM]; + bool hasJoin; + SJoinNode *joinTables[TSDB_MAX_JOIN_TABLE_NUM]; } SJoinInfo; typedef struct STagCond { @@ -205,10 +199,11 @@ typedef struct SQueryInfo { SInterval interval; // tumble time window SSessionWindow sessionWindow; // session time window - SSqlGroupbyExpr groupbyExpr; // groupby tags info + SGroupbyExpr groupbyExpr; // groupby tags info SArray * colList; // SArray SFieldInfo fieldsInfo; SArray * exprList; // SArray + SArray * exprList1; // final exprlist in case of arithmetic expression exists SLimitVal limit; SLimitVal slimit; STagCond tagCond; @@ -232,30 +227,50 @@ typedef struct SQueryInfo { int32_t bufLen; char* buf; SQInfo* pQInfo; // global merge operator - SArray* pDSOperator; // data source operator - SArray* pPhyOperator; // physical query execution plan SQueryAttr* pQueryAttr; // query object struct SQueryInfo *sibling; // sibling SArray *pUpstream; // SArray struct SQueryInfo *pDownstream; int32_t havingFieldNum; + bool stableQuery; + bool groupbyColumn; + bool simpleAgg; + bool arithmeticOnAgg; + bool projectionQuery; + bool hasFilter; + bool onlyTagQuery; } SQueryInfo; +typedef struct { + STableMeta *pTableMeta; + SVgroupsInfo *pVgroupInfo; +} STableMetaVgroupInfo; + +typedef struct SInsertStatementParam { + SName **pTableNameList; // all involved tableMeta list of current insert sql statement. + int32_t numOfTables; // number of tables in table name list + SHashObj *pTableBlockHashList; // data block for each table + SArray *pDataBlocks; // SArray. Merged submit block for each vgroup + int8_t schemaAttached; // denote if submit block is built with table schema or not + STagData tagData; // NOTE: pTagData->data is used as a variant length array + + char msg[512]; // error message + char *sql; // current sql statement position + uint32_t insertType; // insert data from [file|sql statement| bound statement] +} SInsertStatementParam; + +// TODO extract sql parser supporter typedef struct { int command; uint8_t msgType; + SInsertStatementParam insertParam; char reserve1[3]; // fix bus error on arm32 - bool autoCreated; // create table if it is not existed during retrieve table meta in mnode - union { int32_t count; - int32_t numOfTablesInSubmit; }; - uint32_t insertType; // TODO remove it char * curSql; // current sql, resume position of sql after parsing paused - int8_t parseFinished; char reserve2[3]; // fix bus error on arm32 int16_t numOfCols; @@ -264,25 +279,13 @@ typedef struct { char * payload; int32_t payloadLen; - SQueryInfo **pQueryInfo; - int32_t numOfClause; - int32_t clauseIndex; // index of multiple subclause query - SQueryInfo *active; // current active query info - - int32_t batchSize; // for parameter ('?') binding and batch processing + SHashObj *pTableMetaMap; // local buffer to keep the queried table meta, before validating the AST + SQueryInfo *pQueryInfo; + SQueryInfo *active; // current active query info + int32_t batchSize; // for parameter ('?') binding and batch processing int32_t numOfParams; - - int8_t dataSourceType; // load data from file or not - char reserve4[3]; // fix bus error on arm32 - int8_t submitSchema; // submit block is built with table schema - char reserve5[3]; // fix bus error on arm32 STagData tagData; // NOTE: pTagData->data is used as a variant length array - - SName **pTableNameList; // all involved tableMeta list of current insert sql statement. - int32_t numOfTables; - - SHashObj *pTableBlockHashList; // data block for each table - SArray *pDataBlocks; // SArray. Merged submit block for each vgroup + int32_t resColumnId; } SSqlCmd; typedef struct SResRec { @@ -443,7 +446,7 @@ int32_t tscCreateResPointerInfo(SSqlRes *pRes, SQueryInfo *pQueryInfo); void tscSetResRawPtr(SSqlRes* pRes, SQueryInfo* pQueryInfo); void tscSetResRawPtrRv(SSqlRes* pRes, SQueryInfo* pQueryInfo, SSDataBlock* pBlock); -void handleDownstreamOperator(SSqlRes* pRes, SQueryInfo* pQueryInfo); +void handleDownstreamOperator(SSqlObj** pSqlList, int32_t numOfUpstream, SQueryInfo* px, SSqlRes* pOutput); void destroyTableNameList(SSqlCmd* pCmd); void tscResetSqlCmd(SSqlCmd *pCmd, bool removeMeta); @@ -489,7 +492,7 @@ char *tscGetErrorMsgPayload(SSqlCmd *pCmd); int32_t tscInvalidSQLErrMsg(char *msg, const char *additionalInfo, const char *sql); int32_t tscSQLSyntaxErrMsg(char* msg, const char* additionalInfo, const char* sql); -int32_t tscToSQLCmd(SSqlObj *pSql, struct SSqlInfo *pInfo); +int32_t tscValidateSqlInfo(SSqlObj *pSql, struct SSqlInfo *pInfo); extern int32_t sentinel; extern SHashObj *tscVgroupMap; @@ -505,7 +508,7 @@ extern int tscNumOfObj; // number of existed sqlObj in current process. extern int (*tscBuildMsg[TSDB_SQL_MAX])(SSqlObj *pSql, SSqlInfo *pInfo); void tscBuildVgroupTableInfo(SSqlObj* pSql, STableMetaInfo* pTableMetaInfo, SArray* tables); -int16_t getNewResColId(SQueryInfo* pQueryInfo); +int16_t getNewResColId(SSqlCmd* pCmd); #ifdef __cplusplus } diff --git a/src/client/src/tscAsync.c b/src/client/src/tscAsync.c index 09b31e4b19c095cb712a4c0f54bb42db34b9949c..47798f4e339b6f84231ba4f83ef7dfe702be1677 100644 --- a/src/client/src/tscAsync.c +++ b/src/client/src/tscAsync.c @@ -59,6 +59,7 @@ void doAsyncQuery(STscObj* pObj, SSqlObj* pSql, __async_cb_func_t fp, void* para tscDebugL("0x%"PRIx64" SQL: %s", pSql->self, pSql->sqlstr); pCmd->curSql = pSql->sqlstr; + pCmd->resColumnId = TSDB_RES_COL_ID; int32_t code = tsParseSql(pSql, true); if (code == TSDB_CODE_TSC_ACTION_IN_PROGRESS) return; @@ -69,7 +70,7 @@ void doAsyncQuery(STscObj* pObj, SSqlObj* pSql, __async_cb_func_t fp, void* para return; } - SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd, pCmd->clauseIndex); + SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd); executeQuery(pSql, pQueryInfo); } @@ -127,7 +128,8 @@ static void tscAsyncFetchRowsProxy(void *param, TAOS_RES *tres, int numOfRows) { * all available virtual node has been checked already, now we need to check * for the next subclause queries */ - if (pCmd->clauseIndex < pCmd->numOfClause - 1) { + if (pCmd->active->sibling != NULL) { + pCmd->active = pCmd->active->sibling; tscTryQueryNextClause(pSql, tscAsyncQueryRowsForNextVnode); return; } @@ -220,6 +222,17 @@ void taos_fetch_rows_a(TAOS_RES *tres, __async_cb_func_t fp, void *param) { tscResetForNextRetrieve(pRes); // handle the sub queries of join query + SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd); + if (pQueryInfo->pUpstream != NULL && taosArrayGetSize(pQueryInfo->pUpstream) > 0) { + SSchedMsg schedMsg = {0}; + schedMsg.fp = doRetrieveSubqueryData; + schedMsg.ahandle = (void *)pSql; + schedMsg.thandle = (void *)1; + schedMsg.msg = 0; + taosScheduleTask(tscQhandle, &schedMsg); + return; + } + if (pCmd->command == TSDB_SQL_TABLE_JOIN_RETRIEVE) { tscFetchDatablockForSubquery(pSql); } else if (pRes->completed) { @@ -231,7 +244,8 @@ void taos_fetch_rows_a(TAOS_RES *tres, __async_cb_func_t fp, void *param) { * all available virtual nodes in current clause has been checked already, now try the * next one in the following union subclause */ - if (pCmd->clauseIndex < pCmd->numOfClause - 1) { + if (pCmd->active->sibling != NULL) { + pCmd->active = pCmd->active->sibling; // todo refactor tscTryQueryNextClause(pSql, tscAsyncQueryRowsForNextVnode); return; } @@ -255,7 +269,7 @@ void taos_fetch_rows_a(TAOS_RES *tres, __async_cb_func_t fp, void *param) { pCmd->command = (pCmd->command > TSDB_SQL_MGMT) ? TSDB_SQL_RETRIEVE : TSDB_SQL_FETCH; } - SQueryInfo* pQueryInfo1 = tscGetActiveQueryInfo(&pSql->cmd); + SQueryInfo* pQueryInfo1 = tscGetQueryInfo(&pSql->cmd); tscBuildAndSendRequest(pSql, pQueryInfo1); } } @@ -317,26 +331,38 @@ static int32_t updateMetaBeforeRetryQuery(SSqlObj* pSql, STableMetaInfo* pTableM // update the pExpr info, colList info, number of table columns // TODO Re-parse this sql and issue the corresponding subquery as an alternative for this case. if (pSql->retryReason == TSDB_CODE_TDB_INVALID_TABLE_ID) { - int32_t numOfExprs = (int32_t) tscSqlExprNumOfExprs(pQueryInfo); + int32_t numOfExprs = (int32_t) tscNumOfExprs(pQueryInfo); int32_t numOfCols = tscGetNumOfColumns(pTableMetaInfo->pTableMeta); int32_t numOfTags = tscGetNumOfTags(pTableMetaInfo->pTableMeta); SSchema *pSchema = tscGetTableSchema(pTableMetaInfo->pTableMeta); + SSchema *pTagSchema = tscGetTableTagSchema(pTableMetaInfo->pTableMeta); + for (int32_t i = 0; i < numOfExprs; ++i) { - SSqlExpr *pExpr = &(tscSqlExprGet(pQueryInfo, i)->base); + SSqlExpr *pExpr = &(tscExprGet(pQueryInfo, i)->base); + + // update the table uid pExpr->uid = pTableMetaInfo->pTableMeta->id.uid; if (pExpr->colInfo.colIndex >= 0) { int32_t index = pExpr->colInfo.colIndex; if ((TSDB_COL_IS_NORMAL_COL(pExpr->colInfo.flag) && index >= numOfCols) || - (TSDB_COL_IS_TAG(pExpr->colInfo.flag) && (index < numOfCols || index >= (numOfCols + numOfTags)))) { + (TSDB_COL_IS_TAG(pExpr->colInfo.flag) && (index < 0 || index >= numOfTags))) { return pSql->retryReason; } - if ((pSchema[pExpr->colInfo.colIndex].colId != pExpr->colInfo.colId) && - strcasecmp(pExpr->colInfo.name, pSchema[pExpr->colInfo.colIndex].name) != 0) { - return pSql->retryReason; + if (TSDB_COL_IS_TAG(pExpr->colInfo.flag)) { + if ((pTagSchema[pExpr->colInfo.colIndex].colId != pExpr->colInfo.colId) && + strcasecmp(pExpr->colInfo.name, pTagSchema[pExpr->colInfo.colIndex].name) != 0) { + return pSql->retryReason; + } + } else if (TSDB_COL_IS_NORMAL_COL(pExpr->colInfo.flag)) { + if ((pSchema[pExpr->colInfo.colIndex].colId != pExpr->colInfo.colId) && + strcasecmp(pExpr->colInfo.name, pSchema[pExpr->colInfo.colIndex].name) != 0) { + return pSql->retryReason; + } + } else { // do nothing for udc } } } @@ -374,12 +400,12 @@ void tscTableMetaCallBack(void *param, TAOS_RES *res, int code) { tscDebug("0x%"PRIx64" get %s successfully", pSql->self, msg); if (pSql->pStream == NULL) { - SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd, pCmd->clauseIndex); + SQueryInfo *pQueryInfo = tscGetQueryInfo(pCmd); // check if it is a sub-query of super table query first, if true, enter another routine - if (TSDB_QUERY_HAS_TYPE(pQueryInfo->type, (TSDB_QUERY_TYPE_STABLE_SUBQUERY|TSDB_QUERY_TYPE_SUBQUERY|TSDB_QUERY_TYPE_TAG_FILTER_QUERY))) { - tscDebug("0x%"PRIx64" update local table meta, continue to process sql and send the corresponding query", pSql->self); - + if (TSDB_QUERY_HAS_TYPE(pQueryInfo->type, (TSDB_QUERY_TYPE_STABLE_SUBQUERY | TSDB_QUERY_TYPE_SUBQUERY | + TSDB_QUERY_TYPE_TAG_FILTER_QUERY))) { + tscDebug("0x%" PRIx64 " update cached table-meta, continue to process sql and send the corresponding query", pSql->self); STableMetaInfo *pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0); code = tscGetTableMeta(pSql, pTableMetaInfo); @@ -401,42 +427,8 @@ void tscTableMetaCallBack(void *param, TAOS_RES *res, int code) { taosReleaseRef(tscObjRef, pSql->self); return; } else { // continue to process normal async query - if (pCmd->parseFinished) { - tscDebug("0x%"PRIx64" update local table meta, continue to process sql and send corresponding query", pSql->self); - - STableMetaInfo* pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, pCmd->clauseIndex, 0); - code = tscGetTableMeta(pSql, pTableMetaInfo); - - assert(code == TSDB_CODE_TSC_ACTION_IN_PROGRESS || code == TSDB_CODE_SUCCESS); - if (code == TSDB_CODE_TSC_ACTION_IN_PROGRESS) { - taosReleaseRef(tscObjRef, pSql->self); - return; - } - - assert(pCmd->command != TSDB_SQL_INSERT); - - if (pCmd->command == TSDB_SQL_SELECT) { - tscDebug("0x%"PRIx64" redo parse sql string and proceed", pSql->self); - pCmd->parseFinished = false; - tscResetSqlCmd(pCmd, true); - - code = tsParseSql(pSql, true); - if (code == TSDB_CODE_TSC_ACTION_IN_PROGRESS) { - taosReleaseRef(tscObjRef, pSql->self); - return; - } else if (code != TSDB_CODE_SUCCESS) { - goto _error; - } - - tscBuildAndSendRequest(pSql, NULL); - } else { // in all other cases, simple retry - tscBuildAndSendRequest(pSql, NULL); - } - - taosReleaseRef(tscObjRef, pSql->self); - return; - } else { - tscDebug("0x%"PRIx64" continue parse sql after get table meta", pSql->self); + if (TSDB_QUERY_HAS_TYPE(pQueryInfo->type, TSDB_QUERY_TYPE_INSERT)) { + tscDebug("0x%" PRIx64 " continue parse sql after get table-meta", pSql->self); code = tsParseSql(pSql, false); if (code == TSDB_CODE_TSC_ACTION_IN_PROGRESS) { @@ -446,8 +438,8 @@ void tscTableMetaCallBack(void *param, TAOS_RES *res, int code) { goto _error; } - if (pCmd->insertType == TSDB_QUERY_TYPE_STMT_INSERT) { - STableMetaInfo *pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, pCmd->clauseIndex, 0); + if (TSDB_QUERY_HAS_TYPE(pCmd->insertParam.insertType, TSDB_QUERY_TYPE_STMT_INSERT)) { + STableMetaInfo *pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0); code = tscGetTableMeta(pSql, pTableMetaInfo); if (code == TSDB_CODE_TSC_ACTION_IN_PROGRESS) { taosReleaseRef(tscObjRef, pSql->self); @@ -457,59 +449,52 @@ void tscTableMetaCallBack(void *param, TAOS_RES *res, int code) { } (*pSql->fp)(pSql->param, pSql, code); - } else if (TSDB_QUERY_HAS_TYPE(pQueryInfo->type, TSDB_QUERY_TYPE_INSERT)) { - if (pCmd->dataSourceType == DATA_FROM_DATA_FILE) { + } else { + if (TSDB_QUERY_HAS_TYPE(pCmd->insertParam.insertType, TSDB_QUERY_TYPE_FILE_INSERT)) { tscImportDataFromFile(pSql); } else { tscHandleMultivnodeInsert(pSql); } + } + } else { + if (pSql->retryReason != TSDB_CODE_SUCCESS) { + tscDebug("0x%" PRIx64 " update cached table-meta, re-validate sql statement and send query again", + pSql->self); + tscResetSqlCmd(pCmd, false); + pSql->retryReason = TSDB_CODE_SUCCESS; } else { - SQueryInfo* pQueryInfo1 = tscGetQueryInfo(pCmd, pCmd->clauseIndex); - executeQuery(pSql, pQueryInfo1); + tscDebug("0x%" PRIx64 " cached table-meta, continue validate sql statement and send query", pSql->self); } - taosReleaseRef(tscObjRef, pSql->self); - return; - } - } + code = tsParseSql(pSql, true); + if (code == TSDB_CODE_TSC_ACTION_IN_PROGRESS) { + taosReleaseRef(tscObjRef, pSql->self); + return; + } else if (code != TSDB_CODE_SUCCESS) { + goto _error; + } - } else { // stream computing - STableMetaInfo *pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, pCmd->clauseIndex, 0); + SQueryInfo *pQueryInfo1 = tscGetQueryInfo(pCmd); + executeQuery(pSql, pQueryInfo1); + } - code = tscGetTableMeta(pSql, pTableMetaInfo); - if (code == TSDB_CODE_TSC_ACTION_IN_PROGRESS) { taosReleaseRef(tscObjRef, pSql->self); return; - } else if (code != TSDB_CODE_SUCCESS) { - goto _error; - } - - if (UTIL_TABLE_IS_SUPER_TABLE(pTableMetaInfo)) { - code = tscGetSTableVgroupInfo(pSql, pCmd->clauseIndex); - if (code == TSDB_CODE_TSC_ACTION_IN_PROGRESS) { - taosReleaseRef(tscObjRef, pSql->self); - return; - } else if (code != TSDB_CODE_SUCCESS) { - goto _error; - } } + } else { // stream computing + tscDebug("0x%"PRIx64" stream:%p meta is updated, start new query, command:%d", pSql->self, pSql->pStream, pCmd->command); - tscDebug("0x%"PRIx64" stream:%p meta is updated, start new query, command:%d", pSql->self, pSql->pStream, pSql->cmd.command); - if (!pSql->cmd.parseFinished) { + SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd); + if (tscNumOfExprs(pQueryInfo) == 0) { tsParseSql(pSql, false); } (*pSql->fp)(pSql->param, pSql, code); - taosReleaseRef(tscObjRef, pSql->self); - return; } -// tscDoQuery(pSql); - taosReleaseRef(tscObjRef, pSql->self); - return; _error: diff --git a/src/client/src/tscLocal.c b/src/client/src/tscLocal.c index 1be5e29230d6234015b8d443558bd9d3dd85ba7c..abac2407fb7019dd298468a20838fbad680ea7ec 100644 --- a/src/client/src/tscLocal.c +++ b/src/client/src/tscLocal.c @@ -53,7 +53,7 @@ static int32_t tscSetValueToResObj(SSqlObj *pSql, int32_t rowLen) { SSqlRes *pRes = &pSql->res; // one column for each row - SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd, 0); + SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd); STableMetaInfo *pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0); STableMeta * pMeta = pTableMetaInfo->pTableMeta; @@ -154,14 +154,14 @@ static int32_t tscBuildTableSchemaResultFields(SSqlObj *pSql, int32_t numOfCols, pSql->cmd.numOfCols = numOfCols; - SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd, 0); + SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd); pQueryInfo->order.order = TSDB_ORDER_ASC; TAOS_FIELD f = {.type = TSDB_DATA_TYPE_BINARY, .bytes = (TSDB_COL_NAME_LEN - 1) + VARSTR_HEADER_SIZE}; tstrncpy(f.name, "Field", sizeof(f.name)); SInternalField* pInfo = tscFieldInfoAppend(&pQueryInfo->fieldsInfo, &f); - pInfo->pExpr = tscSqlExprAppend(pQueryInfo, TSDB_FUNC_TS_DUMMY, &index, TSDB_DATA_TYPE_BINARY, + pInfo->pExpr = tscExprAppend(pQueryInfo, TSDB_FUNC_TS_DUMMY, &index, TSDB_DATA_TYPE_BINARY, (TSDB_COL_NAME_LEN - 1) + VARSTR_HEADER_SIZE, -1000, (TSDB_COL_NAME_LEN - 1), false); rowLen += ((TSDB_COL_NAME_LEN - 1) + VARSTR_HEADER_SIZE); @@ -171,7 +171,7 @@ static int32_t tscBuildTableSchemaResultFields(SSqlObj *pSql, int32_t numOfCols, tstrncpy(f.name, "Type", sizeof(f.name)); pInfo = tscFieldInfoAppend(&pQueryInfo->fieldsInfo, &f); - pInfo->pExpr = tscSqlExprAppend(pQueryInfo, TSDB_FUNC_TS_DUMMY, &index, TSDB_DATA_TYPE_BINARY, (int16_t)(typeColLength + VARSTR_HEADER_SIZE), + pInfo->pExpr = tscExprAppend(pQueryInfo, TSDB_FUNC_TS_DUMMY, &index, TSDB_DATA_TYPE_BINARY, (int16_t)(typeColLength + VARSTR_HEADER_SIZE), -1000, typeColLength, false); rowLen += typeColLength + VARSTR_HEADER_SIZE; @@ -181,7 +181,7 @@ static int32_t tscBuildTableSchemaResultFields(SSqlObj *pSql, int32_t numOfCols, tstrncpy(f.name, "Length", sizeof(f.name)); pInfo = tscFieldInfoAppend(&pQueryInfo->fieldsInfo, &f); - pInfo->pExpr = tscSqlExprAppend(pQueryInfo, TSDB_FUNC_TS_DUMMY, &index, TSDB_DATA_TYPE_INT, sizeof(int32_t), + pInfo->pExpr = tscExprAppend(pQueryInfo, TSDB_FUNC_TS_DUMMY, &index, TSDB_DATA_TYPE_INT, sizeof(int32_t), -1000, sizeof(int32_t), false); rowLen += sizeof(int32_t); @@ -191,7 +191,7 @@ static int32_t tscBuildTableSchemaResultFields(SSqlObj *pSql, int32_t numOfCols, tstrncpy(f.name, "Note", sizeof(f.name)); pInfo = tscFieldInfoAppend(&pQueryInfo->fieldsInfo, &f); - pInfo->pExpr = tscSqlExprAppend(pQueryInfo, TSDB_FUNC_TS_DUMMY, &index, TSDB_DATA_TYPE_BINARY, (int16_t)(noteColLength + VARSTR_HEADER_SIZE), + pInfo->pExpr = tscExprAppend(pQueryInfo, TSDB_FUNC_TS_DUMMY, &index, TSDB_DATA_TYPE_BINARY, (int16_t)(noteColLength + VARSTR_HEADER_SIZE), -1000, noteColLength, false); rowLen += noteColLength + VARSTR_HEADER_SIZE; @@ -199,7 +199,7 @@ static int32_t tscBuildTableSchemaResultFields(SSqlObj *pSql, int32_t numOfCols, } static int32_t tscProcessDescribeTable(SSqlObj *pSql) { - SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd, 0); + SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd); assert(tscGetMetaInfo(pQueryInfo, 0)->pTableMeta != NULL); @@ -390,7 +390,7 @@ static int32_t tscSCreateBuildResultFields(SSqlObj *pSql, BuildType type, const SColumnIndex index = {0}; pSql->cmd.numOfCols = 2; - SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd, 0); + SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd); pQueryInfo->order.order = TSDB_ORDER_ASC; TAOS_FIELD f; @@ -405,7 +405,7 @@ static int32_t tscSCreateBuildResultFields(SSqlObj *pSql, BuildType type, const } SInternalField* pInfo = tscFieldInfoAppend(&pQueryInfo->fieldsInfo, &f); - pInfo->pExpr = tscSqlExprAppend(pQueryInfo, TSDB_FUNC_TS_DUMMY, &index, TSDB_DATA_TYPE_BINARY, f.bytes, -1000, f.bytes - VARSTR_HEADER_SIZE, false); + pInfo->pExpr = tscExprAppend(pQueryInfo, TSDB_FUNC_TS_DUMMY, &index, TSDB_DATA_TYPE_BINARY, f.bytes, -1000, f.bytes - VARSTR_HEADER_SIZE, false); rowLen += f.bytes; @@ -418,7 +418,7 @@ static int32_t tscSCreateBuildResultFields(SSqlObj *pSql, BuildType type, const } pInfo = tscFieldInfoAppend(&pQueryInfo->fieldsInfo, &f); - pInfo->pExpr = tscSqlExprAppend(pQueryInfo, TSDB_FUNC_TS_DUMMY, &index, TSDB_DATA_TYPE_BINARY, + pInfo->pExpr = tscExprAppend(pQueryInfo, TSDB_FUNC_TS_DUMMY, &index, TSDB_DATA_TYPE_BINARY, (int16_t)(ddlLen + VARSTR_HEADER_SIZE), -1000, ddlLen, false); rowLen += ddlLen + VARSTR_HEADER_SIZE; @@ -428,7 +428,7 @@ static int32_t tscSCreateBuildResultFields(SSqlObj *pSql, BuildType type, const static int32_t tscSCreateSetValueToResObj(SSqlObj *pSql, int32_t rowLen, const char *tableName, const char *ddl) { SSqlRes *pRes = &pSql->res; - SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd, 0); + SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd); int32_t numOfRows = 1; if (strlen(ddl) == 0) { @@ -445,7 +445,7 @@ static int32_t tscSCreateSetValueToResObj(SSqlObj *pSql, int32_t rowLen, const c return 0; } static int32_t tscSCreateBuildResult(SSqlObj *pSql, BuildType type, const char *str, const char *result) { - SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd, 0); + SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd); int32_t rowLen = tscSCreateBuildResultFields(pSql, type, result); tscFieldInfoUpdateOffset(pQueryInfo); @@ -532,7 +532,7 @@ static int32_t tscGetTableTagColumnName(SSqlObj *pSql, char **result) { } buf[0] = 0; - STableMeta *pMeta = tscGetTableMetaInfoFromCmd(&pSql->cmd, 0, 0)->pTableMeta; + STableMeta *pMeta = tscGetTableMetaInfoFromCmd(&pSql->cmd, 0)->pTableMeta; if (pMeta->tableType == TSDB_SUPER_TABLE || pMeta->tableType == TSDB_NORMAL_TABLE || pMeta->tableType == TSDB_STREAM_TABLE) { free(buf); @@ -553,7 +553,7 @@ static int32_t tscGetTableTagColumnName(SSqlObj *pSql, char **result) { return TSDB_CODE_SUCCESS; } static int32_t tscRebuildDDLForSubTable(SSqlObj *pSql, const char *tableName, char *ddl) { - SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd, 0); + SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd); STableMetaInfo *pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0); STableMeta * pMeta = pTableMetaInfo->pTableMeta; @@ -607,7 +607,7 @@ static int32_t tscRebuildDDLForSubTable(SSqlObj *pSql, const char *tableName, ch } static int32_t tscRebuildDDLForNormalTable(SSqlObj *pSql, const char *tableName, char *ddl) { - SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd, 0); + SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd); STableMetaInfo *pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0); STableMeta * pMeta = pTableMetaInfo->pTableMeta; @@ -634,7 +634,7 @@ static int32_t tscRebuildDDLForNormalTable(SSqlObj *pSql, const char *tableName, } static int32_t tscRebuildDDLForSuperTable(SSqlObj *pSql, const char *tableName, char *ddl) { char *result = ddl; - SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd, 0); + SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd); STableMetaInfo *pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0); STableMeta * pMeta = pTableMetaInfo->pTableMeta; @@ -675,7 +675,7 @@ static int32_t tscRebuildDDLForSuperTable(SSqlObj *pSql, const char *tableName, } static int32_t tscProcessShowCreateTable(SSqlObj *pSql) { - SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd, 0); + SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd); STableMetaInfo *pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0); assert(pTableMetaInfo->pTableMeta != NULL); @@ -704,7 +704,7 @@ static int32_t tscProcessShowCreateTable(SSqlObj *pSql) { } static int32_t tscProcessShowCreateDatabase(SSqlObj *pSql) { - SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd, 0); + SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd); STableMetaInfo *pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0); @@ -730,7 +730,7 @@ static int32_t tscProcessShowCreateDatabase(SSqlObj *pSql) { return TSDB_CODE_TSC_ACTION_IN_PROGRESS; } static int32_t tscProcessCurrentUser(SSqlObj *pSql) { - SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd, 0); + SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd); SSqlExpr* pExpr = taosArrayGetP(pQueryInfo->exprList, 0); pExpr->resBytes = TSDB_USER_LEN + TSDB_DATA_TYPE_BINARY; @@ -757,7 +757,7 @@ static int32_t tscProcessCurrentDB(SSqlObj *pSql) { extractDBName(pSql->pTscObj->db, db); pthread_mutex_unlock(&pSql->pTscObj->mutex); - SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd, pSql->cmd.clauseIndex); + SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd); SSqlExpr* pExpr = taosArrayGetP(pQueryInfo->exprList, 0); pExpr->resType = TSDB_DATA_TYPE_BINARY; @@ -784,7 +784,7 @@ static int32_t tscProcessCurrentDB(SSqlObj *pSql) { static int32_t tscProcessServerVer(SSqlObj *pSql) { const char* v = pSql->pTscObj->sversion; - SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd, pSql->cmd.clauseIndex); + SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd); SSqlExpr* pExpr = taosArrayGetP(pQueryInfo->exprList, 0); pExpr->resType = TSDB_DATA_TYPE_BINARY; @@ -807,7 +807,7 @@ static int32_t tscProcessServerVer(SSqlObj *pSql) { } static int32_t tscProcessClientVer(SSqlObj *pSql) { - SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd, 0); + SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd); SSqlExpr* pExpr = taosArrayGetP(pQueryInfo->exprList, 0); pExpr->resType = TSDB_DATA_TYPE_BINARY; @@ -859,7 +859,7 @@ static int32_t tscProcessServStatus(SSqlObj *pSql) { return pSql->res.code; } - SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd, 0); + SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd); SSqlExpr* pExpr = taosArrayGetP(pQueryInfo->exprList, 0); int32_t val = 1; @@ -873,7 +873,7 @@ void tscSetLocalQueryResult(SSqlObj *pSql, const char *val, const char *columnNa pCmd->numOfCols = 1; - SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd, pCmd->clauseIndex); + SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd); pQueryInfo->order.order = TSDB_ORDER_ASC; tscFieldInfoClear(&pQueryInfo->fieldsInfo); @@ -928,7 +928,7 @@ int tscProcessLocalCmd(SSqlObj *pSql) { } else if (pCmd->command == TSDB_SQL_SERV_STATUS) { pRes->code = tscProcessServStatus(pSql); } else { - pRes->code = TSDB_CODE_TSC_INVALID_SQL; + pRes->code = TSDB_CODE_TSC_INVALID_OPERATION; tscError("0x%"PRIx64" not support command:%d", pSql->self, pCmd->command); } diff --git a/src/client/src/tscLocalMerge.c b/src/client/src/tscLocalMerge.c index e4a3ace6b5af128971a5bed21562082602754d97..77a4c7fb464652d8b92b08e4877b6601afb8b529 100644 --- a/src/client/src/tscLocalMerge.c +++ b/src/client/src/tscLocalMerge.c @@ -19,8 +19,6 @@ #include "texpr.h" #include "tlosertree.h" #include "tscLog.h" -#include "tscUtil.h" -#include "tschemautil.h" #include "tsclient.h" #include "qUtil.h" @@ -59,77 +57,25 @@ int32_t treeComparator(const void *pLeft, const void *pRight, void *param) { } } -// todo merge with vnode side function -void tsCreateSQLFunctionCtx(SQueryInfo* pQueryInfo, SQLFunctionCtx* pCtx, SSchema* pSchema) { - size_t size = tscSqlExprNumOfExprs(pQueryInfo); - - for (int32_t i = 0; i < size; ++i) { - SExprInfo *pExpr = tscSqlExprGet(pQueryInfo, i); - - pCtx[i].order = pQueryInfo->order.order; - pCtx[i].functionId = pExpr->base.functionId; - - pCtx[i].order = pQueryInfo->order.order; - pCtx[i].functionId = pExpr->base.functionId; - - // input data format comes from pModel - pCtx[i].inputType = pSchema[i].type; - pCtx[i].inputBytes = pSchema[i].bytes; - - pCtx[i].outputBytes = pExpr->base.resBytes; - pCtx[i].outputType = pExpr->base.resType; - - // input buffer hold only one point data - pCtx[i].size = 1; - pCtx[i].hasNull = true; - pCtx[i].currentStage = MERGE_STAGE; - - // for top/bottom function, the output of timestamp is the first column - int32_t functionId = pExpr->base.functionId; - if (functionId == TSDB_FUNC_TOP || functionId == TSDB_FUNC_BOTTOM || functionId == TSDB_FUNC_DIFF) { - pCtx[i].ptsOutputBuf = pCtx[0].pOutput; - pCtx[i].param[2].i64 = pQueryInfo->order.order; - pCtx[i].param[2].nType = TSDB_DATA_TYPE_BIGINT; - pCtx[i].param[1].i64 = pQueryInfo->order.orderColId; - pCtx[i].param[0].i64 = pExpr->base.param[0].i64; // top/bot parameter - } else if (functionId == TSDB_FUNC_APERCT) { - pCtx[i].param[0].i64 = pExpr->base.param[0].i64; - pCtx[i].param[0].nType = pExpr->base.param[0].nType; - } else if (functionId == TSDB_FUNC_BLKINFO) { - pCtx[i].param[0].i64 = pExpr->base.param[0].i64; - pCtx[i].param[0].nType = pExpr->base.param[0].nType; - pCtx[i].numOfParams = 1; - } - - pCtx[i].interBufBytes = pExpr->base.interBytes; - pCtx[i].stableQuery = true; - } -} - -void tscCreateLocalMerger(tExtMemBuffer **pMemBuffer, int32_t numOfBuffer, tOrderDescriptor *pDesc, - SColumnModel *finalmodel, SColumnModel *pFFModel, SSqlObj *pSql) { - SSqlCmd* pCmd = &pSql->cmd; - SSqlRes* pRes = &pSql->res; - +int32_t tscCreateLocalMerger(tExtMemBuffer **pMemBuffer, int32_t numOfBuffer, tOrderDescriptor *pDesc, + SQueryInfo* pQueryInfo, SLocalMerger **pMerger, int64_t id) { if (pMemBuffer == NULL) { - tscLocalReducerEnvDestroy(pMemBuffer, pDesc, finalmodel, pFFModel, numOfBuffer); - tscError("pMemBuffer:%p is NULL", pMemBuffer); - pRes->code = TSDB_CODE_TSC_APP_ERROR; - return; + tscLocalReducerEnvDestroy(pMemBuffer, pDesc, numOfBuffer); + tscError("0x%"PRIx64" %p pMemBuffer is NULL", id, pMemBuffer); + return TSDB_CODE_TSC_APP_ERROR; } if (pDesc->pColumnModel == NULL) { - tscLocalReducerEnvDestroy(pMemBuffer, pDesc, finalmodel, pFFModel, numOfBuffer); - tscError("0x%"PRIx64" no local buffer or intermediate result format model", pSql->self); - pRes->code = TSDB_CODE_TSC_APP_ERROR; - return; + tscLocalReducerEnvDestroy(pMemBuffer, pDesc, numOfBuffer); + tscError("0x%"PRIx64" no local buffer or intermediate result format model", id); + return TSDB_CODE_TSC_APP_ERROR; } int32_t numOfFlush = 0; for (int32_t i = 0; i < numOfBuffer; ++i) { int32_t len = pMemBuffer[i]->fileMeta.flushoutData.nLength; if (len == 0) { - tscDebug("0x%"PRIx64" no data retrieved from orderOfVnode:%d", pSql->self, i + 1); + tscDebug("0x%"PRIx64" no data retrieved from orderOfVnode:%d", id, i + 1); continue; } @@ -137,41 +83,36 @@ void tscCreateLocalMerger(tExtMemBuffer **pMemBuffer, int32_t numOfBuffer, tOrde } if (numOfFlush == 0 || numOfBuffer == 0) { - tscLocalReducerEnvDestroy(pMemBuffer, pDesc, finalmodel, pFFModel, numOfBuffer); - pCmd->command = TSDB_SQL_RETRIEVE_EMPTY_RESULT; // no result, set the result empty - tscDebug("0x%"PRIx64" retrieved no data", pSql->self); - return; + tscLocalReducerEnvDestroy(pMemBuffer, pDesc, numOfBuffer); + tscDebug("0x%"PRIx64" no data to retrieve", id); + return TSDB_CODE_SUCCESS; } if (pDesc->pColumnModel->capacity >= pMemBuffer[0]->pageSize) { - tscError("0x%"PRIx64" Invalid value of buffer capacity %d and page size %d ", pSql->self, pDesc->pColumnModel->capacity, + tscError("0x%"PRIx64" Invalid value of buffer capacity %d and page size %d ", id, pDesc->pColumnModel->capacity, pMemBuffer[0]->pageSize); - tscLocalReducerEnvDestroy(pMemBuffer, pDesc, finalmodel, pFFModel, numOfBuffer); - pRes->code = TSDB_CODE_TSC_APP_ERROR; - return; + tscLocalReducerEnvDestroy(pMemBuffer, pDesc, numOfBuffer); + return TSDB_CODE_TSC_APP_ERROR; } - size_t size = sizeof(SLocalMerger) + POINTER_BYTES * numOfFlush; - - SLocalMerger *pMerger = (SLocalMerger *) calloc(1, size); - if (pMerger == NULL) { - tscError("0x%"PRIx64" failed to create local merge structure, out of memory", pSql->self); + *pMerger = (SLocalMerger *) calloc(1, sizeof(SLocalMerger)); + if ((*pMerger) == NULL) { + tscError("0x%"PRIx64" failed to create local merge structure, out of memory", id); - tscLocalReducerEnvDestroy(pMemBuffer, pDesc, finalmodel, pFFModel, numOfBuffer); - pRes->code = TSDB_CODE_TSC_OUT_OF_MEMORY; - return; + tscLocalReducerEnvDestroy(pMemBuffer, pDesc, numOfBuffer); + return TSDB_CODE_TSC_OUT_OF_MEMORY; } - pMerger->pExtMemBuffer = pMemBuffer; - pMerger->pLocalDataSrc = (SLocalDataSource **)&pMerger[1]; - assert(pMerger->pLocalDataSrc != NULL); + (*pMerger)->pExtMemBuffer = pMemBuffer; + (*pMerger)->pLocalDataSrc = calloc(numOfFlush, POINTER_BYTES); + assert((*pMerger)->pLocalDataSrc != NULL); - pMerger->numOfBuffer = numOfFlush; - pMerger->numOfVnode = numOfBuffer; + (*pMerger)->numOfBuffer = numOfFlush; + (*pMerger)->numOfVnode = numOfBuffer; - pMerger->pDesc = pDesc; - tscDebug("0x%"PRIx64" the number of merged leaves is: %d", pSql->self, pMerger->numOfBuffer); + (*pMerger)->pDesc = pDesc; + tscDebug("0x%"PRIx64" the number of merged leaves is: %d", id, (*pMerger)->numOfBuffer); int32_t idx = 0; for (int32_t i = 0; i < numOfBuffer; ++i) { @@ -180,13 +121,12 @@ void tscCreateLocalMerger(tExtMemBuffer **pMemBuffer, int32_t numOfBuffer, tOrde for (int32_t j = 0; j < numOfFlushoutInFile; ++j) { SLocalDataSource *ds = (SLocalDataSource *)malloc(sizeof(SLocalDataSource) + pMemBuffer[0]->pageSize); if (ds == NULL) { - tscError("0x%"PRIx64" failed to create merge structure", pSql->self); - pRes->code = TSDB_CODE_TSC_OUT_OF_MEMORY; + tscError("0x%"PRIx64" failed to create merge structure", id); tfree(pMerger); - return; + return TSDB_CODE_TSC_OUT_OF_MEMORY; } - pMerger->pLocalDataSrc[idx] = ds; + (*pMerger)->pLocalDataSrc[idx] = ds; ds->pMemBuffer = pMemBuffer[i]; ds->flushoutIdx = j; @@ -194,12 +134,12 @@ void tscCreateLocalMerger(tExtMemBuffer **pMemBuffer, int32_t numOfBuffer, tOrde ds->pageId = 0; ds->rowIdx = 0; - tscDebug("0x%"PRIx64" load data from disk into memory, orderOfVnode:%d, total:%d", pSql->self, i + 1, idx + 1); + tscDebug("0x%"PRIx64" load data from disk into memory, orderOfVnode:%d, total:%d", id, i + 1, idx + 1); tExtMemBufferLoadData(pMemBuffer[i], &(ds->filePage), j, 0); #ifdef _DEBUG_VIEW printf("load data page into mem for build loser tree: %" PRIu64 " rows\n", ds->filePage.num); SSrcColumnInfo colInfo[256] = {0}; - SQueryInfo * pQueryInfo = tscGetQueryInfo(pCmd, pCmd->clauseIndex); + SQueryInfo * pQueryInfo = tscGetQueryInfo(pCmd); tscGetSrcColumnInfo(colInfo, pQueryInfo); @@ -208,7 +148,7 @@ void tscCreateLocalMerger(tExtMemBuffer **pMemBuffer, int32_t numOfBuffer, tOrde #endif if (ds->filePage.num == 0) { // no data in this flush, the index does not increase - tscDebug("0x%"PRIx64" flush data is empty, ignore %d flush record", pSql->self, idx); + tscDebug("0x%"PRIx64" flush data is empty, ignore %d flush record", id, idx); tfree(ds); continue; } @@ -219,115 +159,54 @@ void tscCreateLocalMerger(tExtMemBuffer **pMemBuffer, int32_t numOfBuffer, tOrde // no data actually, no need to merge result. if (idx == 0) { - tfree(pMerger); - return; + tscDebug("0x%"PRIx64" retrieved no data", id); + tscLocalReducerEnvDestroy(pMemBuffer, pDesc, numOfBuffer); + return TSDB_CODE_SUCCESS; } - pMerger->numOfBuffer = idx; + (*pMerger)->numOfBuffer = idx; SCompareParam *param = malloc(sizeof(SCompareParam)); if (param == NULL) { - tfree(pMerger); - return; + tfree((*pMerger)); + return TSDB_CODE_TSC_OUT_OF_MEMORY; } - param->pLocalData = pMerger->pLocalDataSrc; - param->pDesc = pMerger->pDesc; - param->num = pMerger->pLocalDataSrc[0]->pMemBuffer->numOfElemsPerPage; - SQueryInfo *pQueryInfo = tscGetQueryInfo(pCmd, pCmd->clauseIndex); + param->pLocalData = (*pMerger)->pLocalDataSrc; + param->pDesc = (*pMerger)->pDesc; + param->num = (*pMerger)->pLocalDataSrc[0]->pMemBuffer->numOfElemsPerPage; param->groupOrderType = pQueryInfo->groupbyExpr.orderType; - pMerger->orderPrjOnSTable = tscOrderedProjectionQueryOnSTable(pQueryInfo, 0); - pRes->code = tLoserTreeCreate(&pMerger->pLoserTree, pMerger->numOfBuffer, param, treeComparator); - if (pMerger->pLoserTree == NULL || pRes->code != 0) { + int32_t code = tLoserTreeCreate(&(*pMerger)->pLoserTree, (*pMerger)->numOfBuffer, param, treeComparator); + if ((*pMerger)->pLoserTree == NULL || code != TSDB_CODE_SUCCESS) { tfree(param); - tfree(pMerger); - return; - } - - // the input data format follows the old format, but output in a new format. - // so, all the input must be parsed as old format - pMerger->pCtx = (SQLFunctionCtx *)calloc(tscSqlExprNumOfExprs(pQueryInfo), sizeof(SQLFunctionCtx)); - pMerger->rowSize = pMemBuffer[0]->nElemSize; - - tscFieldInfoUpdateOffset(pQueryInfo); - - if (pMerger->rowSize > pMemBuffer[0]->pageSize) { - assert(false); // todo fixed row size is larger than the minimum page size; + tfree((*pMerger)); + return code; } - // used to keep the latest input row - pMerger->pTempBuffer = (tFilePage *)calloc(1, pMerger->rowSize + sizeof(tFilePage)); + (*pMerger)->rowSize = pMemBuffer[0]->nElemSize; - pMerger->nResultBufSize = pMemBuffer[0]->pageSize * 16; - pMerger->pResultBuf = (tFilePage *)calloc(1, pMerger->nResultBufSize + sizeof(tFilePage)); + // todo fixed row size is larger than the minimum page size; + assert((*pMerger)->rowSize <= pMemBuffer[0]->pageSize); - pMerger->resColModel = finalmodel; - pMerger->resColModel->capacity = pMerger->nResultBufSize; - pMerger->finalModel = pFFModel; - - if (finalmodel->rowSize > 0) { - pMerger->resColModel->capacity /= finalmodel->rowSize; - } - - assert(finalmodel->rowSize > 0 && finalmodel->rowSize <= pMerger->rowSize); - - if (pMerger->pTempBuffer == NULL || pMerger->pLoserTree == NULL) { - tfree(pMerger->pTempBuffer); - tfree(pMerger->pLoserTree); + if ((*pMerger)->pLoserTree == NULL) { + tfree((*pMerger)->pLoserTree); tfree(param); - tfree(pMerger); - pRes->code = TSDB_CODE_TSC_OUT_OF_MEMORY; - return; - } - - pMerger->pTempBuffer->num = 0; - tscCreateResPointerInfo(pRes, pQueryInfo); - - SSchema* pschema = calloc(pDesc->pColumnModel->numOfCols, sizeof(SSchema)); - for(int32_t i = 0; i < pDesc->pColumnModel->numOfCols; ++i) { - pschema[i] = pDesc->pColumnModel->pFields[i].field; - } - - tsCreateSQLFunctionCtx(pQueryInfo, pMerger->pCtx, pschema); -// setCtxInputOutputBuffer(pQueryInfo, pMerger->pCtx, pMerger, pDesc); - - tfree(pschema); - - int32_t maxBufSize = 0; - for (int32_t k = 0; k < tscSqlExprNumOfExprs(pQueryInfo); ++k) { - SExprInfo *pExpr = tscSqlExprGet(pQueryInfo, k); - if (maxBufSize < pExpr->base.resBytes && pExpr->base.functionId == TSDB_FUNC_TAG) { - maxBufSize = pExpr->base.resBytes; - } + tfree((*pMerger)); + return TSDB_CODE_TSC_OUT_OF_MEMORY; } - // we change the capacity of schema to denote that there is only one row in temp buffer - pMerger->pDesc->pColumnModel->capacity = 1; - // restore the limitation value at the last stage if (tscOrderedProjectionQueryOnSTable(pQueryInfo, 0)) { pQueryInfo->limit.limit = pQueryInfo->clauseLimit; pQueryInfo->limit.offset = pQueryInfo->prjOffset; } - pRes->pLocalMerger = pMerger; - pRes->numOfGroups = 0; + // we change the capacity of schema to denote that there is only one row in temp buffer + (*pMerger)->pDesc->pColumnModel->capacity = 1; -// STableMetaInfo *pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, pCmd->clauseIndex, 0); -// STableComInfo tinfo = tscGetTableInfo(pTableMetaInfo->pTableMeta); - -// TSKEY stime = (pQueryInfo->order.order == TSDB_ORDER_ASC)? pQueryInfo->window.skey : pQueryInfo->window.ekey; -// int64_t revisedSTime = taosTimeTruncate(stime, &pQueryInfo->interval, tinfo.precision); - -// if (pQueryInfo->fillType != TSDB_FILL_NONE) { -// SFillColInfo* pFillCol = createFillColInfo(pQueryInfo); -// pMerger->pFillInfo = -// taosCreateFillInfo(pQueryInfo->order.order, revisedSTime, pQueryInfo->groupbyExpr.numOfGroupCols, 4096, -// (int32_t)pQueryInfo->fieldsInfo.numOfOutput, pQueryInfo->interval.sliding, -// pQueryInfo->interval.slidingUnit, tinfo.precision, pQueryInfo->fillType, pFillCol, pSql); -// } + return TSDB_CODE_SUCCESS; } static int32_t tscFlushTmpBufferImpl(tExtMemBuffer *pMemoryBuf, tOrderDescriptor *pDesc, tFilePage *pPage, @@ -418,44 +297,32 @@ int32_t saveToBuffer(tExtMemBuffer *pMemoryBuf, tOrderDescriptor *pDesc, tFilePa return 0; } -void tscDestroyLocalMerger(SSqlObj *pSql) { - if (pSql == NULL) { +void tscDestroyLocalMerger(SLocalMerger* pLocalMerger) { + if (pLocalMerger == NULL) { return; } - SSqlRes *pRes = &(pSql->res); - if (pRes->pLocalMerger == NULL) { - return; + for (int32_t i = 0; i < pLocalMerger->numOfBuffer; ++i) { + tfree(pLocalMerger->pLocalDataSrc[i]); } - // there is no more result, so we release all allocated resource - SLocalMerger *pLocalMerge = (SLocalMerger *)atomic_exchange_ptr(&pRes->pLocalMerger, NULL); - tfree(pLocalMerge->pResultBuf); - tfree(pLocalMerge->pCtx); + pLocalMerger->numOfBuffer = 0; + tscLocalReducerEnvDestroy(pLocalMerger->pExtMemBuffer, pLocalMerger->pDesc, pLocalMerger->numOfVnode); - if (pLocalMerge->pLoserTree) { - tfree(pLocalMerge->pLoserTree->param); - tfree(pLocalMerge->pLoserTree); - } + pLocalMerger->numOfCompleted = 0; - tscLocalReducerEnvDestroy(pLocalMerge->pExtMemBuffer, pLocalMerge->pDesc, pLocalMerge->resColModel, - pLocalMerge->finalModel, pLocalMerge->numOfVnode); - for (int32_t i = 0; i < pLocalMerge->numOfBuffer; ++i) { - tfree(pLocalMerge->pLocalDataSrc[i]); + if (pLocalMerger->pLoserTree) { + tfree(pLocalMerger->pLoserTree->param); + tfree(pLocalMerger->pLoserTree); } - pLocalMerge->numOfBuffer = 0; - pLocalMerge->numOfCompleted = 0; - tfree(pLocalMerge->pTempBuffer); - - free(pLocalMerge); - - tscDebug("0x%"PRIx64" free local reducer finished", pSql->self); + tfree(pLocalMerger->buf); + tfree(pLocalMerger->pLocalDataSrc); + free(pLocalMerger); } -static int32_t createOrderDescriptor(tOrderDescriptor **pOrderDesc, SSqlCmd *pCmd, SColumnModel *pModel) { - int32_t numOfGroupByCols = 0; - SQueryInfo *pQueryInfo = tscGetActiveQueryInfo(pCmd); +static int32_t createOrderDescriptor(tOrderDescriptor **pOrderDesc, SQueryInfo* pQueryInfo, SColumnModel *pModel) { + int32_t numOfGroupByCols = 0; if (pQueryInfo->groupbyExpr.numOfGroupCols > 0) { numOfGroupByCols = pQueryInfo->groupbyExpr.numOfGroupCols; @@ -474,13 +341,13 @@ static int32_t createOrderDescriptor(tOrderDescriptor **pOrderDesc, SSqlCmd *pCm if (numOfGroupByCols > 0) { if (pQueryInfo->groupbyExpr.numOfGroupCols > 0) { - int32_t numOfInternalOutput = (int32_t) tscSqlExprNumOfExprs(pQueryInfo); + int32_t numOfInternalOutput = (int32_t) tscNumOfExprs(pQueryInfo); // the last "pQueryInfo->groupbyExpr.numOfGroupCols" columns are order-by columns for (int32_t i = 0; i < pQueryInfo->groupbyExpr.numOfGroupCols; ++i) { SColIndex* pColIndex = taosArrayGet(pQueryInfo->groupbyExpr.columnInfo, i); for(int32_t j = 0; j < numOfInternalOutput; ++j) { - SExprInfo* pExprInfo = tscSqlExprGet(pQueryInfo, j); + SExprInfo* pExprInfo = tscExprGet(pQueryInfo, j); int32_t functionId = pExprInfo->base.functionId; if (pColIndex->colId == pExprInfo->base.colInfo.colId && (functionId == TSDB_FUNC_PRJ || functionId == TSDB_FUNC_TAG)) { @@ -502,9 +369,9 @@ static int32_t createOrderDescriptor(tOrderDescriptor **pOrderDesc, SSqlCmd *pCm if (pQueryInfo->interval.interval != 0) { orderColIndexList[0] = PRIMARYKEY_TIMESTAMP_COL_INDEX; } else { - size_t size = tscSqlExprNumOfExprs(pQueryInfo); + size_t size = tscNumOfExprs(pQueryInfo); for (int32_t i = 0; i < size; ++i) { - SExprInfo *pExpr = tscSqlExprGet(pQueryInfo, i); + SExprInfo *pExpr = tscExprGet(pQueryInfo, i); if (pExpr->base.functionId == TSDB_FUNC_PRJ && pExpr->base.colInfo.colId == PRIMARYKEY_TIMESTAMP_COL_INDEX) { orderColIndexList[0] = i; } @@ -525,37 +392,30 @@ static int32_t createOrderDescriptor(tOrderDescriptor **pOrderDesc, SSqlCmd *pCm } } -int32_t tscLocalReducerEnvCreate(SSqlObj *pSql, tExtMemBuffer ***pMemBuffer, tOrderDescriptor **pOrderDesc, - SColumnModel **pFinalModel, SColumnModel** pFFModel, uint32_t nBufferSizes) { - SSqlCmd *pCmd = &pSql->cmd; - SSqlRes *pRes = &pSql->res; - - SSchema * pSchema = NULL; +int32_t tscLocalReducerEnvCreate(SQueryInfo *pQueryInfo, tExtMemBuffer ***pMemBuffer, int32_t numOfSub, + tOrderDescriptor **pOrderDesc, uint32_t nBufferSizes, int64_t id) { + SSchema *pSchema = NULL; SColumnModel *pModel = NULL; - *pFinalModel = NULL; - SQueryInfo * pQueryInfo = tscGetActiveQueryInfo(pCmd); STableMetaInfo *pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0); - (*pMemBuffer) = (tExtMemBuffer **)malloc(POINTER_BYTES * pSql->subState.numOfSub); + (*pMemBuffer) = (tExtMemBuffer **)malloc(POINTER_BYTES * numOfSub); if (*pMemBuffer == NULL) { - tscError("0x%"PRIx64" failed to allocate memory", pSql->self); - pRes->code = TSDB_CODE_TSC_OUT_OF_MEMORY; - return pRes->code; + tscError("0x%"PRIx64" failed to allocate memory", id); + return TSDB_CODE_TSC_OUT_OF_MEMORY; } - size_t size = tscSqlExprNumOfExprs(pQueryInfo); + size_t size = tscNumOfExprs(pQueryInfo); pSchema = (SSchema *)calloc(1, sizeof(SSchema) * size); if (pSchema == NULL) { - tscError("0x%"PRIx64" failed to allocate memory", pSql->self); - pRes->code = TSDB_CODE_TSC_OUT_OF_MEMORY; - return pRes->code; + tscError("0x%"PRIx64" failed to allocate memory", id); + return TSDB_CODE_TSC_OUT_OF_MEMORY; } int32_t rlen = 0; for (int32_t i = 0; i < size; ++i) { - SExprInfo *pExpr = tscSqlExprGet(pQueryInfo, i); + SExprInfo *pExpr = tscExprGet(pQueryInfo, i); pSchema[i].bytes = pExpr->base.resBytes; pSchema[i].type = (int8_t)pExpr->base.resType; @@ -570,6 +430,7 @@ int32_t tscLocalReducerEnvCreate(SSqlObj *pSql, tExtMemBuffer ***pMemBuffer, tOr } pModel = createColumnModel(pSchema, (int32_t)size, capacity); + tfree(pSchema); int32_t pg = DEFAULT_PAGE_SIZE; int32_t overhead = sizeof(tFilePage); @@ -577,95 +438,26 @@ int32_t tscLocalReducerEnvCreate(SSqlObj *pSql, tExtMemBuffer ***pMemBuffer, tOr pg *= 2; } - size_t numOfSubs = pSql->subState.numOfSub; - assert(numOfSubs <= pTableMetaInfo->vgroupList->numOfVgroups); - for (int32_t i = 0; i < numOfSubs; ++i) { + assert(numOfSub <= pTableMetaInfo->vgroupList->numOfVgroups); + for (int32_t i = 0; i < numOfSub; ++i) { (*pMemBuffer)[i] = createExtMemBuffer(nBufferSizes, rlen, pg, pModel); (*pMemBuffer)[i]->flushModel = MULTIPLE_APPEND_MODEL; } - if (createOrderDescriptor(pOrderDesc, pCmd, pModel) != TSDB_CODE_SUCCESS) { - pRes->code = TSDB_CODE_TSC_OUT_OF_MEMORY; - tfree(pSchema); - return pRes->code; - } - - // final result depends on the fields number - memset(pSchema, 0, sizeof(SSchema) * size); - - for (int32_t i = 0; i < size; ++i) { - SExprInfo *pExpr = tscSqlExprGet(pQueryInfo, i); - - SSchema p1 = {0}; - if (pExpr->base.colInfo.colIndex == TSDB_TBNAME_COLUMN_INDEX) { - p1 = *tGetTbnameColumnSchema(); - } else if (TSDB_COL_IS_UD_COL(pExpr->base.colInfo.flag)) { - p1.bytes = pExpr->base.resBytes; - p1.type = (uint8_t) pExpr->base.resType; - tstrncpy(p1.name, pExpr->base.aliasName, tListLen(p1.name)); - } else { - p1 = *tscGetTableColumnSchema(pTableMetaInfo->pTableMeta, pExpr->base.colInfo.colIndex); - } - - int32_t inter = 0; - int16_t type = -1; - int16_t bytes = 0; - - // the final result size and type in the same as query on single table. - // so here, set the flag to be false; - int32_t functionId = pExpr->base.functionId; - if (functionId >= TSDB_FUNC_TS && functionId <= TSDB_FUNC_DIFF) { - type = pModel->pFields[i].field.type; - bytes = pModel->pFields[i].field.bytes; - } else { - if (functionId == TSDB_FUNC_FIRST_DST) { - functionId = TSDB_FUNC_FIRST; - } else if (functionId == TSDB_FUNC_LAST_DST) { - functionId = TSDB_FUNC_LAST; - } else if (functionId == TSDB_FUNC_STDDEV_DST) { - functionId = TSDB_FUNC_STDDEV; - } - - int32_t ret = getResultDataInfo(p1.type, p1.bytes, functionId, 0, &type, &bytes, &inter, 0, false); - assert(ret == TSDB_CODE_SUCCESS); - } - - pSchema[i].type = (uint8_t)type; - pSchema[i].bytes = bytes; - strcpy(pSchema[i].name, pModel->pFields[i].field.name); - } - - *pFinalModel = createColumnModel(pSchema, (int32_t)size, capacity); - - memset(pSchema, 0, sizeof(SSchema) * size); - size = tscNumOfFields(pQueryInfo); - - for(int32_t i = 0; i < size; ++i) { - SInternalField* pField = tscFieldInfoGetInternalField(&pQueryInfo->fieldsInfo, i); - pSchema[i].bytes = pField->field.bytes; - pSchema[i].type = pField->field.type; - tstrncpy(pSchema[i].name, pField->field.name, tListLen(pSchema[i].name)); + if (createOrderDescriptor(pOrderDesc, pQueryInfo, pModel) != TSDB_CODE_SUCCESS) { + return TSDB_CODE_TSC_OUT_OF_MEMORY; } - *pFFModel = createColumnModel(pSchema, (int32_t) size, capacity); - - tfree(pSchema); return TSDB_CODE_SUCCESS; } /** * @param pMemBuffer * @param pDesc - * @param pFinalModel * @param numOfVnodes */ -void tscLocalReducerEnvDestroy(tExtMemBuffer **pMemBuffer, tOrderDescriptor *pDesc, SColumnModel *pFinalModel, SColumnModel *pFFModel, - int32_t numOfVnodes) { - destroyColumnModel(pFinalModel); - destroyColumnModel(pFFModel); - +void tscLocalReducerEnvDestroy(tExtMemBuffer **pMemBuffer, tOrderDescriptor *pDesc, int32_t numOfVnodes) { tOrderDescDestroy(pDesc); - for (int32_t i = 0; i < numOfVnodes; ++i) { pMemBuffer[i] = destoryExtMemBuffer(pMemBuffer[i]); } @@ -877,10 +669,12 @@ static bool isAllSourcesCompleted(SLocalMerger *pLocalMerge) { return (pLocalMerge->numOfBuffer == pLocalMerge->numOfCompleted); } -void tscInitResObjForLocalQuery(SSqlObj *pObj, int32_t numOfRes, int32_t rowLen) { - SSqlRes *pRes = &pObj->res; +void tscInitResObjForLocalQuery(SSqlObj *pSql, int32_t numOfRes, int32_t rowLen) { + SSqlRes *pRes = &pSql->res; if (pRes->pLocalMerger != NULL) { - tscDestroyLocalMerger(pObj); + tscDestroyLocalMerger(pRes->pLocalMerger); + pRes->pLocalMerger = NULL; + tscDebug("0x%"PRIx64" free local reducer finished", pSql->self); } pRes->qId = 1; // hack to pass the safety check in fetch_row function @@ -891,14 +685,12 @@ void tscInitResObjForLocalQuery(SSqlObj *pObj, int32_t numOfRes, int32_t rowLen) pRes->pLocalMerger = (SLocalMerger *)calloc(1, sizeof(SLocalMerger)); /* - * we need one additional byte space - * the sprintf function needs one additional space to put '\0' at the end of string + * One more byte space is required, since the sprintf function needs one additional space to put '\0' at + * the end of string */ - size_t allocSize = numOfRes * rowLen + sizeof(tFilePage) + 1; - pRes->pLocalMerger->pResultBuf = (tFilePage *)calloc(1, allocSize); - - pRes->pLocalMerger->pResultBuf->num = numOfRes; - pRes->data = pRes->pLocalMerger->pResultBuf->data; + size_t size = numOfRes * rowLen + 1; + pRes->pLocalMerger->buf = calloc(1, size); + pRes->data = pRes->pLocalMerger->buf; } int32_t doArithmeticCalculate(SQueryInfo* pQueryInfo, tFilePage* pOutput, int32_t rowSize, int32_t finalRowSize) { @@ -910,12 +702,12 @@ int32_t doArithmeticCalculate(SQueryInfo* pQueryInfo, tFilePage* pOutput, int32_ // todo refactor arithSup.offset = 0; - arithSup.numOfCols = (int32_t) tscSqlExprNumOfExprs(pQueryInfo); + arithSup.numOfCols = (int32_t) tscNumOfExprs(pQueryInfo); arithSup.exprList = pQueryInfo->exprList; arithSup.data = calloc(arithSup.numOfCols, POINTER_BYTES); for(int32_t k = 0; k < arithSup.numOfCols; ++k) { - SExprInfo* pExpr = tscSqlExprGet(pQueryInfo, k); + SExprInfo* pExpr = tscExprGet(pQueryInfo, k); arithSup.data[k] = (pOutput->data + pOutput->num* pExpr->base.offset); } @@ -944,8 +736,8 @@ int32_t doArithmeticCalculate(SQueryInfo* pQueryInfo, tFilePage* pOutput, int32_ return offset; } -#define COLMODEL_GET_VAL(data, schema, allrow, rowId, colId) \ - (data + (schema)->pFields[colId].offset * (allrow) + (rowId) * (schema)->pFields[colId].field.bytes) +#define COLMODEL_GET_VAL(data, schema, rowId, colId) \ + (data + (schema)->pFields[colId].offset * ((schema)->capacity) + (rowId) * (schema)->pFields[colId].field.bytes) static void appendOneRowToDataBlock(SSDataBlock *pBlock, char *buf, SColumnModel *pModel, int32_t rowIndex, int32_t maxRows) { @@ -953,7 +745,7 @@ static void appendOneRowToDataBlock(SSDataBlock *pBlock, char *buf, SColumnModel SColumnInfoData* pColInfo = taosArrayGet(pBlock->pDataBlock, i); char* p = pColInfo->pData + pBlock->info.rows * pColInfo->info.bytes; - char *src = COLMODEL_GET_VAL(buf, pModel, maxRows, rowIndex, i); + char *src = COLMODEL_GET_VAL(buf, pModel, rowIndex, i); memmove(p, src, pColInfo->info.bytes); } @@ -970,8 +762,6 @@ SSDataBlock* doMultiwayMergeSort(void* param, bool* newgroup) { SLocalMerger *pMerger = pInfo->pMerge; SLoserTreeInfo *pTree = pMerger->pLoserTree; - SColumnModel *pModel = pMerger->pDesc->pColumnModel; - tFilePage *tmpBuffer = pMerger->pTempBuffer; pInfo->binfo.pRes->info.rows = 0; @@ -984,7 +774,7 @@ SSDataBlock* doMultiwayMergeSort(void* param, bool* newgroup) { printf("chosen data in pTree[0] = %d\n", pTree->pNode[0].index); #endif - assert((pTree->pNode[0].index < pMerger->numOfBuffer) && (pTree->pNode[0].index >= 0) && tmpBuffer->num == 0); + assert((pTree->pNode[0].index < pMerger->numOfBuffer) && (pTree->pNode[0].index >= 0)); // chosen from loser tree SLocalDataSource *pOneDataSrc = pMerger->pLocalDataSrc[pTree->pNode[0].index]; @@ -997,11 +787,10 @@ SSDataBlock* doMultiwayMergeSort(void* param, bool* newgroup) { SColIndex * pIndex = taosArrayGet(pInfo->orderColumnList, i); SColumnInfoData *pColInfo = taosArrayGet(pInfo->binfo.pRes->pDataBlock, pIndex->colIndex); - char *newRow = - COLMODEL_GET_VAL(pOneDataSrc->filePage.data, pModel, pOneDataSrc->pMemBuffer->pColumnModel->capacity, - pOneDataSrc->rowIdx, pIndex->colIndex); + char *newRow = COLMODEL_GET_VAL(pOneDataSrc->filePage.data, pOneDataSrc->pMemBuffer->pColumnModel, + pOneDataSrc->rowIdx, pIndex->colIndex); - char * data = pInfo->prevRow[i]; + char *data = pInfo->prevRow[i]; int32_t ret = columnValueAscendingComparator(data, newRow, pColInfo->info.type, pColInfo->info.bytes); if (ret == 0) { continue; @@ -1020,9 +809,8 @@ SSDataBlock* doMultiwayMergeSort(void* param, bool* newgroup) { SColIndex * pIndex = taosArrayGet(pInfo->orderColumnList, i); SColumnInfoData *pColInfo = taosArrayGet(pInfo->binfo.pRes->pDataBlock, pIndex->colIndex); - char *curCol = - COLMODEL_GET_VAL(pOneDataSrc->filePage.data, pModel, pOneDataSrc->pMemBuffer->pColumnModel->capacity, - pOneDataSrc->rowIdx, pIndex->colIndex); + char *curCol = COLMODEL_GET_VAL(pOneDataSrc->filePage.data, pOneDataSrc->pMemBuffer->pColumnModel, + pOneDataSrc->rowIdx, pIndex->colIndex); memcpy(pInfo->prevRow[i], curCol, pColInfo->info.bytes); } @@ -1033,7 +821,8 @@ SSDataBlock* doMultiwayMergeSort(void* param, bool* newgroup) { return pInfo->binfo.pRes; } - appendOneRowToDataBlock(pInfo->binfo.pRes, pOneDataSrc->filePage.data, pModel, pOneDataSrc->rowIdx, pOneDataSrc->pMemBuffer->pColumnModel->capacity); + appendOneRowToDataBlock(pInfo->binfo.pRes, pOneDataSrc->filePage.data, pOneDataSrc->pMemBuffer->pColumnModel, + pOneDataSrc->rowIdx, pOneDataSrc->pMemBuffer->pColumnModel->capacity); #if defined(_DEBUG_VIEW) printf("chosen row:\t"); @@ -1082,7 +871,7 @@ SSDataBlock* doGlobalAggregate(void* param, bool* newgroup) { } SMultiwayMergeInfo *pAggInfo = pOperator->info; - SOperatorInfo *upstream = pOperator->upstream; + SOperatorInfo *upstream = pOperator->upstream[0]; *newgroup = false; bool handleData = false; @@ -1166,7 +955,6 @@ SSDataBlock* doGlobalAggregate(void* param, bool* newgroup) { if (pInfoData->info.type == TSDB_DATA_TYPE_TIMESTAMP && pRes->info.rows > 0) { STimeWindow* w = &pRes->info.window; - // TODO in case of desc order, swap it w->skey = *(int64_t*)pInfoData->pData; w->ekey = *(int64_t*)(((char*)pInfoData->pData) + TSDB_KEYSIZE * (pRes->info.rows - 1)); @@ -1186,7 +974,7 @@ static SSDataBlock* skipGroupBlock(SOperatorInfo* pOperator, bool* newgroup) { SSDataBlock* pBlock = NULL; if (pInfo->currentGroupOffset == 0) { - pBlock = pOperator->upstream->exec(pOperator->upstream, newgroup); + pBlock = pOperator->upstream[0]->exec(pOperator->upstream[0], newgroup); if (pBlock == NULL) { setQueryStatus(pOperator->pRuntimeEnv, QUERY_COMPLETED); pOperator->status = OP_EXEC_DONE; @@ -1194,7 +982,7 @@ static SSDataBlock* skipGroupBlock(SOperatorInfo* pOperator, bool* newgroup) { if (*newgroup == false && pInfo->limit.limit > 0 && pInfo->rowsTotal >= pInfo->limit.limit) { while ((*newgroup) == false) { // ignore the remain blocks - pBlock = pOperator->upstream->exec(pOperator->upstream, newgroup); + pBlock = pOperator->upstream[0]->exec(pOperator->upstream[0], newgroup); if (pBlock == NULL) { setQueryStatus(pOperator->pRuntimeEnv, QUERY_COMPLETED); pOperator->status = OP_EXEC_DONE; @@ -1206,7 +994,7 @@ static SSDataBlock* skipGroupBlock(SOperatorInfo* pOperator, bool* newgroup) { return pBlock; } - pBlock = pOperator->upstream->exec(pOperator->upstream, newgroup); + pBlock = pOperator->upstream[0]->exec(pOperator->upstream[0], newgroup); if (pBlock == NULL) { setQueryStatus(pOperator->pRuntimeEnv, QUERY_COMPLETED); pOperator->status = OP_EXEC_DONE; @@ -1220,7 +1008,7 @@ static SSDataBlock* skipGroupBlock(SOperatorInfo* pOperator, bool* newgroup) { } while ((*newgroup) == false) { - pBlock = pOperator->upstream->exec(pOperator->upstream, newgroup); + pBlock = pOperator->upstream[0]->exec(pOperator->upstream[0], newgroup); if (pBlock == NULL) { setQueryStatus(pOperator->pRuntimeEnv, QUERY_COMPLETED); pOperator->status = OP_EXEC_DONE; diff --git a/src/client/src/tscParseInsert.c b/src/client/src/tscParseInsert.c index d96e25dd3799f24d5f846553b7e23161d7107419..cc89fd6220cfc24faf36157ba7541aa497f2e523 100644 --- a/src/client/src/tscParseInsert.c +++ b/src/client/src/tscParseInsert.c @@ -107,7 +107,7 @@ int tsParseTime(SStrToken *pToken, int64_t *time, char **next, char *error, int1 } if (parseAbsoluteDuration(valueToken.z, valueToken.n, &interval) != TSDB_CODE_SUCCESS) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } if (timePrec == TSDB_TIME_PRECISION_MILLI) { @@ -441,7 +441,7 @@ int tsParseOneRow(char **str, STableDataBlocks *pDataBlocks, SSqlCmd *pCmd, int1 *str += index; if (sToken.type == TK_QUESTION) { - if (pCmd->insertType != TSDB_QUERY_TYPE_STMT_INSERT) { + if (pCmd->insertParam.insertType != TSDB_QUERY_TYPE_STMT_INSERT) { return tscSQLSyntaxErrMsg(pCmd->payload, "? only allowed in binding insertion", *str); } @@ -647,7 +647,7 @@ static int32_t tsSetBlockInfo(SSubmitBlk *pBlocks, const STableMeta *pTableMeta, pBlocks->sversion = pTableMeta->sversion; if (pBlocks->numOfRows + numOfRows >= INT16_MAX) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } else { pBlocks->numOfRows += numOfRows; return TSDB_CODE_SUCCESS; @@ -708,7 +708,7 @@ static int32_t doParseInsertStatement(SSqlCmd* pCmd, char **str, STableDataBlock return TSDB_CODE_TSC_OUT_OF_MEMORY; } - code = TSDB_CODE_TSC_INVALID_SQL; + code = TSDB_CODE_TSC_INVALID_OPERATION; char tmpTokenBuf[16*1024] = {0}; // used for deleting Escape character: \\, \', \" int32_t numOfRows = 0; @@ -747,12 +747,10 @@ static int32_t tscCheckIfCreateTable(char **sqlstr, SSqlObj *pSql, char** boundC const int32_t STABLE_INDEX = 1; SSqlCmd * pCmd = &pSql->cmd; - SQueryInfo *pQueryInfo = tscGetQueryInfo(pCmd, 0); + SQueryInfo *pQueryInfo = tscGetQueryInfo(pCmd); char *sql = *sqlstr; - pSql->cmd.autoCreated = false; - // get the token of specified table index = 0; tableToken = tStrGetToken(sql, &index, false); @@ -786,7 +784,7 @@ static int32_t tscCheckIfCreateTable(char **sqlstr, SSqlObj *pSql, char** boundC } if (numOfColList == 0 && (*boundColumn) != NULL) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, TABLE_INDEX); @@ -802,7 +800,7 @@ static int32_t tscCheckIfCreateTable(char **sqlstr, SSqlObj *pSql, char** boundC } STableMetaInfo *pSTableMetaInfo = tscGetMetaInfo(pQueryInfo, STABLE_INDEX); - code = tscSetTableFullName(pSTableMetaInfo, &sToken, pSql); + code = tscSetTableFullName(&pSTableMetaInfo->name, &sToken, pSql); if (code != TSDB_CODE_SUCCESS) { return code; } @@ -879,7 +877,7 @@ static int32_t tscCheckIfCreateTable(char **sqlstr, SSqlObj *pSql, char** boundC if (TK_ILLEGAL == sToken.type) { tdDestroyKVRowBuilder(&kvRowBuilder); tscDestroyBoundColumnInfo(&spd); - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } if (sToken.n == 0 || sToken.type == TK_RP) { @@ -961,7 +959,7 @@ static int32_t tscCheckIfCreateTable(char **sqlstr, SSqlObj *pSql, char** boundC } if (numOfColsAfterTags == 0 && (*boundColumn) != NULL) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } sToken = tStrGetToken(sql, &index, false); @@ -973,13 +971,13 @@ static int32_t tscCheckIfCreateTable(char **sqlstr, SSqlObj *pSql, char** boundC return tscInvalidSQLErrMsg(pCmd->payload, "invalid table name", *sqlstr); } - int32_t ret = tscSetTableFullName(pTableMetaInfo, &tableToken, pSql); + int32_t ret = tscSetTableFullName(&pTableMetaInfo->name, &tableToken, pSql); if (ret != TSDB_CODE_SUCCESS) { return ret; } if (sql == NULL) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } code = tscGetTableMetaEx(pSql, pTableMetaInfo, true); @@ -991,7 +989,7 @@ static int32_t tscCheckIfCreateTable(char **sqlstr, SSqlObj *pSql, char** boundC sql = sToken.z; if (sql == NULL) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } code = tscGetTableMetaEx(pSql, pTableMetaInfo, false); @@ -1015,12 +1013,17 @@ int validateTableName(char *tblName, int len, SStrToken* psTblToken) { return tscValidateName(psTblToken); } -static int32_t validateDataSource(SSqlCmd *pCmd, int8_t type, const char *sql) { - if (pCmd->dataSourceType != 0 && pCmd->dataSourceType != type) { - return tscInvalidSQLErrMsg(pCmd->payload, "keyword VALUES and FILE are not allowed to mix up", sql); +static int32_t validateDataSource(SSqlCmd *pCmd, int32_t type, const char *sql) { + uint32_t *insertType = &pCmd->insertParam.insertType; + if (*insertType == TSDB_QUERY_TYPE_STMT_INSERT && type == TSDB_QUERY_TYPE_INSERT) { + return TSDB_CODE_SUCCESS; + } + + if ((*insertType) != 0 && (*insertType) != type) { + return tscInvalidSQLErrMsg(pCmd->payload, "keyword VALUES and FILE are not allowed to mixed up", sql); } - pCmd->dataSourceType = type; + *insertType = type; return TSDB_CODE_SUCCESS; } @@ -1090,7 +1093,6 @@ static int32_t parseBoundColumns(SSqlCmd* pCmd, SParsedDataColInfo* pColInfo, SS _clean: pCmd->curSql = NULL; - pCmd->parseFinished = 1; return code; } @@ -1106,7 +1108,7 @@ int tsParseInsertSql(SSqlObj *pSql) { int32_t totalNum = 0; int32_t code = TSDB_CODE_SUCCESS; - SQueryInfo *pQueryInfo = tscGetQueryInfo(pCmd, 0); + SQueryInfo *pQueryInfo = tscGetQueryInfo(pCmd); assert(pQueryInfo != NULL); STableMetaInfo *pTableMetaInfo = (pQueryInfo->numOfTables == 0)? tscAddEmptyMetaInfo(pQueryInfo):tscGetMetaInfo(pQueryInfo, 0); @@ -1120,9 +1122,9 @@ int tsParseInsertSql(SSqlObj *pSql) { return code; } - if (NULL == pCmd->pTableBlockHashList) { - pCmd->pTableBlockHashList = taosHashInit(128, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BIGINT), true, false); - if (NULL == pCmd->pTableBlockHashList) { + if (NULL == pCmd->insertParam.pTableBlockHashList) { + pCmd->insertParam.pTableBlockHashList = taosHashInit(128, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BIGINT), true, false); + if (NULL == pCmd->insertParam.pTableBlockHashList) { code = TSDB_CODE_TSC_OUT_OF_MEMORY; goto _clean; } @@ -1130,7 +1132,7 @@ int tsParseInsertSql(SSqlObj *pSql) { str = pCmd->curSql; } - tscDebug("0x%"PRIx64" create data block list hashList:%p", pSql->self, pCmd->pTableBlockHashList); + tscDebug("0x%"PRIx64" create data block list hashList:%p", pSql->self, pCmd->insertParam.pTableBlockHashList); while (1) { int32_t index = 0; @@ -1142,7 +1144,7 @@ int tsParseInsertSql(SSqlObj *pSql) { * if the data is from the data file, no data has been generated yet. So, there no data to * merge or submit, save the file path and parse the file in other routines. */ - if (pCmd->dataSourceType == DATA_FROM_DATA_FILE) { + if (TSDB_QUERY_HAS_TYPE(pCmd->insertParam.insertType, TSDB_QUERY_TYPE_FILE_INSERT)) { goto _clean; } @@ -1151,7 +1153,7 @@ int tsParseInsertSql(SSqlObj *pSql) { * Otherwise, create the first submit block and submit to virtual node. */ if (totalNum == 0) { - code = TSDB_CODE_TSC_INVALID_SQL; + code = TSDB_CODE_TSC_INVALID_OPERATION; goto _clean; } else { break; @@ -1168,7 +1170,7 @@ int tsParseInsertSql(SSqlObj *pSql) { goto _clean; } - if ((code = tscSetTableFullName(pTableMetaInfo, &sTblToken, pSql)) != TSDB_CODE_SUCCESS) { + if ((code = tscSetTableFullName(&pTableMetaInfo->name, &sTblToken, pSql)) != TSDB_CODE_SUCCESS) { goto _clean; } @@ -1203,7 +1205,7 @@ int tsParseInsertSql(SSqlObj *pSql) { STableComInfo tinfo = tscGetTableInfo(pTableMetaInfo->pTableMeta); if (sToken.type == TK_FILE) { - if (validateDataSource(pCmd, DATA_FROM_DATA_FILE, sToken.z) != TSDB_CODE_SUCCESS) { + if (validateDataSource(pCmd, TSDB_QUERY_TYPE_FILE_INSERT, sToken.z) != TSDB_CODE_SUCCESS) { goto _clean; } @@ -1236,12 +1238,12 @@ int tsParseInsertSql(SSqlObj *pSql) { if (bindedColumns == NULL) { STableMeta *pTableMeta = pTableMetaInfo->pTableMeta; - if (validateDataSource(pCmd, DATA_FROM_SQL_STRING, sToken.z) != TSDB_CODE_SUCCESS) { + if (validateDataSource(pCmd, TSDB_QUERY_TYPE_INSERT, sToken.z) != TSDB_CODE_SUCCESS) { goto _clean; } STableDataBlocks *dataBuf = NULL; - int32_t ret = tscGetDataBlockFromList(pCmd->pTableBlockHashList, pTableMeta->id.uid, TSDB_DEFAULT_PAYLOAD_SIZE, + int32_t ret = tscGetDataBlockFromList(pCmd->insertParam.pTableBlockHashList, pTableMeta->id.uid, TSDB_DEFAULT_PAYLOAD_SIZE, sizeof(SSubmitBlk), tinfo.rowSize, &pTableMetaInfo->name, pTableMeta, &dataBuf, NULL); if (ret != TSDB_CODE_SUCCESS) { @@ -1254,14 +1256,14 @@ int tsParseInsertSql(SSqlObj *pSql) { } } else { // bindedColumns != NULL // insert into tablename(col1, col2,..., coln) values(v1, v2,... vn); - STableMeta *pTableMeta = tscGetTableMetaInfoFromCmd(pCmd, pCmd->clauseIndex, 0)->pTableMeta; + STableMeta *pTableMeta = tscGetTableMetaInfoFromCmd(pCmd, 0)->pTableMeta; - if (validateDataSource(pCmd, DATA_FROM_SQL_STRING, sToken.z) != TSDB_CODE_SUCCESS) { + if (validateDataSource(pCmd, TSDB_QUERY_TYPE_INSERT, sToken.z) != TSDB_CODE_SUCCESS) { goto _clean; } STableDataBlocks *dataBuf = NULL; - int32_t ret = tscGetDataBlockFromList(pCmd->pTableBlockHashList, pTableMeta->id.uid, TSDB_DEFAULT_PAYLOAD_SIZE, + int32_t ret = tscGetDataBlockFromList(pCmd->insertParam.pTableBlockHashList, pTableMeta->id.uid, TSDB_DEFAULT_PAYLOAD_SIZE, sizeof(SSubmitBlk), tinfo.rowSize, &pTableMetaInfo->name, pTableMeta, &dataBuf, NULL); if (ret != TSDB_CODE_SUCCESS) { @@ -1297,7 +1299,8 @@ int tsParseInsertSql(SSqlObj *pSql) { goto _clean; } - if ((pCmd->insertType != TSDB_QUERY_TYPE_STMT_INSERT) && taosHashGetSize(pCmd->pTableBlockHashList) > 0) { // merge according to vgId + // merge according to vgId + if (!TSDB_QUERY_HAS_TYPE(pCmd->insertParam.insertType, TSDB_QUERY_TYPE_STMT_INSERT) && taosHashGetSize(pCmd->insertParam.pTableBlockHashList) > 0) { if ((code = tscMergeTableDataBlocks(pSql, true)) != TSDB_CODE_SUCCESS) { goto _clean; } @@ -1308,7 +1311,6 @@ int tsParseInsertSql(SSqlObj *pSql) { _clean: pCmd->curSql = NULL; - pCmd->parseFinished = 1; return code; } @@ -1326,9 +1328,8 @@ int tsInsertInitialCheck(SSqlObj *pSql) { pCmd->count = 0; pCmd->command = TSDB_SQL_INSERT; - SQueryInfo *pQueryInfo = tscGetQueryInfoS(pCmd, pCmd->clauseIndex); - - TSDB_QUERY_SET_TYPE(pQueryInfo->type, TSDB_QUERY_TYPE_INSERT | pCmd->insertType); + SQueryInfo *pQueryInfo = tscGetQueryInfoS(pCmd); + TSDB_QUERY_SET_TYPE(pQueryInfo->type, TSDB_QUERY_TYPE_INSERT); sToken = tStrGetToken(pSql->sqlstr, &index, false); if (sToken.type != TK_INTO) { @@ -1343,11 +1344,11 @@ int tsParseSql(SSqlObj *pSql, bool initial) { int32_t ret = TSDB_CODE_SUCCESS; SSqlCmd* pCmd = &pSql->cmd; - if ((!pCmd->parseFinished) && (!initial)) { + if (!initial) { tscDebug("0x%"PRIx64" resume to parse sql: %s", pSql->self, pCmd->curSql); } - ret = tscAllocPayload(&pSql->cmd, TSDB_DEFAULT_PAYLOAD_SIZE); + ret = tscAllocPayload(pCmd, TSDB_DEFAULT_PAYLOAD_SIZE); if (TSDB_CODE_SUCCESS != ret) { return ret; } @@ -1357,31 +1358,32 @@ int tsParseSql(SSqlObj *pSql, bool initial) { return ret; } - // make a backup as tsParseInsertSql may modify the string - char* sqlstr = strdup(pSql->sqlstr); ret = tsParseInsertSql(pSql); - if ((sqlstr == NULL) || (pSql->parseRetry >= 1) || - (ret != TSDB_CODE_TSC_SQL_SYNTAX_ERROR && ret != TSDB_CODE_TSC_INVALID_SQL)) { - free(sqlstr); - } else { + assert(ret == TSDB_CODE_SUCCESS || ret == TSDB_CODE_TSC_ACTION_IN_PROGRESS || ret == TSDB_CODE_TSC_SQL_SYNTAX_ERROR || ret == TSDB_CODE_TSC_INVALID_OPERATION); + + if (pSql->parseRetry < 1 && (ret == TSDB_CODE_TSC_SQL_SYNTAX_ERROR || ret == TSDB_CODE_TSC_INVALID_OPERATION)) { + tscDebug("0x%"PRIx64 " parse insert sql statement failed, code:%s, clear meta cache and retry ", pSql->self, tstrerror(ret)); + tscResetSqlCmd(pCmd, true); - free(pSql->sqlstr); - pSql->sqlstr = sqlstr; pSql->parseRetry++; + if ((ret = tsInsertInitialCheck(pSql)) == TSDB_CODE_SUCCESS) { ret = tsParseInsertSql(pSql); } } } else { - SSqlInfo SQLInfo = qSqlParse(pSql->sqlstr); - ret = tscToSQLCmd(pSql, &SQLInfo); - if (ret == TSDB_CODE_TSC_INVALID_SQL && pSql->parseRetry == 0 && SQLInfo.type == TSDB_SQL_NULL) { + SSqlInfo sqlInfo = qSqlParse(pSql->sqlstr); + ret = tscValidateSqlInfo(pSql, &sqlInfo); + if (ret == TSDB_CODE_TSC_INVALID_OPERATION && pSql->parseRetry < 1 && sqlInfo.type == TSDB_SQL_SELECT) { + tscDebug("0x%"PRIx64 " parse query sql statement failed, code:%s, clear meta cache and retry ", pSql->self, tstrerror(ret)); + tscResetSqlCmd(pCmd, true); pSql->parseRetry++; - ret = tscToSQLCmd(pSql, &SQLInfo); + + ret = tscValidateSqlInfo(pSql, &sqlInfo); } - SqlInfoDestroy(&SQLInfo); + SqlInfoDestroy(&sqlInfo); } /* @@ -1398,8 +1400,7 @@ static int doPackSendDataBlock(SSqlObj *pSql, int32_t numOfRows, STableDataBlock SSqlCmd *pCmd = &pSql->cmd; pSql->res.numOfRows = 0; - assert(pCmd->numOfClause == 1); - STableMeta *pTableMeta = tscGetTableMetaInfoFromCmd(pCmd, pCmd->clauseIndex, 0)->pTableMeta; + STableMeta *pTableMeta = tscGetTableMetaInfoFromCmd(pCmd, 0)->pTableMeta; SSubmitBlk *pBlocks = (SSubmitBlk *)(pTableDataBlocks->pData); code = tsSetBlockInfo(pBlocks, pTableMeta, numOfRows); @@ -1411,7 +1412,7 @@ static int doPackSendDataBlock(SSqlObj *pSql, int32_t numOfRows, STableDataBlock return code; } - STableDataBlocks *pDataBlock = taosArrayGetP(pCmd->pDataBlocks, 0); + STableDataBlocks *pDataBlock = taosArrayGetP(pCmd->insertParam.pDataBlocks, 0); if ((code = tscCopyDataBlockToPayload(pSql, pDataBlock)) != TSDB_CODE_SUCCESS) { return code; } @@ -1461,17 +1462,17 @@ static void parseFileSendDataBlock(void *param, TAOS_RES *tres, int32_t numOfRow // accumulate the total submit records pParentSql->res.numOfRows += pSql->res.numOfRows; - STableMetaInfo *pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, pCmd->clauseIndex, 0); + STableMetaInfo *pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, 0); STableMeta * pTableMeta = pTableMetaInfo->pTableMeta; STableComInfo tinfo = tscGetTableInfo(pTableMeta); destroyTableNameList(pCmd); - pCmd->pDataBlocks = tscDestroyBlockArrayList(pCmd->pDataBlocks); + pCmd->insertParam.pDataBlocks = tscDestroyBlockArrayList(pCmd->insertParam.pDataBlocks); - if (pCmd->pTableBlockHashList == NULL) { - pCmd->pTableBlockHashList = taosHashInit(16, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BIGINT), true, false); - if (pCmd->pTableBlockHashList == NULL) { + if (pCmd->insertParam.pTableBlockHashList == NULL) { + pCmd->insertParam.pTableBlockHashList = taosHashInit(16, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BIGINT), true, false); + if (pCmd->insertParam.pTableBlockHashList == NULL) { code = TSDB_CODE_TSC_OUT_OF_MEMORY; goto _error; } @@ -1479,7 +1480,7 @@ static void parseFileSendDataBlock(void *param, TAOS_RES *tres, int32_t numOfRow STableDataBlocks *pTableDataBlock = NULL; int32_t ret = - tscGetDataBlockFromList(pCmd->pTableBlockHashList, pTableMeta->id.uid, TSDB_PAYLOAD_SIZE, sizeof(SSubmitBlk), + tscGetDataBlockFromList(pCmd->insertParam.pTableBlockHashList, pTableMeta->id.uid, TSDB_PAYLOAD_SIZE, sizeof(SSubmitBlk), tinfo.rowSize, &pTableMetaInfo->name, pTableMeta, &pTableDataBlock, NULL); if (ret != TSDB_CODE_SUCCESS) { pParentSql->res.code = TSDB_CODE_TSC_OUT_OF_MEMORY; @@ -1561,8 +1562,8 @@ void tscImportDataFromFile(SSqlObj *pSql) { return; } - assert(pCmd->dataSourceType == DATA_FROM_DATA_FILE && strlen(pCmd->payload) != 0); - pCmd->active = pCmd->pQueryInfo[0]; + assert(TSDB_QUERY_HAS_TYPE(pCmd->insertParam.insertType, TSDB_QUERY_TYPE_FILE_INSERT) && strlen(pCmd->payload) != 0); + pCmd->active = pCmd->pQueryInfo; SImportFileSupport *pSupporter = calloc(1, sizeof(SImportFileSupport)); SSqlObj *pNew = createSubqueryObj(pSql, 0, parseFileSendDataBlock, pSupporter, TSDB_SQL_INSERT, NULL); diff --git a/src/client/src/tscPrepare.c b/src/client/src/tscPrepare.c index c95003fc4df8d71d2c97e37d20d99957b8cdc60f..b80bdabbfcd0a13d20b661760443f0f16c704379 100644 --- a/src/client/src/tscPrepare.c +++ b/src/client/src/tscPrepare.c @@ -51,9 +51,9 @@ typedef struct SMultiTbStmt { uint32_t tbNum; SStrToken tbname; SStrToken stbname; - SStrToken values; + SStrToken values; SArray *tags; - SHashObj *pTableHash; + SHashObj *pTableHash; SHashObj *pTableBlockHashList; // data block for each table } SMultiTbStmt; @@ -287,9 +287,9 @@ static int fillColumnsNull(STableDataBlocks* pBlock, int32_t rowNum) { for (int32_t i = 0; i < spd->numOfCols; ++i) { if (!spd->cols[i].hasVal) { // current column do not have any value to insert, set it to null - for (int32_t n = 0; n < rowNum; ++n) { + for (int32_t n = 0; n < rowNum; ++n) { char *ptr = pBlock->pData + sizeof(SSubmitBlk) + pBlock->rowSize * n + offset; - + if (schema[i].type == TSDB_DATA_TYPE_BINARY) { varDataSetLen(ptr, sizeof(int8_t)); *(uint8_t*) varDataVal(ptr) = TSDB_DATA_BINARY_NULL; @@ -301,7 +301,7 @@ static int fillColumnsNull(STableDataBlocks* pBlock, int32_t rowNum) { } } } - + offset += schema[i].bytes; } @@ -312,7 +312,7 @@ static int fillColumnsNull(STableDataBlocks* pBlock, int32_t rowNum) { int32_t fillTablesColumnsNull(SSqlObj* pSql) { SSqlCmd* pCmd = &pSql->cmd; - STableDataBlocks** p = taosHashIterate(pCmd->pTableBlockHashList, NULL); + STableDataBlocks** p = taosHashIterate(pCmd->insertParam.pTableBlockHashList, NULL); STableDataBlocks* pOneTableBlock = *p; while(pOneTableBlock) { @@ -320,8 +320,8 @@ int32_t fillTablesColumnsNull(SSqlObj* pSql) { if (pBlocks->numOfRows > 0 && pOneTableBlock->boundColumnInfo.numOfBound < pOneTableBlock->boundColumnInfo.numOfCols) { fillColumnsNull(pOneTableBlock, pBlocks->numOfRows); } - - p = taosHashIterate(pCmd->pTableBlockHashList, p); + + p = taosHashIterate(pCmd->insertParam.pTableBlockHashList, p); if (p == NULL) { break; } @@ -779,7 +779,7 @@ static int doBindParam(STableDataBlocks* pBlock, char* data, SParamInfo* param, return TSDB_CODE_TSC_INVALID_VALUE; } } - + return TSDB_CODE_SUCCESS; } @@ -804,7 +804,7 @@ static int doBindBatchParam(STableDataBlocks* pBlock, SParamInfo* param, TAOS_MU if (!IS_VAR_DATA_TYPE(param->type)) { memcpy(data + param->offset, (char *)bind->buffer + bind->buffer_length * i, tDataTypes[param->type].bytes); - + if (param->offset == 0) { if (tsCheckTimestamp(pBlock, data + param->offset) != TSDB_CODE_SUCCESS) { tscError("invalid timestamp"); @@ -833,23 +833,23 @@ static int doBindBatchParam(STableDataBlocks* pBlock, SParamInfo* param, TAOS_MU varDataSetLen(data + param->offset, output); } } - + return TSDB_CODE_SUCCESS; } static int insertStmtBindParam(STscStmt* stmt, TAOS_BIND* bind) { SSqlCmd* pCmd = &stmt->pSql->cmd; STscStmt* pStmt = (STscStmt*)stmt; - + STableDataBlocks* pBlock = NULL; - + if (pStmt->multiTbInsert) { - if (pCmd->pTableBlockHashList == NULL) { + if (pCmd->insertParam.pTableBlockHashList == NULL) { tscError("0x%"PRIx64" Table block hash list is empty", pStmt->pSql->self); return TSDB_CODE_TSC_APP_ERROR; } - - STableDataBlocks** t1 = (STableDataBlocks**)taosHashGet(pCmd->pTableBlockHashList, (const char*)&pStmt->mtb.currentUid, sizeof(pStmt->mtb.currentUid)); + + STableDataBlocks** t1 = (STableDataBlocks**)taosHashGet(pCmd->insertParam.pTableBlockHashList, (const char*)&pStmt->mtb.currentUid, sizeof(pStmt->mtb.currentUid)); if (t1 == NULL) { tscError("0x%"PRIx64" no table data block in hash list, uid:%" PRId64 , pStmt->pSql->self, pStmt->mtb.currentUid); return TSDB_CODE_TSC_APP_ERROR; @@ -857,15 +857,15 @@ static int insertStmtBindParam(STscStmt* stmt, TAOS_BIND* bind) { pBlock = *t1; } else { - STableMetaInfo* pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, 0, 0); + STableMetaInfo* pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, 0); STableMeta* pTableMeta = pTableMetaInfo->pTableMeta; - if (pCmd->pTableBlockHashList == NULL) { - pCmd->pTableBlockHashList = taosHashInit(16, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BIGINT), true, false); + if (pCmd->insertParam.pTableBlockHashList == NULL) { + pCmd->insertParam.pTableBlockHashList = taosHashInit(16, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BIGINT), true, false); } int32_t ret = - tscGetDataBlockFromList(pCmd->pTableBlockHashList, pTableMeta->id.uid, TSDB_PAYLOAD_SIZE, sizeof(SSubmitBlk), + tscGetDataBlockFromList(pCmd->insertParam.pTableBlockHashList, pTableMeta->id.uid, TSDB_PAYLOAD_SIZE, sizeof(SSubmitBlk), pTableMeta->tableInfo.rowSize, &pTableMetaInfo->name, pTableMeta, &pBlock, NULL); if (ret != 0) { return ret; @@ -904,16 +904,16 @@ static int insertStmtBindParamBatch(STscStmt* stmt, TAOS_MULTI_BIND* bind, int c SSqlCmd* pCmd = &stmt->pSql->cmd; STscStmt* pStmt = (STscStmt*)stmt; int rowNum = bind->num; - + STableDataBlocks* pBlock = NULL; - + if (pStmt->multiTbInsert) { - if (pCmd->pTableBlockHashList == NULL) { + if (pCmd->insertParam.pTableBlockHashList == NULL) { tscError("0x%"PRIx64" Table block hash list is empty", pStmt->pSql->self); return TSDB_CODE_TSC_APP_ERROR; } - STableDataBlocks** t1 = (STableDataBlocks**)taosHashGet(pCmd->pTableBlockHashList, (const char*)&pStmt->mtb.currentUid, sizeof(pStmt->mtb.currentUid)); + STableDataBlocks** t1 = (STableDataBlocks**)taosHashGet(pCmd->insertParam.pTableBlockHashList, (const char*)&pStmt->mtb.currentUid, sizeof(pStmt->mtb.currentUid)); if (t1 == NULL) { tscError("0x%"PRIx64" no table data block in hash list, uid:%" PRId64 , pStmt->pSql->self, pStmt->mtb.currentUid); return TSDB_CODE_TSC_APP_ERROR; @@ -921,15 +921,15 @@ static int insertStmtBindParamBatch(STscStmt* stmt, TAOS_MULTI_BIND* bind, int c pBlock = *t1; } else { - STableMetaInfo* pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, 0, 0); + STableMetaInfo* pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, 0); STableMeta* pTableMeta = pTableMetaInfo->pTableMeta; - if (pCmd->pTableBlockHashList == NULL) { - pCmd->pTableBlockHashList = taosHashInit(16, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BIGINT), true, false); + if (pCmd->insertParam.pTableBlockHashList == NULL) { + pCmd->insertParam.pTableBlockHashList = taosHashInit(16, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BIGINT), true, false); } int32_t ret = - tscGetDataBlockFromList(pCmd->pTableBlockHashList, pTableMeta->id.uid, TSDB_PAYLOAD_SIZE, sizeof(SSubmitBlk), + tscGetDataBlockFromList(pCmd->insertParam.pTableBlockHashList, pTableMeta->id.uid, TSDB_PAYLOAD_SIZE, sizeof(SSubmitBlk), pTableMeta->tableInfo.rowSize, &pTableMetaInfo->name, pTableMeta, &pBlock, NULL); if (ret != 0) { return ret; @@ -958,7 +958,7 @@ static int insertStmtBindParamBatch(STscStmt* stmt, TAOS_MULTI_BIND* bind, int c tscError("0x%"PRIx64" param %d: num[%d:%d] not match", pStmt->pSql->self, param->idx, rowNum, bind[param->idx].num); return TSDB_CODE_TSC_INVALID_VALUE; } - + int code = doBindBatchParam(pBlock, param, &bind[param->idx], pCmd->batchSize); if (code != TSDB_CODE_SUCCESS) { tscError("0x%"PRIx64" bind column %d: type mismatch or invalid", pStmt->pSql->self, param->idx); @@ -969,7 +969,7 @@ static int insertStmtBindParamBatch(STscStmt* stmt, TAOS_MULTI_BIND* bind, int c pCmd->batchSize += rowNum - 1; } else { SParamInfo* param = &pBlock->params[colIdx]; - + int code = doBindBatchParam(pBlock, param, bind, pCmd->batchSize); if (code != TSDB_CODE_SUCCESS) { tscError("0x%"PRIx64" bind column %d: type mismatch or invalid", pStmt->pSql->self, param->idx); @@ -994,13 +994,12 @@ static int insertStmtUpdateBatch(STscStmt* stmt) { tscError("too many record:%d", pCmd->batchSize); return TSDB_CODE_TSC_APP_ERROR; } - - assert(pCmd->numOfClause == 1); - if (taosHashGetSize(pCmd->pTableBlockHashList) == 0) { + + if (taosHashGetSize(pCmd->insertParam.pTableBlockHashList) == 0) { return TSDB_CODE_SUCCESS; } - STableDataBlocks** t1 = (STableDataBlocks**)taosHashGet(pCmd->pTableBlockHashList, (const char*)&stmt->mtb.currentUid, sizeof(stmt->mtb.currentUid)); + STableDataBlocks** t1 = (STableDataBlocks**)taosHashGet(pCmd->insertParam.pTableBlockHashList, (const char*)&stmt->mtb.currentUid, sizeof(stmt->mtb.currentUid)); if (t1 == NULL) { tscError("0x%"PRIx64" no table data block in hash list, uid:%" PRId64 , pSql->self, stmt->mtb.currentUid); return TSDB_CODE_TSC_APP_ERROR; @@ -1023,11 +1022,11 @@ static int insertStmtUpdateBatch(STscStmt* stmt) { static int insertStmtAddBatch(STscStmt* stmt) { SSqlCmd* pCmd = &stmt->pSql->cmd; ++pCmd->batchSize; - + if (stmt->multiTbInsert) { return insertStmtUpdateBatch(stmt); } - + return TSDB_CODE_SUCCESS; } @@ -1036,9 +1035,9 @@ static int insertStmtReset(STscStmt* pStmt) { if (pCmd->batchSize > 2) { int32_t alloced = (pCmd->batchSize + 1) / 2; - size_t size = taosArrayGetSize(pCmd->pDataBlocks); + size_t size = taosArrayGetSize(pCmd->insertParam.pDataBlocks); for (int32_t i = 0; i < size; ++i) { - STableDataBlocks* pBlock = taosArrayGetP(pCmd->pDataBlocks, i); + STableDataBlocks* pBlock = taosArrayGetP(pCmd->insertParam.pDataBlocks, i); uint32_t totalDataSize = pBlock->size - sizeof(SSubmitBlk); pBlock->size = sizeof(SSubmitBlk) + totalDataSize / alloced; @@ -1049,7 +1048,7 @@ static int insertStmtReset(STscStmt* pStmt) { } pCmd->batchSize = 0; - STableMetaInfo* pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, pCmd->clauseIndex, 0); + STableMetaInfo* pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, 0); pTableMetaInfo->vgroupIndex = 0; return TSDB_CODE_SUCCESS; } @@ -1060,22 +1059,21 @@ static int insertStmtExecute(STscStmt* stmt) { return TSDB_CODE_TSC_INVALID_VALUE; } - assert(pCmd->numOfClause == 1); - if (taosHashGetSize(pCmd->pTableBlockHashList) == 0) { + if (taosHashGetSize(pCmd->insertParam.pTableBlockHashList) == 0) { return TSDB_CODE_SUCCESS; } - STableMetaInfo* pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, 0, 0); + STableMetaInfo* pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, 0); STableMeta* pTableMeta = pTableMetaInfo->pTableMeta; - if (pCmd->pTableBlockHashList == NULL) { - pCmd->pTableBlockHashList = taosHashInit(16, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BIGINT), true, false); + if (pCmd->insertParam.pTableBlockHashList == NULL) { + pCmd->insertParam.pTableBlockHashList = taosHashInit(16, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BIGINT), true, false); } STableDataBlocks* pBlock = NULL; int32_t ret = - tscGetDataBlockFromList(pCmd->pTableBlockHashList, pTableMeta->id.uid, TSDB_PAYLOAD_SIZE, sizeof(SSubmitBlk), + tscGetDataBlockFromList(pCmd->insertParam.pTableBlockHashList, pTableMeta->id.uid, TSDB_PAYLOAD_SIZE, sizeof(SSubmitBlk), pTableMeta->tableInfo.rowSize, &pTableMetaInfo->name, pTableMeta, &pBlock, NULL); assert(ret == 0); pBlock->size = sizeof(SSubmitBlk) + pCmd->batchSize * pBlock->rowSize; @@ -1092,7 +1090,7 @@ static int insertStmtExecute(STscStmt* stmt) { return code; } - STableDataBlocks* pDataBlock = taosArrayGetP(pCmd->pDataBlocks, 0); + STableDataBlocks* pDataBlock = taosArrayGetP(pCmd->insertParam.pDataBlocks, 0); code = tscCopyDataBlockToPayload(stmt->pSql, pDataBlock); if (code != TSDB_CODE_SUCCESS) { return code; @@ -1110,15 +1108,15 @@ static int insertStmtExecute(STscStmt* stmt) { // data block reset pCmd->batchSize = 0; - for(int32_t i = 0; i < pCmd->numOfTables; ++i) { - if (pCmd->pTableNameList && pCmd->pTableNameList[i]) { - tfree(pCmd->pTableNameList[i]); + for(int32_t i = 0; i < pCmd->insertParam.numOfTables; ++i) { + if (pCmd->insertParam.pTableNameList && pCmd->insertParam.pTableNameList[i]) { + tfree(pCmd->insertParam.pTableNameList[i]); } } - pCmd->numOfTables = 0; - tfree(pCmd->pTableNameList); - pCmd->pDataBlocks = tscDestroyBlockArrayList(pCmd->pDataBlocks); + pCmd->insertParam.numOfTables = 0; + tfree(pCmd->insertParam.pTableNameList); + pCmd->insertParam.pDataBlocks = tscDestroyBlockArrayList(pCmd->insertParam.pDataBlocks); return pSql->res.code; } @@ -1126,32 +1124,32 @@ static int insertStmtExecute(STscStmt* stmt) { static void insertBatchClean(STscStmt* pStmt) { SSqlCmd *pCmd = &pStmt->pSql->cmd; SSqlObj *pSql = pStmt->pSql; - int32_t size = taosHashGetSize(pCmd->pTableBlockHashList); - + int32_t size = taosHashGetSize(pCmd->insertParam.pTableBlockHashList); + // data block reset pCmd->batchSize = 0; - + for(int32_t i = 0; i < size; ++i) { - if (pCmd->pTableNameList && pCmd->pTableNameList[i]) { - tfree(pCmd->pTableNameList[i]); + if (pCmd->insertParam.pTableNameList && pCmd->insertParam.pTableNameList[i]) { + tfree(pCmd->insertParam.pTableNameList[i]); } } - tfree(pCmd->pTableNameList); + tfree(pCmd->insertParam.pTableNameList); /* - STableDataBlocks** p = taosHashIterate(pCmd->pTableBlockHashList, NULL); + STableDataBlocks** p = taosHashIterate(pCmd->insertParam.pTableBlockHashList, NULL); STableDataBlocks* pOneTableBlock = *p; - while (1) { + while (1) { SSubmitBlk* pBlocks = (SSubmitBlk*) pOneTableBlock->pData; - + pOneTableBlock->size = sizeof(SSubmitBlk); pBlocks->numOfRows = 0; - - p = taosHashIterate(pCmd->pTableBlockHashList, p); + + p = taosHashIterate(pCmd->insertParam.pTableBlockHashList, p); if (p == NULL) { break; } @@ -1160,27 +1158,27 @@ static void insertBatchClean(STscStmt* pStmt) { } */ - pCmd->pDataBlocks = tscDestroyBlockArrayList(pCmd->pDataBlocks); - pCmd->numOfTables = 0; + pCmd->insertParam.pDataBlocks = tscDestroyBlockArrayList(pCmd->insertParam.pDataBlocks); + pCmd->insertParam.numOfTables = 0; - taosHashEmpty(pCmd->pTableBlockHashList); + taosHashEmpty(pCmd->insertParam.pTableBlockHashList); tscFreeSqlResult(pSql); tscFreeSubobj(pSql); tfree(pSql->pSubs); - pSql->subState.numOfSub = 0; + pSql->subState.numOfSub = 0; } static int insertBatchStmtExecute(STscStmt* pStmt) { int32_t code = 0; - + if(pStmt->mtb.nameSet == false) { tscError("0x%"PRIx64" no table name set", pStmt->pSql->self); return TSDB_CODE_TSC_APP_ERROR; } - + pStmt->pSql->retry = pStmt->pSql->maxRetry + 1; //no retry - if (taosHashGetSize(pStmt->pSql->cmd.pTableBlockHashList) <= 0) { // merge according to vgId + if (taosHashGetSize(pStmt->pSql->cmd.insertParam.pTableBlockHashList) <= 0) { // merge according to vgId tscError("0x%"PRIx64" no data block to insert", pStmt->pSql->self); return TSDB_CODE_TSC_APP_ERROR; } @@ -1196,12 +1194,12 @@ static int insertBatchStmtExecute(STscStmt* pStmt) { if (code != TSDB_CODE_SUCCESS) { return code; } - + // wait for the callback function to post the semaphore tsem_wait(&pStmt->pSql->rspSem); insertBatchClean(pStmt); - + return pStmt->pSql->res.code; } @@ -1213,14 +1211,13 @@ int stmtParseInsertTbTags(SSqlObj* pSql, STscStmt* pStmt) { if ((ret = tsInsertInitialCheck(pSql)) != TSDB_CODE_SUCCESS) { return ret; } - + int32_t index = 0; SStrToken sToken = tStrGetToken(pCmd->curSql, &index, false); - if (sToken.n == 0) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } - + if (sToken.n == 1 && sToken.type == TK_QUESTION) { pStmt->multiTbInsert = true; pStmt->mtb.tbname = sToken; @@ -1228,7 +1225,7 @@ int stmtParseInsertTbTags(SSqlObj* pSql, STscStmt* pStmt) { if (pStmt->mtb.pTableHash == NULL) { pStmt->mtb.pTableHash = taosHashInit(16, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY), true, false); } - + if (pStmt->mtb.pTableBlockHashList == NULL) { pStmt->mtb.pTableBlockHashList = taosHashInit(16, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BIGINT), true, false); } @@ -1239,35 +1236,35 @@ int stmtParseInsertTbTags(SSqlObj* pSql, STscStmt* pStmt) { if (sToken.n > 0 && sToken.type == TK_VALUES) { return TSDB_CODE_SUCCESS; } - + if (sToken.n <= 0 || sToken.type != TK_USING) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } sToken = tStrGetToken(pCmd->curSql, &index, false); if (sToken.n <= 0 || ((sToken.type != TK_ID) && (sToken.type != TK_STRING))) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } pStmt->mtb.stbname = sToken; sToken = tStrGetToken(pCmd->curSql, &index, false); if (sToken.n <= 0 || sToken.type != TK_TAGS) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } sToken = tStrGetToken(pCmd->curSql, &index, false); if (sToken.n <= 0 || sToken.type != TK_LP) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } pStmt->mtb.tags = taosArrayInit(4, sizeof(SStrToken)); int32_t loopCont = 1; - + while (loopCont) { sToken = tStrGetToken(pCmd->curSql, &index, false); if (sToken.n <= 0) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } switch (sToken.type) { @@ -1275,7 +1272,7 @@ int stmtParseInsertTbTags(SSqlObj* pSql, STscStmt* pStmt) { loopCont = 0; break; case TK_VALUES: - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; case TK_QUESTION: pStmt->mtb.tagSet = false; //continue default: @@ -1285,12 +1282,12 @@ int stmtParseInsertTbTags(SSqlObj* pSql, STscStmt* pStmt) { } if (taosArrayGetSize(pStmt->mtb.tags) <= 0) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } sToken = tStrGetToken(pCmd->curSql, &index, false); if (sToken.n <= 0 || sToken.type != TK_VALUES) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } pStmt->mtb.values = sToken; @@ -1320,7 +1317,7 @@ int stmtGenInsertStatement(SSqlObj* pSql, STscStmt* pStmt, const char* name, TAO } j = 0; - + for (size_t i = 0; i < tagNum && len < (size - 1); ++i) { SStrToken *t = taosArrayGet(pStmt->mtb.tags, i); if (t->type == TK_QUESTION) { @@ -1342,12 +1339,12 @@ int stmtGenInsertStatement(SSqlObj* pSql, STscStmt* pStmt, const char* name, TAO } ++j; - + if (ret) { free(str); return ret; } - + len += l; } else { len += (size_t)snprintf(str + len, size - len - 1, i > 0 ? ",%.*s" : "%.*s", t->n, t->z); @@ -1360,7 +1357,7 @@ int stmtGenInsertStatement(SSqlObj* pSql, STscStmt* pStmt, const char* name, TAO str = calloc(1, size); continue; } - + strcat(str, ") "); len += 2; @@ -1372,7 +1369,7 @@ int stmtGenInsertStatement(SSqlObj* pSql, STscStmt* pStmt, const char* name, TAO } strcat(str, pStmt->mtb.values.z); - + break; } @@ -1404,6 +1401,7 @@ TAOS_STMT* taos_stmt_init(TAOS* taos) { pStmt->taos = pObj; SSqlObj* pSql = calloc(1, sizeof(SSqlObj)); + if (pSql == NULL) { free(pStmt); terrno = TSDB_CODE_TSC_OUT_OF_MEMORY; @@ -1436,7 +1434,7 @@ int taos_stmt_prepare(TAOS_STMT* stmt, const char* sql, unsigned long length) { } pStmt->last = STMT_PREPARE; - + SSqlObj* pSql = pStmt->pSql; size_t sqlLen = strlen(sql); @@ -1446,7 +1444,7 @@ int taos_stmt_prepare(TAOS_STMT* stmt, const char* sql, unsigned long length) { pSql->fp = waitForQueryRsp; pSql->fetchFp = waitForQueryRsp; - pCmd->insertType = TSDB_QUERY_TYPE_STMT_INSERT; + pCmd->insertParam.insertType = TSDB_QUERY_TYPE_STMT_INSERT; if (TSDB_CODE_SUCCESS != tscAllocPayload(pCmd, TSDB_DEFAULT_PAYLOAD_SIZE)) { tscError("%p failed to malloc payload buffer", pSql); @@ -1500,8 +1498,6 @@ int taos_stmt_prepare(TAOS_STMT* stmt, const char* sql, unsigned long length) { return normalStmtPrepare(pStmt); } - - int taos_stmt_set_tbname_tags(TAOS_STMT* stmt, const char* name, TAOS_BIND* tags) { STscStmt* pStmt = (STscStmt*)stmt; SSqlObj* pSql = pStmt->pSql; @@ -1541,11 +1537,11 @@ int taos_stmt_set_tbname_tags(TAOS_STMT* stmt, const char* name, TAOS_BIND* tags return TSDB_CODE_TSC_APP_ERROR; } - SSubmitBlk* pBlk = (SSubmitBlk*) (*t1)->pData; + SSubmitBlk* pBlk = (SSubmitBlk*) (*t1)->pData; pCmd->batchSize = pBlk->numOfRows; - taosHashPut(pCmd->pTableBlockHashList, (void *)&pStmt->mtb.currentUid, sizeof(pStmt->mtb.currentUid), (void*)t1, POINTER_BYTES); - + taosHashPut(pCmd->insertParam.pTableBlockHashList, (void *)&pStmt->mtb.currentUid, sizeof(pStmt->mtb.currentUid), (void*)t1, POINTER_BYTES); + tscDebug("0x%"PRIx64" table:%s is already prepared, uid:%" PRIu64, pSql->self, name, pStmt->mtb.currentUid); return TSDB_CODE_SUCCESS; } @@ -1557,42 +1553,42 @@ int taos_stmt_set_tbname_tags(TAOS_STMT* stmt, const char* name, TAOS_BIND* tags tscError("No tags set"); return TSDB_CODE_TSC_APP_ERROR; } - + int32_t ret = stmtGenInsertStatement(pSql, pStmt, name, tags); if (ret != TSDB_CODE_SUCCESS) { return ret; } } - + pStmt->mtb.nameSet = true; pStmt->mtb.tagSet = true; tscDebug("0x%"PRIx64" SQL: %s", pSql->self, pSql->sqlstr); - pSql->cmd.parseFinished = 0; pSql->cmd.numOfParams = 0; pSql->cmd.batchSize = 0; - if (taosHashGetSize(pCmd->pTableBlockHashList) > 0) { - SHashObj* hashList = pCmd->pTableBlockHashList; - pCmd->pTableBlockHashList = NULL; + if (taosHashGetSize(pCmd->insertParam.pTableBlockHashList) > 0) { + SHashObj* hashList = pCmd->insertParam.pTableBlockHashList; + pCmd->insertParam.pTableBlockHashList = NULL; tscResetSqlCmd(pCmd, true); - pCmd->pTableBlockHashList = hashList; + pCmd->insertParam.pTableBlockHashList = hashList; } - + int32_t code = tsParseSql(pStmt->pSql, true); if (code == TSDB_CODE_TSC_ACTION_IN_PROGRESS) { // wait for the callback function to post the semaphore tsem_wait(&pStmt->pSql->rspSem); - + code = pStmt->pSql->res.code; } if (code == TSDB_CODE_SUCCESS) { - STableMetaInfo* pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, 0, 0); + STableMetaInfo* pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, 0); + STableMeta* pTableMeta = pTableMetaInfo->pTableMeta; STableDataBlocks* pBlock = NULL; - code = tscGetDataBlockFromList(pCmd->pTableBlockHashList, pTableMeta->id.uid, TSDB_PAYLOAD_SIZE, sizeof(SSubmitBlk), + code = tscGetDataBlockFromList(pCmd->insertParam.pTableBlockHashList, pTableMeta->id.uid, TSDB_PAYLOAD_SIZE, sizeof(SSubmitBlk), pTableMeta->tableInfo.rowSize, &pTableMetaInfo->name, pTableMeta, &pBlock, NULL); if (code != TSDB_CODE_SUCCESS) { return code; @@ -1605,12 +1601,11 @@ int taos_stmt_set_tbname_tags(TAOS_STMT* stmt, const char* name, TAOS_BIND* tags pStmt->mtb.tbNum++; taosHashPut(pStmt->mtb.pTableBlockHashList, (void *)&pStmt->mtb.currentUid, sizeof(pStmt->mtb.currentUid), (void*)&pBlock, POINTER_BYTES); - taosHashPut(pStmt->mtb.pTableHash, name, strlen(name), (char*) &pTableMeta->id.uid, sizeof(pTableMeta->id.uid)); tscDebug("0x%"PRIx64" table:%s is prepared, uid:%" PRIx64, pSql->self, name, pStmt->mtb.currentUid); } - + return code; } @@ -1636,8 +1631,8 @@ int taos_stmt_close(TAOS_STMT* stmt) { if (pStmt->multiTbInsert) { taosHashCleanup(pStmt->mtb.pTableHash); pStmt->mtb.pTableBlockHashList = tscDestroyBlockHashTable(pStmt->mtb.pTableBlockHashList, true); - taosHashCleanup(pStmt->pSql->cmd.pTableBlockHashList); - pStmt->pSql->cmd.pTableBlockHashList = NULL; + taosHashCleanup(pStmt->pSql->cmd.insertParam.pTableBlockHashList); + pStmt->pSql->cmd.insertParam.pTableBlockHashList = NULL; taosArrayDestroy(pStmt->mtb.tags); } } @@ -1653,7 +1648,7 @@ int taos_stmt_bind_param(TAOS_STMT* stmt, TAOS_BIND* bind) { terrno = TSDB_CODE_TSC_DISCONNECTED; return TSDB_CODE_TSC_DISCONNECTED; } - + if (pStmt->isInsert) { if (pStmt->multiTbInsert) { if (pStmt->last != STMT_SETTBNAME && pStmt->last != STMT_ADD_BATCH) { @@ -1668,7 +1663,7 @@ int taos_stmt_bind_param(TAOS_STMT* stmt, TAOS_BIND* bind) { } pStmt->last = STMT_BIND; - + return insertStmtBindParam(pStmt, bind); } else { return normalStmtBindParam(pStmt, bind); @@ -1688,7 +1683,7 @@ int taos_stmt_bind_param_batch(TAOS_STMT* stmt, TAOS_MULTI_BIND* bind) { tscError("0x%"PRIx64" invalid parameter", pStmt->pSql->self); return TSDB_CODE_TSC_APP_ERROR; } - + if (!pStmt->isInsert) { tscError("0x%"PRIx64" not or invalid batch insert", pStmt->pSql->self); return TSDB_CODE_TSC_APP_ERROR; @@ -1698,7 +1693,7 @@ int taos_stmt_bind_param_batch(TAOS_STMT* stmt, TAOS_MULTI_BIND* bind) { if (pStmt->last != STMT_SETTBNAME && pStmt->last != STMT_ADD_BATCH) { tscError("0x%"PRIx64" bind param status error, last:%d", pStmt->pSql->self, pStmt->last); return TSDB_CODE_TSC_APP_ERROR; - } + } } else { if (pStmt->last != STMT_PREPARE && pStmt->last != STMT_ADD_BATCH && pStmt->last != STMT_EXECUTE) { tscError("0x%"PRIx64" bind param status error, last:%d", pStmt->pSql->self, pStmt->last); @@ -1707,7 +1702,7 @@ int taos_stmt_bind_param_batch(TAOS_STMT* stmt, TAOS_MULTI_BIND* bind) { } pStmt->last = STMT_BIND; - + return insertStmtBindParamBatch(pStmt, bind, -1); } @@ -1732,7 +1727,7 @@ int taos_stmt_bind_single_param_batch(TAOS_STMT* stmt, TAOS_MULTI_BIND* bind, in if (pStmt->last != STMT_SETTBNAME && pStmt->last != STMT_ADD_BATCH && pStmt->last != STMT_BIND_COL) { tscError("0x%"PRIx64" bind param status error, last:%d", pStmt->pSql->self, pStmt->last); return TSDB_CODE_TSC_APP_ERROR; - } + } } else { if (pStmt->last != STMT_PREPARE && pStmt->last != STMT_ADD_BATCH && pStmt->last != STMT_BIND_COL && pStmt->last != STMT_EXECUTE) { tscError("0x%"PRIx64" bind param status error, last:%d", pStmt->pSql->self, pStmt->last); @@ -1753,7 +1748,7 @@ int taos_stmt_add_batch(TAOS_STMT* stmt) { terrno = TSDB_CODE_TSC_DISCONNECTED; return TSDB_CODE_TSC_DISCONNECTED; } - + if (pStmt->isInsert) { if (pStmt->last != STMT_BIND && pStmt->last != STMT_BIND_COL) { tscError("0x%"PRIx64" add batch status error, last:%d", pStmt->pSql->self, pStmt->last); @@ -1761,10 +1756,10 @@ int taos_stmt_add_batch(TAOS_STMT* stmt) { } pStmt->last = STMT_ADD_BATCH; - + return insertStmtAddBatch(pStmt); } - + return TSDB_CODE_COM_OPS_NOT_SUPPORT; } @@ -1783,7 +1778,7 @@ int taos_stmt_execute(TAOS_STMT* stmt) { terrno = TSDB_CODE_TSC_DISCONNECTED; return TSDB_CODE_TSC_DISCONNECTED; } - + if (pStmt->isInsert) { if (pStmt->last != STMT_ADD_BATCH) { tscError("0x%"PRIx64" exec status error, last:%d", pStmt->pSql->self, pStmt->last); @@ -1791,7 +1786,7 @@ int taos_stmt_execute(TAOS_STMT* stmt) { } pStmt->last = STMT_EXECUTE; - + if (pStmt->multiTbInsert) { ret = insertBatchStmtExecute(pStmt); } else { @@ -1803,9 +1798,10 @@ int taos_stmt_execute(TAOS_STMT* stmt) { ret = TSDB_CODE_TSC_OUT_OF_MEMORY; } else { if (pStmt->pSql != NULL) { - taos_free_result(pStmt->pSql); + tscFreeSqlObj(pStmt->pSql); pStmt->pSql = NULL; } + pStmt->pSql = taos_query((TAOS*)pStmt->taos, sql); ret = taos_errno(pStmt->pSql); free(sql); @@ -1875,16 +1871,16 @@ int taos_stmt_get_param(TAOS_STMT *stmt, int idx, int *type, int *bytes) { if (pStmt->isInsert) { SSqlCmd* pCmd = &pStmt->pSql->cmd; - STableMetaInfo* pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, 0, 0); + STableMetaInfo* pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, 0); STableMeta* pTableMeta = pTableMetaInfo->pTableMeta; - if (pCmd->pTableBlockHashList == NULL) { - pCmd->pTableBlockHashList = taosHashInit(16, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BIGINT), true, false); + if (pCmd->insertParam.pTableBlockHashList == NULL) { + pCmd->insertParam.pTableBlockHashList = taosHashInit(16, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BIGINT), true, false); } STableDataBlocks* pBlock = NULL; int32_t ret = - tscGetDataBlockFromList(pCmd->pTableBlockHashList, pTableMeta->id.uid, TSDB_PAYLOAD_SIZE, sizeof(SSubmitBlk), + tscGetDataBlockFromList(pCmd->insertParam.pTableBlockHashList, pTableMeta->id.uid, TSDB_PAYLOAD_SIZE, sizeof(SSubmitBlk), pTableMeta->tableInfo.rowSize, &pTableMetaInfo->name, pTableMeta, &pBlock, NULL); if (ret != 0) { // todo handle error diff --git a/src/client/src/tscSQLParser.c b/src/client/src/tscSQLParser.c index acfc1b0cf54e4c6f4485c98d4e583bfc12475f4c..d9be43b412342ec810cb07b1a29d3beaaa75cd89 100644 --- a/src/client/src/tscSQLParser.c +++ b/src/client/src/tscSQLParser.c @@ -35,6 +35,7 @@ #include "ttokendef.h" #include "ttype.h" #include "qUtil.h" +#include "qPlan.h" #define DEFAULT_PRIMARY_TIMESTAMP_COL_NAME "_c0" @@ -57,7 +58,7 @@ typedef struct SConvertFunc { int32_t execFuncId; } SConvertFunc; -static SExprInfo* doAddProjectCol(SQueryInfo* pQueryInfo, int32_t colIndex, int32_t tableIndex); +static SExprInfo* doAddProjectCol(SQueryInfo* pQueryInfo, int32_t colIndex, int32_t tableIndex, int32_t colId); static int32_t setShowInfo(SSqlObj* pSql, SSqlInfo* pInfo); static char* getAccountId(SSqlObj* pSql); @@ -70,7 +71,7 @@ static bool validateTagParams(SArray* pTagsList, SArray* pFieldList, SSqlCmd* pC static int32_t setObjFullName(char* fullName, const char* account, SStrToken* pDB, SStrToken* tableName, int32_t* len); -static void getColumnName(tSqlExprItem* pItem, char* resultFieldName, int32_t nameLength); +static void getColumnName(tSqlExprItem* pItem, char* resultFieldName, char* rawName, int32_t nameLength); static int32_t addExprAndResultField(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, int32_t colIndex, tSqlExprItem* pItem, bool finalResult); static int32_t insertResultField(SQueryInfo* pQueryInfo, int32_t outputIndex, SColumnList* pIdList, int16_t bytes, @@ -111,7 +112,7 @@ static bool validateOneTags(SSqlCmd* pCmd, TAOS_FIELD* pTagField); static bool hasTimestampForPointInterpQuery(SQueryInfo* pQueryInfo); static bool hasNormalColumnFilter(SQueryInfo* pQueryInfo); -static int32_t validateLimitNode(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, int32_t index, SSqlNode* pSqlNode, SSqlObj* pSql); +static int32_t validateLimitNode(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SSqlNode* pSqlNode, SSqlObj* pSql); static int32_t parseCreateDBOptions(SSqlCmd* pCmd, SCreateDbInfo* pCreateDbSql); static int32_t getColumnIndexByName(SSqlCmd* pCmd, const SStrToken* pToken, SQueryInfo* pQueryInfo, SColumnIndex* pIndex); static int32_t getTableIndexByName(SStrToken* pToken, SQueryInfo* pQueryInfo, SColumnIndex* pIndex); @@ -126,17 +127,18 @@ static SColumnList createColumnList(int32_t num, int16_t tableIndex, int32_t col static int32_t doCheckForCreateTable(SSqlObj* pSql, int32_t subClauseIndex, SSqlInfo* pInfo); static int32_t doCheckForCreateFromStable(SSqlObj* pSql, SSqlInfo* pInfo); static int32_t doCheckForStream(SSqlObj* pSql, SSqlInfo* pInfo); -static int32_t validateSqlNode(SSqlObj* pSql, SSqlNode* pSqlNode, int32_t index); +static int32_t validateSqlNode(SSqlObj* pSql, SSqlNode* pSqlNode, SQueryInfo* pQueryInfo); static int32_t exprTreeFromSqlExpr(SSqlCmd* pCmd, tExprNode **pExpr, const tSqlExpr* pSqlExpr, SQueryInfo* pQueryInfo, SArray* pCols, uint64_t *uid); static bool validateDebugFlag(int32_t v); static int32_t checkQueryRangeForFill(SSqlCmd* pCmd, SQueryInfo* pQueryInfo); +static int32_t loadAllTableMeta(SSqlObj* pSql, struct SSqlInfo* pInfo); static bool isTimeWindowQuery(SQueryInfo* pQueryInfo) { return pQueryInfo->interval.interval > 0 || pQueryInfo->sessionWindow.gap > 0; } -int16_t getNewResColId(SQueryInfo* pQueryInfo) { - return pQueryInfo->resColumnId--; +int16_t getNewResColId(SSqlCmd* pCmd) { + return pCmd->resColumnId--; } static uint8_t convertOptr(SStrToken *pToken) { @@ -192,7 +194,7 @@ static bool validateDebugFlag(int32_t v) { * Used during parsing query sql. Since the query sql usually small in length, error position * is not needed in the final error message. */ -static int32_t invalidSqlErrMsg(char* dstBuffer, const char* errMsg) { +static int32_t invalidOperationMsg(char* dstBuffer, const char* errMsg) { return tscInvalidSQLErrMsg(dstBuffer, errMsg, NULL); } @@ -208,11 +210,11 @@ static int setColumnFilterInfoForTimestamp(SSqlCmd* pCmd, SQueryInfo* pQueryInfo if (seg != NULL) { if (taosParseTime(pVar->pz, &time, pVar->nLen, tinfo.precision, tsDaylight) != TSDB_CODE_SUCCESS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg); } } else { if (tVariantDump(pVar, (char*)&time, TSDB_DATA_TYPE_BIGINT, true)) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg); } } @@ -228,24 +230,56 @@ static int32_t handlePassword(SSqlCmd* pCmd, SStrToken* pPwd) { const char* msg3 = "password needs single quote marks enclosed"; if (pPwd->type != TK_STRING) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg3); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); } strdequote(pPwd->z); pPwd->n = (uint32_t)strtrim(pPwd->z); // trim space before and after passwords if (pPwd->n <= 0) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } if (pPwd->n >= TSDB_KEY_LEN) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); } return TSDB_CODE_SUCCESS; } -int32_t tscToSQLCmd(SSqlObj* pSql, struct SSqlInfo* pInfo) { +// validate the out put field type for "UNION ALL" subclause +static int32_t normalizeVarDataTypeLength(SSqlCmd* pCmd) { + const char* msg1 = "columns in select clause not identical"; + + int32_t diffSize = 0; + + // if there is only one element, the limit of clause is the limit of global result. + SQueryInfo* pQueryInfo1 = pCmd->pQueryInfo; + SQueryInfo* pSibling = pQueryInfo1->sibling; + + while(pSibling != NULL) { + int32_t ret = tscFieldInfoCompare(&pQueryInfo1->fieldsInfo, &pSibling->fieldsInfo, &diffSize); + if (ret != 0) { + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); + } + + pSibling = pSibling->sibling; + } + + if (diffSize) { + pQueryInfo1 = pCmd->pQueryInfo; + pSibling = pQueryInfo1->sibling; + + while(pSibling->sibling != NULL) { + tscFieldInfoSetSize(&pQueryInfo1->fieldsInfo, &pSibling->fieldsInfo); + pSibling = pSibling->sibling; + } + } + + return TSDB_CODE_SUCCESS; +} + +int32_t tscValidateSqlInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) { if (pInfo == NULL || pSql == NULL) { return TSDB_CODE_TSC_APP_ERROR; } @@ -259,7 +293,7 @@ int32_t tscToSQLCmd(SSqlObj* pSql, struct SSqlInfo* pInfo) { return tscSQLSyntaxErrMsg(tscGetErrorMsgPayload(pCmd), NULL, pInfo->msg); } - SQueryInfo* pQueryInfo = tscGetQueryInfoS(pCmd, pCmd->clauseIndex); + SQueryInfo* pQueryInfo = tscGetQueryInfoS(pCmd); if (pQueryInfo == NULL) { pRes->code = terrno; return pRes->code; @@ -281,37 +315,36 @@ int32_t tscToSQLCmd(SSqlObj* pSql, struct SSqlInfo* pInfo) { case TSDB_SQL_DROP_DB: { const char* msg2 = "invalid name"; const char* msg3 = "param name too long"; - const char* msg4 = "table is not super table"; SStrToken* pzName = taosArrayGet(pInfo->pMiscInfo->a, 0); if ((pInfo->type != TSDB_SQL_DROP_DNODE) && (tscValidateName(pzName) != TSDB_CODE_SUCCESS)) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); } if (pInfo->type == TSDB_SQL_DROP_DB) { assert(taosArrayGetSize(pInfo->pMiscInfo->a) == 1); code = tNameSetDbName(&pTableMetaInfo->name, getAccountId(pSql), pzName); if (code != TSDB_CODE_SUCCESS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); } } else if (pInfo->type == TSDB_SQL_DROP_TABLE) { assert(taosArrayGetSize(pInfo->pMiscInfo->a) == 1); - code = tscSetTableFullName(pTableMetaInfo, pzName, pSql); + code = tscSetTableFullName(&pTableMetaInfo->name, pzName, pSql); if(code != TSDB_CODE_SUCCESS) { return code; } if (pInfo->pMiscInfo->tableType == TSDB_SUPER_TABLE) { - code = tscGetTableMeta(pSql, pTableMetaInfo); - if (code != TSDB_CODE_SUCCESS) { - return code; - } - - if (!UTIL_TABLE_IS_SUPER_TABLE(pTableMetaInfo)) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg4); - } +//// code = tscGetTableMeta(pSql, pTableMetaInfo); +//// if (code != TSDB_CODE_SUCCESS) { +//// return code; +//// } +// +// if (!UTIL_TABLE_IS_SUPER_TABLE(pTableMetaInfo)) { +// return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg4); +// } } } else if (pInfo->type == TSDB_SQL_DROP_DNODE) { @@ -319,7 +352,7 @@ int32_t tscToSQLCmd(SSqlObj* pSql, struct SSqlInfo* pInfo) { strncpy(pCmd->payload, pzName->z, pzName->n); } else { // drop user/account if (pzName->n >= TSDB_USER_LEN) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg3); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); } strncpy(pCmd->payload, pzName->z, pzName->n); @@ -333,12 +366,12 @@ int32_t tscToSQLCmd(SSqlObj* pSql, struct SSqlInfo* pInfo) { SStrToken* pToken = taosArrayGet(pInfo->pMiscInfo->a, 0); if (tscValidateName(pToken) != TSDB_CODE_SUCCESS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg); } int32_t ret = tNameSetDbName(&pTableMetaInfo->name, getAccountId(pSql), pToken); if (ret != TSDB_CODE_SUCCESS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg); } break; @@ -350,7 +383,7 @@ int32_t tscToSQLCmd(SSqlObj* pSql, struct SSqlInfo* pInfo) { case TSDB_SQL_SHOW: { if (setShowInfo(pSql, pInfo) != TSDB_CODE_SUCCESS) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } break; @@ -363,16 +396,16 @@ int32_t tscToSQLCmd(SSqlObj* pSql, struct SSqlInfo* pInfo) { SCreateDbInfo* pCreateDB = &(pInfo->pMiscInfo->dbOpt); if (tscValidateName(&pCreateDB->dbname) != TSDB_CODE_SUCCESS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } int32_t ret = tNameSetDbName(&pTableMetaInfo->name, getAccountId(pSql), &(pCreateDB->dbname)); if (ret != TSDB_CODE_SUCCESS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); } if (parseCreateDBOptions(pCmd, pCreateDB) != TSDB_CODE_SUCCESS) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } break; @@ -382,7 +415,7 @@ int32_t tscToSQLCmd(SSqlObj* pSql, struct SSqlInfo* pInfo) { const char* msg = "invalid host name (ip address)"; if (taosArrayGetSize(pInfo->pMiscInfo->a) > 1) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg); } SStrToken* id = taosArrayGet(pInfo->pMiscInfo->a, 0); @@ -400,15 +433,15 @@ int32_t tscToSQLCmd(SSqlObj* pSql, struct SSqlInfo* pInfo) { SStrToken* pPwd = &pInfo->pMiscInfo->user.passwd; if (handlePassword(pCmd, pPwd) != TSDB_CODE_SUCCESS) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } if (pName->n >= TSDB_USER_LEN) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg3); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); } if (tscValidateName(pName) != TSDB_CODE_SUCCESS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); } SCreateAcctInfo* pAcctOpt = &pInfo->pMiscInfo->acctOpt; @@ -418,7 +451,7 @@ int32_t tscToSQLCmd(SSqlObj* pSql, struct SSqlInfo* pInfo) { } else if (strncmp(pAcctOpt->stat.z, "all", 3) == 0 && pAcctOpt->stat.n == 3) { } else if (strncmp(pAcctOpt->stat.z, "no", 2) == 0 && pAcctOpt->stat.n == 2) { } else { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } } @@ -430,26 +463,26 @@ int32_t tscToSQLCmd(SSqlObj* pSql, struct SSqlInfo* pInfo) { SStrToken* pToken = taosArrayGet(pInfo->pMiscInfo->a, 0); if (tscValidateName(pToken) != TSDB_CODE_SUCCESS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } // additional msg has been attached already - code = tscSetTableFullName(pTableMetaInfo, pToken, pSql); + code = tscSetTableFullName(&pTableMetaInfo->name, pToken, pSql); if (code != TSDB_CODE_SUCCESS) { return code; } return tscGetTableMeta(pSql, pTableMetaInfo); } - case TSDB_SQL_SHOW_CREATE_STABLE: + case TSDB_SQL_SHOW_CREATE_STABLE: case TSDB_SQL_SHOW_CREATE_TABLE: { const char* msg1 = "invalid table name"; SStrToken* pToken = taosArrayGet(pInfo->pMiscInfo->a, 0); if (tscValidateName(pToken) != TSDB_CODE_SUCCESS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } - code = tscSetTableFullName(pTableMetaInfo, pToken, pSql); + code = tscSetTableFullName(&pTableMetaInfo->name, pToken, pSql); if (code != TSDB_CODE_SUCCESS) { return code; } @@ -461,11 +494,11 @@ int32_t tscToSQLCmd(SSqlObj* pSql, struct SSqlInfo* pInfo) { SStrToken* pToken = taosArrayGet(pInfo->pMiscInfo->a, 0); if (tscValidateName(pToken) != TSDB_CODE_SUCCESS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } if (pToken->n > TSDB_DB_NAME_LEN) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } return tNameSetDbName(&pTableMetaInfo->name, getAccountId(pSql), pToken); } @@ -478,7 +511,7 @@ int32_t tscToSQLCmd(SSqlObj* pSql, struct SSqlInfo* pInfo) { /* validate the parameter names and options */ if (validateDNodeConfig(pMiscInfo) != TSDB_CODE_SUCCESS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); } char* pMsg = pCmd->payload; @@ -492,7 +525,7 @@ int32_t tscToSQLCmd(SSqlObj* pSql, struct SSqlInfo* pInfo) { strncpy(pCfg->ep, t0->z, t0->n); if (validateEp(pCfg->ep) != TSDB_CODE_SUCCESS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg3); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); } strncpy(pCfg->config, t1->z, t1->n); @@ -521,21 +554,21 @@ int32_t tscToSQLCmd(SSqlObj* pSql, struct SSqlInfo* pInfo) { SStrToken* pPwd = &pUser->passwd; if (pName->n >= TSDB_USER_LEN) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg3); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); } if (tscValidateName(pName) != TSDB_CODE_SUCCESS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); } if (pCmd->command == TSDB_SQL_CREATE_USER) { if (handlePassword(pCmd, pPwd) != TSDB_CODE_SUCCESS) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } } else { if (pUser->type == TSDB_ALTER_USER_PASSWD) { if (handlePassword(pCmd, pPwd) != TSDB_CODE_SUCCESS) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } } else if (pUser->type == TSDB_ALTER_USER_PRIVILEGES) { assert(pPwd->type == TSDB_DATA_TYPE_NULL); @@ -549,10 +582,10 @@ int32_t tscToSQLCmd(SSqlObj* pSql, struct SSqlInfo* pInfo) { } else if (strncasecmp(pPrivilege->z, "write", 5) == 0 && pPrivilege->n == 5) { pCmd->count = 3; } else { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg5); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg5); } } else { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg7); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg7); } } @@ -565,7 +598,7 @@ int32_t tscToSQLCmd(SSqlObj* pSql, struct SSqlInfo* pInfo) { // validate the parameter names and options if (validateLocalConfig(pMiscInfo) != TSDB_CODE_SUCCESS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg); } int32_t numOfToken = (int32_t) taosArrayGetSize(pMiscInfo->a); @@ -605,56 +638,41 @@ int32_t tscToSQLCmd(SSqlObj* pSql, struct SSqlInfo* pInfo) { } case TSDB_SQL_SELECT: { - const char* msg1 = "columns in select clause not identical"; - - size_t size = taosArrayGetSize(pInfo->list); - for (int32_t i = pCmd->numOfClause; i < size; ++i) { - SQueryInfo* p = tscGetQueryInfoS(pCmd, i); - if (p == NULL) { - pRes->code = terrno; - return pRes->code; - } + code = loadAllTableMeta(pSql, pInfo); + if (code != TSDB_CODE_SUCCESS) { + return code; } - assert(pCmd->numOfClause == size); - for (int32_t i = pCmd->clauseIndex; i < size; ++i) { + pQueryInfo = tscGetQueryInfo(pCmd); + + size_t size = taosArrayGetSize(pInfo->list); + for (int32_t i = 0; i < size; ++i) { SSqlNode* pSqlNode = taosArrayGetP(pInfo->list, i); - tscTrace("%p start to parse %dth subclause, total:%d", pSql, i, (int32_t) size); - if ((code = validateSqlNode(pSql, pSqlNode, i)) != TSDB_CODE_SUCCESS) { + + tscTrace("%p start to parse %dth subclause, total:%"PRIzu, pSql, i, size); + if ((code = validateSqlNode(pSql, pSqlNode, pQueryInfo)) != TSDB_CODE_SUCCESS) { return code; } tscPrintSelNodeList(pSql, i); - pCmd->clauseIndex += 1; - } - // restore the clause index - pCmd->clauseIndex = 0; + if ((i + 1) < size && pQueryInfo->sibling == NULL) { + if ((code = tscAddQueryInfo(pCmd)) != TSDB_CODE_SUCCESS) { + return code; + } - // set the command/global limit parameters from the first subclause to the sqlcmd object - SQueryInfo* pQueryInfo1 = tscGetQueryInfo(pCmd, 0); - pCmd->command = pQueryInfo1->command; - int32_t diffSize = 0; - - // if there is only one element, the limit of clause is the limit of global result. - // validate the select node for "UNION ALL" subclause - for (int32_t i = 1; i < pCmd->numOfClause; ++i) { - SQueryInfo* pQueryInfo2 = tscGetQueryInfo(pCmd, i); - - int32_t ret = tscFieldInfoCompare(&pQueryInfo1->fieldsInfo, &pQueryInfo2->fieldsInfo, &diffSize); - if (ret != 0) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + pQueryInfo = pCmd->active; } } - if (diffSize) { - for (int32_t i = 1; i < pCmd->numOfClause; ++i) { - SQueryInfo* pQueryInfo2 = tscGetQueryInfo(pCmd, i); - tscFieldInfoSetSize(&pQueryInfo1->fieldsInfo, &pQueryInfo2->fieldsInfo); - } + if ((code = normalizeVarDataTypeLength(pCmd)) != TSDB_CODE_SUCCESS) { + return code; } - pCmd->parseFinished = 1; + // set the command/global limit parameters from the first subclause to the sqlcmd object + pCmd->active = pCmd->pQueryInfo; + pCmd->command = pCmd->pQueryInfo->command; + return TSDB_CODE_SUCCESS; // do not build query message here } @@ -682,20 +700,19 @@ int32_t tscToSQLCmd(SSqlObj* pSql, struct SSqlInfo* pInfo) { assert(taosArrayGetSize(pInfo->pMiscInfo->a) == 1); code = tNameSetDbName(&pTableMetaInfo->name, getAccountId(pSql), pzName); if (code != TSDB_CODE_SUCCESS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } break; } default: - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), "not support sql expression"); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), "not support sql expression"); } - pSql->cmd.parseFinished = 1; if (tscBuildMsg[pCmd->command] != NULL) { return tscBuildMsg[pCmd->command](pSql, pInfo); } else { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), "not support sql expression"); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), "not support sql expression"); } } @@ -704,10 +721,10 @@ int32_t tscToSQLCmd(SSqlObj* pSql, struct SSqlInfo* pInfo) { * are available. */ static bool isTopBottomQuery(SQueryInfo* pQueryInfo) { - size_t size = tscSqlExprNumOfExprs(pQueryInfo); + size_t size = tscNumOfExprs(pQueryInfo); for (int32_t i = 0; i < size; ++i) { - int32_t functionId = tscSqlExprGet(pQueryInfo, i)->base.functionId; + int32_t functionId = tscExprGet(pQueryInfo, i)->base.functionId; if (functionId == TSDB_FUNC_TOP || functionId == TSDB_FUNC_BOTTOM) { return true; @@ -718,8 +735,8 @@ static bool isTopBottomQuery(SQueryInfo* pQueryInfo) { } // need to add timestamp column in result set, if it is a time window query -static int32_t addPrimaryTsColumnForTimeWindowQuery(SQueryInfo* pQueryInfo) { - uint64_t uid = tscSqlExprGet(pQueryInfo, 0)->base.uid; +static int32_t addPrimaryTsColumnForTimeWindowQuery(SQueryInfo* pQueryInfo, SSqlCmd* pCmd) { + uint64_t uid = tscExprGet(pQueryInfo, 0)->base.uid; int32_t tableIndex = COLUMN_INDEX_INITIAL_VAL; for (int32_t i = 0; i < pQueryInfo->numOfTables; ++i) { @@ -731,14 +748,14 @@ static int32_t addPrimaryTsColumnForTimeWindowQuery(SQueryInfo* pQueryInfo) { } if (tableIndex == COLUMN_INDEX_INITIAL_VAL) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } SSchema s = {.bytes = TSDB_KEYSIZE, .type = TSDB_DATA_TYPE_TIMESTAMP, .colId = PRIMARYKEY_TIMESTAMP_COL_INDEX}; tstrncpy(s.name, aAggs[TSDB_FUNC_TS].name, sizeof(s.name)); SColumnIndex index = {tableIndex, PRIMARYKEY_TIMESTAMP_COL_INDEX}; - tscAddFuncInSelectClause(pQueryInfo, 0, TSDB_FUNC_TS, &index, &s, TSDB_COL_NORMAL); + tscAddFuncInSelectClause(pQueryInfo, 0, TSDB_FUNC_TS, &index, &s, TSDB_COL_NORMAL, getNewResColId(pCmd)); return TSDB_CODE_SUCCESS; } @@ -754,7 +771,7 @@ static int32_t checkInvalidExprForTimeWindow(SSqlCmd* pCmd, SQueryInfo* pQueryIn // order by normal column is not supported int32_t colId = pQueryInfo->order.orderColId; if (isTimeWindowQuery(pQueryInfo) && colId != PRIMARYKEY_TIMESTAMP_COL_INDEX) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); } return TSDB_CODE_SUCCESS; @@ -764,11 +781,11 @@ static int32_t checkInvalidExprForTimeWindow(SSqlCmd* pCmd, SQueryInfo* pQueryIn * invalid sql: * select count(tbname)/count(tag1)/count(tag2) from super_table_name [interval(1d)|session(ts, 1d)]; */ - size_t size = tscSqlExprNumOfExprs(pQueryInfo); + size_t size = tscNumOfExprs(pQueryInfo); for (int32_t i = 0; i < size; ++i) { - SExprInfo* pExpr = tscSqlExprGet(pQueryInfo, i); + SExprInfo* pExpr = tscExprGet(pQueryInfo, i); if (pExpr->base.functionId == TSDB_FUNC_COUNT && TSDB_COL_IS_TAG(pExpr->base.colInfo.flag)) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } } @@ -777,10 +794,10 @@ static int32_t checkInvalidExprForTimeWindow(SSqlCmd* pCmd, SQueryInfo* pQueryIn * select tbname, tags_fields from super_table_name [interval(1s)|session(ts,1s)] */ if (tscQueryTags(pQueryInfo) && isTimeWindowQuery(pQueryInfo)) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } - return addPrimaryTsColumnForTimeWindowQuery(pQueryInfo); + return addPrimaryTsColumnForTimeWindowQuery(pQueryInfo, pCmd); } int32_t validateIntervalNode(SSqlObj* pSql, SQueryInfo* pQueryInfo, SSqlNode* pSqlNode) { @@ -794,7 +811,7 @@ int32_t validateIntervalNode(SSqlObj* pSql, SQueryInfo* pQueryInfo, SSqlNode* pS if (!TPARSER_HAS_TOKEN(pSqlNode->interval.interval)) { if (TPARSER_HAS_TOKEN(pSqlNode->sliding)) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg3); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); } return TSDB_CODE_SUCCESS; @@ -808,7 +825,7 @@ int32_t validateIntervalNode(SSqlObj* pSql, SQueryInfo* pQueryInfo, SSqlNode* pS // interval is not null SStrToken *t = &pSqlNode->interval.interval; if (parseNatualDuration(t->z, t->n, &pQueryInfo->interval.interval, &pQueryInfo->interval.intervalUnit) != TSDB_CODE_SUCCESS) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } if (pQueryInfo->interval.intervalUnit != 'n' && pQueryInfo->interval.intervalUnit != 'y') { @@ -819,16 +836,16 @@ int32_t validateIntervalNode(SSqlObj* pSql, SQueryInfo* pQueryInfo, SSqlNode* pS // interval cannot be less than 10 milliseconds if (pQueryInfo->interval.interval < tsMinIntervalTime) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); } } if (parseIntervalOffset(pCmd, pQueryInfo, &pSqlNode->interval.offset) != TSDB_CODE_SUCCESS) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } if (parseSlidingClause(pCmd, pQueryInfo, &pSqlNode->sliding) != TSDB_CODE_SUCCESS) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } // The following part is used to check for the invalid query expression. @@ -851,11 +868,11 @@ int32_t validateSessionNode(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SSqlNode * pS char timeUnit = 0; if (parseNatualDuration(gap->z, gap->n, &pQueryInfo->sessionWindow.gap, &timeUnit) != TSDB_CODE_SUCCESS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg4); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg4); } if (timeUnit == 'y' || timeUnit == 'n') { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } // if the unit of time window value is millisecond, change the value from microsecond @@ -866,12 +883,12 @@ int32_t validateSessionNode(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SSqlNode * pS } if (pQueryInfo->sessionWindow.gap != 0 && pQueryInfo->interval.interval != 0) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); } SColumnIndex index = COLUMN_INDEX_INITIALIZER; if (getColumnIndexByName(pCmd, col, pQueryInfo, &index) != TSDB_CODE_SUCCESS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg3); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); } pQueryInfo->sessionWindow.primaryColId = PRIMARYKEY_TIMESTAMP_COL_INDEX; @@ -896,11 +913,11 @@ int32_t parseIntervalOffset(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SStrToken* of } if (parseNatualDuration(t->z, t->n, &pQueryInfo->interval.offset, &pQueryInfo->interval.offsetUnit) != TSDB_CODE_SUCCESS) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } if (pQueryInfo->interval.offset < 0) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } if (pQueryInfo->interval.offsetUnit != 'n' && pQueryInfo->interval.offsetUnit != 'y') { @@ -910,18 +927,18 @@ int32_t parseIntervalOffset(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SStrToken* of } if (pQueryInfo->interval.intervalUnit != 'n' && pQueryInfo->interval.intervalUnit != 'y') { if (pQueryInfo->interval.offset >= pQueryInfo->interval.interval) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); } } } else if (pQueryInfo->interval.offsetUnit == pQueryInfo->interval.intervalUnit) { if (pQueryInfo->interval.offset >= pQueryInfo->interval.interval) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); } } else if (pQueryInfo->interval.intervalUnit == 'n' && pQueryInfo->interval.offsetUnit == 'y') { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg3); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); } else if (pQueryInfo->interval.intervalUnit == 'y' && pQueryInfo->interval.offsetUnit == 'n') { if (pQueryInfo->interval.interval * 12 <= pQueryInfo->interval.offset) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); } } else { // TODO: offset should be shorter than interval, but how to check @@ -949,7 +966,7 @@ int32_t parseSlidingClause(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SStrToken* pSl } if (pQueryInfo->interval.intervalUnit == 'n' || pQueryInfo->interval.intervalUnit == 'y') { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg3); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); } parseAbsoluteDuration(pSliding->z, pSliding->n, &pQueryInfo->interval.sliding); @@ -958,25 +975,25 @@ int32_t parseSlidingClause(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SStrToken* pSl } if (pQueryInfo->interval.sliding < tsMinSlidingTime) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg0); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg0); } if (pQueryInfo->interval.sliding > pQueryInfo->interval.interval) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } if ((pQueryInfo->interval.interval != 0) && (pQueryInfo->interval.interval/pQueryInfo->interval.sliding > INTERVAL_SLIDING_FACTOR)) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); } // if (pQueryInfo->interval.sliding != pQueryInfo->interval.interval && pSql->pStream == NULL) { -// return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg4); +// return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg4); // } return TSDB_CODE_SUCCESS; } -int32_t tscSetTableFullName(STableMetaInfo* pTableMetaInfo, SStrToken* pTableName, SSqlObj* pSql) { +int32_t tscSetTableFullName(SName* pName, SStrToken* pTableName, SSqlObj* pSql) { const char* msg1 = "name too long"; const char* msg2 = "acctId too long"; const char* msg3 = "no acctId"; @@ -989,27 +1006,27 @@ int32_t tscSetTableFullName(STableMetaInfo* pTableMetaInfo, SStrToken* pTableNam if (idx != -1) { // db has been specified in sql string so we ignore current db path char* acctId = getAccountId(pSql); if (acctId == NULL || strlen(acctId) <= 0) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg3); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); } - code = tNameSetAcctId(&pTableMetaInfo->name, acctId); + code = tNameSetAcctId(pName, acctId); if (code != 0) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); } - if (idx >= TSDB_DB_NAME_LEN) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg4); + if (idx >= TSDB_DB_NAME_LEN) { + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg4); } if (pTableName->n - 1 - idx >= TSDB_TABLE_NAME_LEN) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg5); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg5); } - + char name[TSDB_TABLE_FNAME_LEN] = {0}; strncpy(name, pTableName->z, pTableName->n); - code = tNameFromString(&pTableMetaInfo->name, name, T_NAME_DB|T_NAME_TABLE); + code = tNameFromString(pName, name, T_NAME_DB|T_NAME_TABLE); if (code != 0) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } } else { // get current DB name first, and then set it into path char* t = cloneCurrentDBName(pSql); @@ -1017,7 +1034,7 @@ int32_t tscSetTableFullName(STableMetaInfo* pTableMetaInfo, SStrToken* pTableNam return TSDB_CODE_TSC_DB_NOT_SELECTED; } - code = tNameFromString(&pTableMetaInfo->name, t, T_NAME_ACCT | T_NAME_DB); + code = tNameFromString(pName, t, T_NAME_ACCT | T_NAME_DB); if (code != 0) { free(t); return TSDB_CODE_TSC_DB_NOT_SELECTED; @@ -1026,15 +1043,15 @@ int32_t tscSetTableFullName(STableMetaInfo* pTableMetaInfo, SStrToken* pTableNam free(t); if (pTableName->n >= TSDB_TABLE_NAME_LEN) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } char name[TSDB_TABLE_FNAME_LEN] = {0}; strncpy(name, pTableName->z, pTableName->n); - code = tNameFromString(&pTableMetaInfo->name, name, T_NAME_TABLE); + code = tNameFromString(pName, name, T_NAME_TABLE); if (code != 0) { - code = invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + code = invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } } @@ -1055,14 +1072,14 @@ static bool validateTableColumnInfo(SArray* pFieldList, SSqlCmd* pCmd) { // number of fields no less than 2 size_t numOfCols = taosArrayGetSize(pFieldList); if (numOfCols <= 1 || numOfCols > TSDB_MAX_COLUMNS) { - invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg); + invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg); return false; } // first column must be timestamp TAOS_FIELD* pField = taosArrayGet(pFieldList, 0); if (pField->type != TSDB_DATA_TYPE_TIMESTAMP) { - invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); return false; } @@ -1070,29 +1087,29 @@ static bool validateTableColumnInfo(SArray* pFieldList, SSqlCmd* pCmd) { for (int32_t i = 0; i < numOfCols; ++i) { pField = taosArrayGet(pFieldList, i); if (!isValidDataType(pField->type)) { - invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg4); + invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg4); return false; } if (pField->bytes == 0) { - invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg5); + invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg5); return false; } if ((pField->type == TSDB_DATA_TYPE_BINARY && (pField->bytes <= 0 || pField->bytes > TSDB_MAX_BINARY_LEN)) || (pField->type == TSDB_DATA_TYPE_NCHAR && (pField->bytes <= 0 || pField->bytes > TSDB_MAX_NCHAR_LEN))) { - invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg5); + invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg5); return false; } if (validateColumnName(pField->name) != TSDB_CODE_SUCCESS) { - invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg6); + invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg6); return false; } // field name must be unique if (has(pFieldList, i + 1, pField->name) == true) { - invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg3); + invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); return false; } @@ -1101,7 +1118,7 @@ static bool validateTableColumnInfo(SArray* pFieldList, SSqlCmd* pCmd) { // max row length must be less than TSDB_MAX_BYTES_PER_ROW if (nLen > TSDB_MAX_BYTES_PER_ROW) { - invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); return false; } @@ -1123,7 +1140,7 @@ static bool validateTagParams(SArray* pTagsList, SArray* pFieldList, SSqlCmd* pC // number of fields at least 1 size_t numOfTags = taosArrayGetSize(pTagsList); if (numOfTags < 1 || numOfTags > TSDB_MAX_TAGS) { - invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); return false; } @@ -1132,28 +1149,28 @@ static bool validateTagParams(SArray* pTagsList, SArray* pFieldList, SSqlCmd* pC TAOS_FIELD* p = taosArrayGet(pTagsList, i); if (p->type == TSDB_DATA_TYPE_TIMESTAMP) { - invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg4); + invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg4); return false; } if (!isValidDataType(p->type)) { - invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg5); + invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg5); return false; } if ((p->type == TSDB_DATA_TYPE_BINARY && p->bytes <= 0) || (p->type == TSDB_DATA_TYPE_NCHAR && p->bytes <= 0)) { - invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg7); + invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg7); return false; } if (validateColumnName(p->name) != TSDB_CODE_SUCCESS) { - invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg6); + invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg6); return false; } if (has(pTagsList, i + 1, p->name) == true) { - invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg3); + invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); return false; } } @@ -1162,7 +1179,7 @@ static bool validateTagParams(SArray* pTagsList, SArray* pFieldList, SSqlCmd* pC for (int32_t i = 0; i < numOfTags; ++i) { TAOS_FIELD* p = taosArrayGet(pTagsList, i); if (p->bytes == 0) { - invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg7); + invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg7); return false; } @@ -1171,7 +1188,7 @@ static bool validateTagParams(SArray* pTagsList, SArray* pFieldList, SSqlCmd* pC // max tag row length must be less than TSDB_MAX_TAGS_LEN if (nLen > TSDB_MAX_TAGS_LEN) { - invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); return false; } @@ -1180,7 +1197,7 @@ static bool validateTagParams(SArray* pTagsList, SArray* pFieldList, SSqlCmd* pC TAOS_FIELD* p = taosArrayGet(pTagsList, i); if (has(pFieldList, 0, p->name) == true) { - invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg3); + invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); return false; } } @@ -1199,9 +1216,7 @@ bool validateOneTags(SSqlCmd* pCmd, TAOS_FIELD* pTagField) { const char* msg5 = "invalid binary/nchar tag length"; const char* msg6 = "invalid data type in tags"; - assert(pCmd->numOfClause == 1); - - STableMetaInfo* pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, pCmd->clauseIndex, 0); + STableMetaInfo* pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, 0); STableMeta* pTableMeta = pTableMetaInfo->pTableMeta; int32_t numOfTags = tscGetNumOfTags(pTableMeta); @@ -1212,18 +1227,18 @@ bool validateOneTags(SSqlCmd* pCmd, TAOS_FIELD* pTagField) { char msg[128] = {0}; sprintf(msg, "tags no more than %d", TSDB_MAX_TAGS); - invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg); + invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg); return false; } // no timestamp allowable if (pTagField->type == TSDB_DATA_TYPE_TIMESTAMP) { - invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); return false; } if ((pTagField->type < TSDB_DATA_TYPE_BOOL) || (pTagField->type > TSDB_DATA_TYPE_UBIGINT)) { - invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg6); + invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg6); return false; } @@ -1236,19 +1251,19 @@ bool validateOneTags(SSqlCmd* pCmd, TAOS_FIELD* pTagField) { // length less than TSDB_MAX_TASG_LEN if (nLen + pTagField->bytes > TSDB_MAX_TAGS_LEN) { - invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg3); + invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); return false; } // tags name can not be a keyword if (validateColumnName(pTagField->name) != TSDB_CODE_SUCCESS) { - invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg4); + invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg4); return false; } // binary(val), val can not be equalled to or less than 0 if ((pTagField->type == TSDB_DATA_TYPE_BINARY || pTagField->type == TSDB_DATA_TYPE_NCHAR) && pTagField->bytes <= 0) { - invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg5); + invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg5); return false; } @@ -1257,7 +1272,7 @@ bool validateOneTags(SSqlCmd* pCmd, TAOS_FIELD* pTagField) { for (int32_t i = 0; i < numOfTags + numOfCols; ++i) { if (strncasecmp(pTagField->name, pSchema[i].name, sizeof(pTagField->name) - 1) == 0) { - invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); return false; } } @@ -1273,8 +1288,8 @@ bool validateOneColumn(SSqlCmd* pCmd, TAOS_FIELD* pColField) { const char* msg5 = "invalid column name"; const char* msg6 = "invalid column length"; - assert(pCmd->numOfClause == 1); - STableMetaInfo* pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, pCmd->clauseIndex, 0); +// assert(pCmd->numOfClause == 1); + STableMetaInfo* pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, 0); STableMeta* pTableMeta = pTableMetaInfo->pTableMeta; int32_t numOfTags = tscGetNumOfTags(pTableMeta); @@ -1282,17 +1297,17 @@ bool validateOneColumn(SSqlCmd* pCmd, TAOS_FIELD* pColField) { // no more max columns if (numOfCols >= TSDB_MAX_COLUMNS || numOfTags + numOfCols >= TSDB_MAX_COLUMNS) { - invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); return false; } if (pColField->type < TSDB_DATA_TYPE_BOOL || pColField->type > TSDB_DATA_TYPE_UBIGINT) { - invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg4); + invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg4); return false; } if (validateColumnName(pColField->name) != TSDB_CODE_SUCCESS) { - invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg5); + invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg5); return false; } @@ -1304,20 +1319,20 @@ bool validateOneColumn(SSqlCmd* pCmd, TAOS_FIELD* pColField) { } if (pColField->bytes <= 0) { - invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg6); + invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg6); return false; } // length less than TSDB_MAX_BYTES_PER_ROW if (nLen + pColField->bytes > TSDB_MAX_BYTES_PER_ROW) { - invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg3); + invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); return false; } // field name must be unique for (int32_t i = 0; i < numOfTags + numOfCols; ++i) { if (strncasecmp(pColField->name, pSchema[i].name, sizeof(pColField->name) - 1) == 0) { - invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); return false; } } @@ -1348,8 +1363,8 @@ static char* cloneCurrentDBName(SSqlObj* pSql) { /* length limitation, strstr cannot be applied */ static int32_t getDelimiterIndex(SStrToken* pTableName) { - for (uint32_t i = 0; i < pTableName->n; ++i) { - if (pTableName->z[i] == TS_PATH_DELIMITER[0]) { + for (uint32_t i = 0; i < pTableName->n; ++i) { + if (pTableName->z[i] == TS_PATH_DELIMITER[0]) { return i; } } @@ -1369,7 +1384,7 @@ int32_t setObjFullName(char* fullName, const char* account, SStrToken* pDB, SStr /* db name is not specified, the tableName dose not include db name */ if (pDB != NULL) { if (pDB->n >= TSDB_ACCT_ID_LEN + TSDB_DB_NAME_LEN || pDB->n == 0) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } memcpy(&fullName[totalLen], pDB->z, pDB->n); @@ -1383,12 +1398,12 @@ int32_t setObjFullName(char* fullName, const char* account, SStrToken* pDB, SStr /* here we only check the table name length limitation */ if (!tscValidateTableNameLength(tableName->n)) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } } else { // pDB == NULL, the db prefix name is specified in tableName /* the length limitation includes tablename + dbname + sep */ if (tableName->n >= TSDB_TABLE_NAME_LEN + TSDB_DB_NAME_LEN) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } } @@ -1404,7 +1419,7 @@ int32_t setObjFullName(char* fullName, const char* account, SStrToken* pDB, SStr fullName[totalLen] = 0; } - return (totalLen < TSDB_TABLE_FNAME_LEN) ? TSDB_CODE_SUCCESS : TSDB_CODE_TSC_INVALID_SQL; + return (totalLen < TSDB_TABLE_FNAME_LEN) ? TSDB_CODE_SUCCESS : TSDB_CODE_TSC_INVALID_OPERATION; } void tscInsertPrimaryTsSourceColumn(SQueryInfo* pQueryInfo, uint64_t tableUid) { @@ -1422,7 +1437,7 @@ static int32_t handleArithmeticExpr(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, int32 int32_t arithmeticType = NON_ARITHMEIC_EXPR; if (validateArithmeticSQLExpr(pCmd, pItem->pNode, pQueryInfo, &columnList, &arithmeticType) != TSDB_CODE_SUCCESS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } int32_t tableIndex = columnList.ids[0].tableIndex; @@ -1432,15 +1447,15 @@ static int32_t handleArithmeticExpr(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, int32 // all columns in arithmetic expression must belong to the same table for (int32_t f = 1; f < columnList.num; ++f) { if (columnList.ids[f].tableIndex != tableIndex) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg4); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg4); } } // expr string is set as the parameter of function SColumnIndex index = {.tableIndex = tableIndex}; - SExprInfo* pExpr = tscSqlExprAppend(pQueryInfo, TSDB_FUNC_ARITHM, &index, TSDB_DATA_TYPE_DOUBLE, sizeof(double), - getNewResColId(pQueryInfo), sizeof(double), false); + SExprInfo* pExpr = tscExprAppend(pQueryInfo, TSDB_FUNC_ARITHM, &index, TSDB_DATA_TYPE_DOUBLE, sizeof(double), + getNewResColId(pCmd), sizeof(double), false); char* name = (pItem->aliasName != NULL)? pItem->aliasName:pItem->pNode->token.z; size_t len = MIN(sizeof(pExpr->base.aliasName), pItem->pNode->token.n + 1); @@ -1453,7 +1468,7 @@ static int32_t handleArithmeticExpr(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, int32 if (ret != TSDB_CODE_SUCCESS) { taosArrayDestroy(colList); tExprTreeDestroy(pNode, NULL); - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); } // check for if there is a tag in the arithmetic express @@ -1464,7 +1479,7 @@ static int32_t handleArithmeticExpr(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, int32 tExprTreeDestroy(pNode, NULL); taosArrayDestroy(colList); - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg3); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); } } @@ -1482,7 +1497,7 @@ static int32_t handleArithmeticExpr(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, int32 char* c = tbufGetData(&bw, false); // set the serialized binary string as the parameter of arithmetic expression - addExprParams(&pExpr->base, c, TSDB_DATA_TYPE_BINARY, (int32_t)len); + tscExprAddParams(&pExpr->base, c, TSDB_DATA_TYPE_BINARY, (int32_t)len); insertResultField(pQueryInfo, exprIndex, &columnList, sizeof(double), TSDB_DATA_TYPE_DOUBLE, pExpr->base.aliasName, pExpr); // add ts column @@ -1495,40 +1510,37 @@ static int32_t handleArithmeticExpr(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, int32 columnList.num = 0; columnList.ids[0] = (SColumnIndex) {0, 0}; + char rawName[TSDB_COL_NAME_LEN] = {0}; char aliasName[TSDB_COL_NAME_LEN] = {0}; - if (pItem->aliasName != NULL) { - tstrncpy(aliasName, pItem->aliasName, TSDB_COL_NAME_LEN); - } else { - int32_t nameLen = MIN(TSDB_COL_NAME_LEN, pItem->pNode->token.n + 1); - tstrncpy(aliasName, pItem->pNode->token.z, nameLen); - } + getColumnName(pItem, aliasName, rawName, TSDB_COL_NAME_LEN); insertResultField(pQueryInfo, exprIndex, &columnList, sizeof(double), TSDB_DATA_TYPE_DOUBLE, aliasName, NULL); int32_t slot = tscNumOfFields(pQueryInfo) - 1; SInternalField* pInfo = tscFieldInfoGetInternalField(&pQueryInfo->fieldsInfo, slot); + assert(pInfo->pExpr == NULL); - if (pInfo->pExpr == NULL) { - SExprInfo* pExprInfo = calloc(1, sizeof(SExprInfo)); + SExprInfo* pExprInfo = calloc(1, sizeof(SExprInfo)); - // arithmetic expression always return result in the format of double float - pExprInfo->base.resBytes = sizeof(double); - pExprInfo->base.interBytes = sizeof(double); - pExprInfo->base.resType = TSDB_DATA_TYPE_DOUBLE; + // arithmetic expression always return result in the format of double float + pExprInfo->base.resBytes = sizeof(double); + pExprInfo->base.interBytes = sizeof(double); + pExprInfo->base.resType = TSDB_DATA_TYPE_DOUBLE; - pExprInfo->base.functionId = TSDB_FUNC_ARITHM; - pExprInfo->base.numOfParams = 1; - pExprInfo->base.resColId = getNewResColId(pQueryInfo); + pExprInfo->base.functionId = TSDB_FUNC_ARITHM; + pExprInfo->base.numOfParams = 1; + pExprInfo->base.resColId = getNewResColId(pCmd); + strncpy(pExprInfo->base.aliasName, aliasName, tListLen(pExprInfo->base.aliasName)); + strncpy(pExprInfo->base.token, rawName, tListLen(pExprInfo->base.token)); - int32_t ret = exprTreeFromSqlExpr(pCmd, &pExprInfo->pExpr, pItem->pNode, pQueryInfo, NULL, &(pExprInfo->base.uid)); - if (ret != TSDB_CODE_SUCCESS) { - tExprTreeDestroy(pExprInfo->pExpr, NULL); - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), "invalid expression in select clause"); - } - - pInfo->pExpr = pExprInfo; + int32_t ret = exprTreeFromSqlExpr(pCmd, &pExprInfo->pExpr, pItem->pNode, pQueryInfo, NULL, &(pExprInfo->base.uid)); + if (ret != TSDB_CODE_SUCCESS) { + tExprTreeDestroy(pExprInfo->pExpr, NULL); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), "invalid expression in select clause"); } + pInfo->pExpr = pExprInfo; + SBufferWriter bw = tbufInitWriter(NULL, false); TRY(0) { @@ -1550,8 +1562,8 @@ static int32_t handleArithmeticExpr(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, int32 return TSDB_CODE_SUCCESS; } -static void addProjectQueryCol(SQueryInfo* pQueryInfo, int32_t startPos, SColumnIndex* pIndex, tSqlExprItem* pItem) { - SExprInfo* pExpr = doAddProjectCol(pQueryInfo, pIndex->columnIndex, pIndex->tableIndex); +static void addProjectQueryCol(SQueryInfo* pQueryInfo, int32_t startPos, SColumnIndex* pIndex, tSqlExprItem* pItem, int32_t colId) { + SExprInfo* pExpr = doAddProjectCol(pQueryInfo, pIndex->columnIndex, pIndex->tableIndex, colId); STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, pIndex->tableIndex); STableMeta* pTableMeta = pTableMetaInfo->pTableMeta; @@ -1573,11 +1585,11 @@ static void addProjectQueryCol(SQueryInfo* pQueryInfo, int32_t startPos, SColumn insertResultField(pQueryInfo, startPos, &ids, pExpr->base.resBytes, (int8_t)pExpr->base.resType, pExpr->base.aliasName, pExpr); } -static void addPrimaryTsColIntoResult(SQueryInfo* pQueryInfo) { +static void addPrimaryTsColIntoResult(SQueryInfo* pQueryInfo, SSqlCmd* pCmd) { // primary timestamp column has been added already - size_t size = tscSqlExprNumOfExprs(pQueryInfo); + size_t size = tscNumOfExprs(pQueryInfo); for (int32_t i = 0; i < size; ++i) { - SExprInfo* pExpr = tscSqlExprGet(pQueryInfo, i); + SExprInfo* pExpr = tscExprGet(pQueryInfo, i); if (pExpr->base.functionId == TSDB_FUNC_PRJ && pExpr->base.colInfo.colId == PRIMARYKEY_TIMESTAMP_COL_INDEX) { return; } @@ -1590,8 +1602,8 @@ static void addPrimaryTsColIntoResult(SQueryInfo* pQueryInfo) { // add the timestamp column into the output columns SColumnIndex index = {0}; // primary timestamp column info - int32_t numOfCols = (int32_t)tscSqlExprNumOfExprs(pQueryInfo); - tscAddFuncInSelectClause(pQueryInfo, numOfCols, TSDB_FUNC_PRJ, &index, pSchema, TSDB_COL_NORMAL); + int32_t numOfCols = (int32_t)tscNumOfExprs(pQueryInfo); + tscAddFuncInSelectClause(pQueryInfo, numOfCols, TSDB_FUNC_PRJ, &index, pSchema, TSDB_COL_NORMAL, getNewResColId(pCmd)); SInternalField* pSupInfo = tscFieldInfoGetInternalField(&pQueryInfo->fieldsInfo, numOfCols); pSupInfo->visible = false; @@ -1606,7 +1618,7 @@ bool isValidDistinctSql(SQueryInfo* pQueryInfo) { if ((pQueryInfo->type & TSDB_QUERY_TYPE_STABLE_QUERY) != TSDB_QUERY_TYPE_STABLE_QUERY) { return false; } - if (tscQueryTags(pQueryInfo) && tscSqlExprNumOfExprs(pQueryInfo) == 1){ + if (tscQueryTags(pQueryInfo) && tscNumOfExprs(pQueryInfo) == 1){ return true; } return false; @@ -1640,7 +1652,7 @@ int32_t validateSelectNodeList(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SArray* pS // too many result columns not support order by in query if (taosArrayGetSize(pSelNodeList) > TSDB_MAX_COLUMNS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } if (pQueryInfo->colList == NULL) { @@ -1650,7 +1662,7 @@ int32_t validateSelectNodeList(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SArray* pS bool hasDistinct = false; size_t numOfExpr = taosArrayGetSize(pSelNodeList); for (int32_t i = 0; i < numOfExpr; ++i) { - int32_t outputIndex = (int32_t)tscSqlExprNumOfExprs(pQueryInfo); + int32_t outputIndex = (int32_t)tscNumOfExprs(pQueryInfo); tSqlExprItem* pItem = taosArrayGet(pSelNodeList, i); if (hasDistinct == false) { @@ -1661,18 +1673,18 @@ int32_t validateSelectNodeList(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SArray* pS if (type == SQL_NODE_SQLFUNCTION) { pItem->pNode->functionId = isValidFunction(pItem->pNode->operand.z, pItem->pNode->operand.n); if (pItem->pNode->functionId < 0) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg5); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg5); } // sql function in selection clause, append sql function info in pSqlCmd structure sequentially if (addExprAndResultField(pCmd, pQueryInfo, outputIndex, pItem, true) != TSDB_CODE_SUCCESS) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } } else if (type == SQL_NODE_TABLE_COLUMN || type == SQL_NODE_VALUE) { // use the dynamic array list to decide if the function is valid or not // select table_name1.field_name1, table_name2.field_name2 from table_name1, table_name2 if (addProjectionExprAndResultField(pCmd, pQueryInfo, pItem) != TSDB_CODE_SUCCESS) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } } else if (type == SQL_NODE_EXPR) { int32_t code = handleArithmeticExpr(pCmd, pQueryInfo, i, pItem); @@ -1680,17 +1692,17 @@ int32_t validateSelectNodeList(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SArray* pS return code; } } else { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg3); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); } if (pQueryInfo->fieldsInfo.numOfOutput > TSDB_MAX_COLUMNS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } } if (hasDistinct == true) { if (!isValidDistinctSql(pQueryInfo)) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg4); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg4); } pQueryInfo->distinctTag = true; } @@ -1698,11 +1710,11 @@ int32_t validateSelectNodeList(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SArray* pS // there is only one user-defined column in the final result field, add the timestamp column. size_t numOfSrcCols = taosArrayGetSize(pQueryInfo->colList); if ((numOfSrcCols <= 0 || !hasNoneUserDefineExpr(pQueryInfo)) && !tscQueryTags(pQueryInfo) && !tscQueryBlockInfo(pQueryInfo)) { - addPrimaryTsColIntoResult(pQueryInfo); + addPrimaryTsColIntoResult(pQueryInfo, pCmd); } if (!functionCompatibleCheck(pQueryInfo, joinQuery, timeWindowQuery)) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); } return TSDB_CODE_SUCCESS; @@ -1731,7 +1743,7 @@ int32_t insertResultField(SQueryInfo* pQueryInfo, int32_t outputIndex, SColumnLi return TSDB_CODE_SUCCESS; } -SExprInfo* doAddProjectCol(SQueryInfo* pQueryInfo, int32_t colIndex, int32_t tableIndex) { +SExprInfo* doAddProjectCol(SQueryInfo* pQueryInfo, int32_t colIndex, int32_t tableIndex, int32_t colId) { STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, tableIndex); STableMeta* pTableMeta = pTableMetaInfo->pTableMeta; int32_t numOfCols = tscGetNumOfColumns(pTableMeta); @@ -1748,18 +1760,16 @@ SExprInfo* doAddProjectCol(SQueryInfo* pQueryInfo, int32_t colIndex, int32_t tab index.columnIndex = colIndex; } - int16_t colId = getNewResColId(pQueryInfo); - return tscSqlExprAppend(pQueryInfo, functionId, &index, pSchema->type, pSchema->bytes, colId, pSchema->bytes, + return tscExprAppend(pQueryInfo, functionId, &index, pSchema->type, pSchema->bytes, colId, pSchema->bytes, (functionId == TSDB_FUNC_TAGPRJ)); } SExprInfo* tscAddFuncInSelectClause(SQueryInfo* pQueryInfo, int32_t outputColIndex, int16_t functionId, - SColumnIndex* pIndex, SSchema* pColSchema, int16_t flag) { - int16_t colId = getNewResColId(pQueryInfo); - - SExprInfo* pExpr = tscSqlExprInsert(pQueryInfo, outputColIndex, functionId, pIndex, pColSchema->type, + SColumnIndex* pIndex, SSchema* pColSchema, int16_t flag, int16_t colId) { + SExprInfo* pExpr = tscExprInsert(pQueryInfo, outputColIndex, functionId, pIndex, pColSchema->type, pColSchema->bytes, colId, pColSchema->bytes, TSDB_COL_IS_TAG(flag)); tstrncpy(pExpr->base.aliasName, pColSchema->name, sizeof(pExpr->base.aliasName)); + tstrncpy(pExpr->base.token, pColSchema->name, sizeof(pExpr->base.token)); SColumnList ids = createColumnList(1, pIndex->tableIndex, pIndex->columnIndex); if (TSDB_COL_IS_TAG(flag)) { @@ -1778,7 +1788,7 @@ SExprInfo* tscAddFuncInSelectClause(SQueryInfo* pQueryInfo, int32_t outputColInd return pExpr; } -static int32_t doAddProjectionExprAndResultFields(SQueryInfo* pQueryInfo, SColumnIndex* pIndex, int32_t startPos) { +static int32_t doAddProjectionExprAndResultFields(SQueryInfo* pQueryInfo, SColumnIndex* pIndex, int32_t startPos, SSqlCmd* pCmd) { STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, pIndex->tableIndex); int32_t numOfTotalColumns = 0; @@ -1794,7 +1804,7 @@ static int32_t doAddProjectionExprAndResultFields(SQueryInfo* pQueryInfo, SColum } for (int32_t j = 0; j < numOfTotalColumns; ++j) { - SExprInfo* pExpr = doAddProjectCol(pQueryInfo, j, pIndex->tableIndex); + SExprInfo* pExpr = doAddProjectCol(pQueryInfo, j, pIndex->tableIndex, getNewResColId(pCmd)); tstrncpy(pExpr->base.aliasName, pSchema[j].name, sizeof(pExpr->base.aliasName)); pIndex->columnIndex = j; @@ -1812,7 +1822,7 @@ int32_t addProjectionExprAndResultField(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, t const char* msg0 = "invalid column name"; const char* msg1 = "tag for normal table query is not allowed"; - int32_t startPos = (int32_t)tscSqlExprNumOfExprs(pQueryInfo); + int32_t startPos = (int32_t)tscNumOfExprs(pQueryInfo); int32_t optr = pItem->pNode->tokenId; if (optr == TK_ALL) { // project on all fields @@ -1820,18 +1830,18 @@ int32_t addProjectionExprAndResultField(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, t SColumnIndex index = COLUMN_INDEX_INITIALIZER; if (getTableIndexByName(&pItem->pNode->colInfo, pQueryInfo, &index) != TSDB_CODE_SUCCESS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg0); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg0); } // all meters columns are required if (index.tableIndex == COLUMN_INDEX_INITIAL_VAL) { // all table columns are required. for (int32_t i = 0; i < pQueryInfo->numOfTables; ++i) { index.tableIndex = i; - int32_t inc = doAddProjectionExprAndResultFields(pQueryInfo, &index, startPos); + int32_t inc = doAddProjectionExprAndResultFields(pQueryInfo, &index, startPos, pCmd); startPos += inc; } } else { - doAddProjectionExprAndResultFields(pQueryInfo, &index, startPos); + doAddProjectionExprAndResultFields(pQueryInfo, &index, startPos, pCmd); } // add the primary timestamp column even though it is not required by user @@ -1848,7 +1858,7 @@ int32_t addProjectionExprAndResultField(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, t SSchema colSchema = tGetUserSpecifiedColumnSchema(&pItem->pNode->value, &pItem->pNode->token, pItem->aliasName); SExprInfo* pExpr = - tscAddFuncInSelectClause(pQueryInfo, startPos, TSDB_FUNC_PRJ, &index, &colSchema, TSDB_COL_UDC); + tscAddFuncInSelectClause(pQueryInfo, startPos, TSDB_FUNC_PRJ, &index, &colSchema, TSDB_COL_UDC, getNewResColId(pCmd)); // NOTE: the first parameter is reserved for the tag column id during join query process. pExpr->base.numOfParams = 2; @@ -1857,32 +1867,34 @@ int32_t addProjectionExprAndResultField(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, t SColumnIndex index = COLUMN_INDEX_INITIALIZER; if (getColumnIndexByName(pCmd, &pItem->pNode->colInfo, pQueryInfo, &index) != TSDB_CODE_SUCCESS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg0); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg0); } if (index.columnIndex == TSDB_TBNAME_COLUMN_INDEX) { SSchema* colSchema = tGetTbnameColumnSchema(); - tscAddFuncInSelectClause(pQueryInfo, startPos, TSDB_FUNC_TAGPRJ, &index, colSchema, TSDB_COL_TAG); + tscAddFuncInSelectClause(pQueryInfo, startPos, TSDB_FUNC_TAGPRJ, &index, colSchema, TSDB_COL_TAG, getNewResColId(pCmd)); } else if (index.columnIndex == TSDB_BLOCK_DIST_COLUMN_INDEX) { SSchema colSchema = tGetBlockDistColumnSchema(); - tscAddFuncInSelectClause(pQueryInfo, startPos, TSDB_FUNC_PRJ, &index, &colSchema, TSDB_COL_TAG); + tscAddFuncInSelectClause(pQueryInfo, startPos, TSDB_FUNC_PRJ, &index, &colSchema, TSDB_COL_TAG, getNewResColId(pCmd)); } else { STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, index.tableIndex); STableMeta* pTableMeta = pTableMetaInfo->pTableMeta; if (index.columnIndex >= tscGetNumOfColumns(pTableMeta) && UTIL_TABLE_IS_NORMAL_TABLE(pTableMetaInfo)) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } - addProjectQueryCol(pQueryInfo, startPos, &index, pItem); + addProjectQueryCol(pQueryInfo, startPos, &index, pItem, getNewResColId(pCmd)); pQueryInfo->type |= TSDB_QUERY_TYPE_PROJECTION_QUERY; } // add the primary timestamp column even though it is not required by user STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, index.tableIndex); - tscInsertPrimaryTsSourceColumn(pQueryInfo, pTableMetaInfo->pTableMeta->id.uid); + if (!UTIL_TABLE_IS_TMP_TABLE(pTableMetaInfo)) { + tscInsertPrimaryTsSourceColumn(pQueryInfo, pTableMetaInfo->pTableMeta->id.uid); + } } else { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } return TSDB_CODE_SUCCESS; @@ -1899,7 +1911,7 @@ static int32_t setExprInfoForFunctions(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SS if (functionID == TSDB_FUNC_SPREAD) { int32_t t1 = pSchema->type; if (t1 == TSDB_DATA_TYPE_BINARY || t1 == TSDB_DATA_TYPE_NCHAR || t1 == TSDB_DATA_TYPE_BOOL) { - invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); return -1; } else { type = TSDB_DATA_TYPE_DOUBLE; @@ -1910,7 +1922,7 @@ static int32_t setExprInfoForFunctions(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SS bytes = pSchema->bytes; } - SExprInfo* pExpr = tscSqlExprAppend(pQueryInfo, functionID, pColIndex, type, bytes, getNewResColId(pQueryInfo), bytes, false); + SExprInfo* pExpr = tscExprAppend(pQueryInfo, functionID, pColIndex, type, bytes, getNewResColId(pCmd), bytes, false); tstrncpy(pExpr->base.aliasName, name, tListLen(pExpr->base.aliasName)); if (cvtFunc.originFuncId == TSDB_FUNC_LAST_ROW && cvtFunc.originFuncId != functionID) { @@ -1964,9 +1976,9 @@ void setResultColName(char* name, tSqlExprItem* pItem, int32_t functionId, SStrT static void updateLastScanOrderIfNeeded(SQueryInfo* pQueryInfo) { if (pQueryInfo->sessionWindow.gap > 0 || tscGroupbyColumn(pQueryInfo)) { - size_t numOfExpr = tscSqlExprNumOfExprs(pQueryInfo); + size_t numOfExpr = tscNumOfExprs(pQueryInfo); for (int32_t i = 0; i < numOfExpr; ++i) { - SExprInfo* pExpr = tscSqlExprGet(pQueryInfo, i); + SExprInfo* pExpr = tscExprGet(pQueryInfo, i); if (pExpr->base.functionId != TSDB_FUNC_LAST && pExpr->base.functionId != TSDB_FUNC_LAST_DST) { continue; } @@ -1996,7 +2008,7 @@ int32_t addExprAndResultField(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, int32_t col case TSDB_FUNC_COUNT: { /* more than one parameter for count() function */ if (pItem->pNode->pParam != NULL && taosArrayGetSize(pItem->pNode->pParam) != 1) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); } SExprInfo* pExpr = NULL; @@ -2005,43 +2017,28 @@ int32_t addExprAndResultField(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, int32_t col if (pItem->pNode->pParam != NULL) { tSqlExprItem* pParamElem = taosArrayGet(pItem->pNode->pParam, 0); SStrToken* pToken = &pParamElem->pNode->colInfo; - int16_t sqlOptr = pParamElem->pNode->tokenId; - if ((pToken->z == NULL || pToken->n == 0) - && (TK_INTEGER != sqlOptr)) /*select count(1) from table*/ { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg3); + int16_t tokenId = pParamElem->pNode->tokenId; + if ((pToken->z == NULL || pToken->n == 0) && (TK_INTEGER != tokenId)) { + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); } - if (sqlOptr == TK_ALL) { - // select table.* + // select count(table.*) + // select count(1)|count(2) + if (tokenId == TK_ALL || tokenId == TK_INTEGER) { // check if the table name is valid or not SStrToken tmpToken = pParamElem->pNode->colInfo; if (getTableIndexByName(&tmpToken, pQueryInfo, &index) != TSDB_CODE_SUCCESS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg4); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg4); } index = (SColumnIndex){0, PRIMARYKEY_TIMESTAMP_COL_INDEX}; int32_t size = tDataTypes[TSDB_DATA_TYPE_BIGINT].bytes; - pExpr = tscSqlExprAppend(pQueryInfo, functionId, &index, TSDB_DATA_TYPE_BIGINT, size, getNewResColId(pQueryInfo), size, false); - } else if (sqlOptr == TK_INTEGER) { // select count(1) from table1 - char buf[8] = {0}; - int64_t val = -1; - tVariant* pVariant = &pParamElem->pNode->value; - if (pVariant->nType == TSDB_DATA_TYPE_BIGINT) { - tVariantDump(pVariant, buf, TSDB_DATA_TYPE_BIGINT, true); - val = GET_INT64_VAL(buf); - } - if (val == 1) { - index = (SColumnIndex){0, PRIMARYKEY_TIMESTAMP_COL_INDEX}; - int32_t size = tDataTypes[TSDB_DATA_TYPE_BIGINT].bytes; - pExpr = tscSqlExprAppend(pQueryInfo, functionId, &index, TSDB_DATA_TYPE_BIGINT, size, getNewResColId(pQueryInfo), size, false); - } else { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg3); - } + pExpr = tscExprAppend(pQueryInfo, functionId, &index, TSDB_DATA_TYPE_BIGINT, size, getNewResColId(pCmd), size, false); } else { - // count the number of meters created according to the super table + // count the number of table created according to the super table if (getColumnIndexByName(pCmd, pToken, pQueryInfo, &index) != TSDB_CODE_SUCCESS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg3); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); } pTableMetaInfo = tscGetMetaInfo(pQueryInfo, index.tableIndex); @@ -2054,18 +2051,18 @@ int32_t addExprAndResultField(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, int32_t col } int32_t size = tDataTypes[TSDB_DATA_TYPE_BIGINT].bytes; - pExpr = tscSqlExprAppend(pQueryInfo, functionId, &index, TSDB_DATA_TYPE_BIGINT, size, getNewResColId(pQueryInfo), size, isTag); + pExpr = tscExprAppend(pQueryInfo, functionId, &index, TSDB_DATA_TYPE_BIGINT, size, getNewResColId(pCmd), size, isTag); } } else { // count(*) is equalled to count(primary_timestamp_key) index = (SColumnIndex){0, PRIMARYKEY_TIMESTAMP_COL_INDEX}; int32_t size = tDataTypes[TSDB_DATA_TYPE_BIGINT].bytes; - pExpr = tscSqlExprAppend(pQueryInfo, functionId, &index, TSDB_DATA_TYPE_BIGINT, size, getNewResColId(pQueryInfo), size, false); + pExpr = tscExprAppend(pQueryInfo, functionId, &index, TSDB_DATA_TYPE_BIGINT, size, getNewResColId(pCmd), size, false); } pTableMetaInfo = tscGetMetaInfo(pQueryInfo, index.tableIndex); memset(pExpr->base.aliasName, 0, tListLen(pExpr->base.aliasName)); - getColumnName(pItem, pExpr->base.aliasName, sizeof(pExpr->base.aliasName) - 1); + getColumnName(pItem, pExpr->base.aliasName, pExpr->base.token,sizeof(pExpr->base.aliasName) - 1); SColumnList list = createColumnList(1, index.tableIndex, index.columnIndex); if (finalResult) { @@ -2104,21 +2101,21 @@ int32_t addExprAndResultField(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, int32_t col if (pItem->pNode->pParam == NULL || (functionId != TSDB_FUNC_LEASTSQR && taosArrayGetSize(pItem->pNode->pParam) != 1) || (functionId == TSDB_FUNC_LEASTSQR && taosArrayGetSize(pItem->pNode->pParam) != 3)) { /* no parameters or more than one parameter for function */ - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); } tSqlExprItem* pParamElem = taosArrayGet(pItem->pNode->pParam, 0); if (pParamElem->pNode->tokenId != TK_ALL && pParamElem->pNode->tokenId != TK_ID) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); } SColumnIndex index = COLUMN_INDEX_INITIALIZER; if ((getColumnIndexByName(pCmd, &pParamElem->pNode->colInfo, pQueryInfo, &index) != TSDB_CODE_SUCCESS)) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg3); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); } if (index.columnIndex == TSDB_TBNAME_COLUMN_INDEX) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg6); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg6); } // 2. check if sql function can be applied on this column data type @@ -2126,9 +2123,9 @@ int32_t addExprAndResultField(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, int32_t col SSchema* pSchema = tscGetTableColumnSchema(pTableMetaInfo->pTableMeta, index.columnIndex); if (!IS_NUMERIC_TYPE(pSchema->type)) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } else if (IS_UNSIGNED_NUMERIC_TYPE(pSchema->type) && functionId == TSDB_FUNC_DIFF) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg9); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg9); } int16_t resultType = 0; @@ -2137,15 +2134,15 @@ int32_t addExprAndResultField(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, int32_t col if (getResultDataInfo(pSchema->type, pSchema->bytes, functionId, 0, &resultType, &resultSize, &intermediateResSize, 0, false) != TSDB_CODE_SUCCESS) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } // set the first column ts for diff query if (functionId == TSDB_FUNC_DIFF) { colIndex += 1; SColumnIndex indexTS = {.tableIndex = index.tableIndex, .columnIndex = 0}; - SExprInfo* pExpr = tscSqlExprAppend(pQueryInfo, TSDB_FUNC_TS_DUMMY, &indexTS, TSDB_DATA_TYPE_TIMESTAMP, TSDB_KEYSIZE, - getNewResColId(pQueryInfo), TSDB_KEYSIZE, false); + SExprInfo* pExpr = tscExprAppend(pQueryInfo, TSDB_FUNC_TS_DUMMY, &indexTS, TSDB_DATA_TYPE_TIMESTAMP, TSDB_KEYSIZE, + getNewResColId(pCmd), TSDB_KEYSIZE, false); SColumnList ids = createColumnList(1, 0, 0); insertResultField(pQueryInfo, 0, &ids, TSDB_KEYSIZE, TSDB_DATA_TYPE_TIMESTAMP, aAggs[TSDB_FUNC_TS_DUMMY].name, pExpr); @@ -2153,32 +2150,32 @@ int32_t addExprAndResultField(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, int32_t col // functions can not be applied to tags if (index.columnIndex >= tscGetNumOfColumns(pTableMetaInfo->pTableMeta)) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg6); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg6); } - SExprInfo* pExpr = tscSqlExprAppend(pQueryInfo, functionId, &index, resultType, resultSize, getNewResColId(pQueryInfo), resultSize, false); + SExprInfo* pExpr = tscExprAppend(pQueryInfo, functionId, &index, resultType, resultSize, getNewResColId(pCmd), resultSize, false); if (functionId == TSDB_FUNC_LEASTSQR) { /* set the leastsquares parameters */ char val[8] = {0}; if (tVariantDump(&pParamElem[1].pNode->value, val, TSDB_DATA_TYPE_DOUBLE, true) < 0) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } - addExprParams(&pExpr->base, val, TSDB_DATA_TYPE_DOUBLE, DOUBLE_BYTES); + tscExprAddParams(&pExpr->base, val, TSDB_DATA_TYPE_DOUBLE, DOUBLE_BYTES); memset(val, 0, tListLen(val)); if (tVariantDump(&pParamElem[2].pNode->value, val, TSDB_DATA_TYPE_DOUBLE, true) < 0) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } - addExprParams(&pExpr->base, val, TSDB_DATA_TYPE_DOUBLE, sizeof(double)); + tscExprAddParams(&pExpr->base, val, TSDB_DATA_TYPE_DOUBLE, sizeof(double)); } SColumnList ids = createColumnList(1, index.tableIndex, index.columnIndex); memset(pExpr->base.aliasName, 0, tListLen(pExpr->base.aliasName)); - getColumnName(pItem, pExpr->base.aliasName, sizeof(pExpr->base.aliasName) - 1); + getColumnName(pItem, pExpr->base.aliasName, pExpr->base.token,sizeof(pExpr->base.aliasName) - 1); if (finalResult) { int32_t numOfOutput = tscNumOfFields(pQueryInfo); @@ -2207,18 +2204,18 @@ int32_t addExprAndResultField(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, int32_t col if (!requireAllFields) { if (taosArrayGetSize(pItem->pNode->pParam) < 1) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg3); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); } if (taosArrayGetSize(pItem->pNode->pParam) > 1 && (pItem->aliasName != NULL && strlen(pItem->aliasName) > 0)) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg8); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg8); } /* in first/last function, multiple columns can be add to resultset */ for (int32_t i = 0; i < taosArrayGetSize(pItem->pNode->pParam); ++i) { tSqlExprItem* pParamElem = taosArrayGet(pItem->pNode->pParam, i); if (pParamElem->pNode->tokenId != TK_ALL && pParamElem->pNode->tokenId != TK_ID) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg3); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); } SColumnIndex index = COLUMN_INDEX_INITIALIZER; @@ -2227,7 +2224,7 @@ int32_t addExprAndResultField(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, int32_t col SStrToken tmpToken = pParamElem->pNode->colInfo; if (getTableIndexByName(&tmpToken, pQueryInfo, &index) != TSDB_CODE_SUCCESS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg4); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg4); } pTableMetaInfo = tscGetMetaInfo(pQueryInfo, index.tableIndex); @@ -2240,20 +2237,20 @@ int32_t addExprAndResultField(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, int32_t col setResultColName(name, pItem, cvtFunc.originFuncId, &t, true); if (setExprInfoForFunctions(pCmd, pQueryInfo, &pSchema[j], cvtFunc, name, colIndex++, &index, finalResult) != 0) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } } } else { if (getColumnIndexByName(pCmd, &pParamElem->pNode->colInfo, pQueryInfo, &index) != TSDB_CODE_SUCCESS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg3); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); } pTableMetaInfo = tscGetMetaInfo(pQueryInfo, index.tableIndex); // functions can not be applied to tags if ((index.columnIndex >= tscGetNumOfColumns(pTableMetaInfo->pTableMeta)) || (index.columnIndex < 0)) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg6); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg6); } char name[TSDB_COL_NAME_LEN] = {0}; @@ -2263,7 +2260,7 @@ int32_t addExprAndResultField(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, int32_t col setResultColName(name, pItem, cvtFunc.originFuncId, &pParamElem->pNode->colInfo, multiColOutput); if (setExprInfoForFunctions(pCmd, pQueryInfo, pSchema, cvtFunc, name, colIndex++, &index, finalResult) != 0) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } } } @@ -2274,7 +2271,7 @@ int32_t addExprAndResultField(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, int32_t col // multicolumn selection does not support alias name if (pItem->aliasName != NULL && strlen(pItem->aliasName) > 0) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg8); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg8); } for (int32_t j = 0; j < pQueryInfo->numOfTables; ++j) { @@ -2289,7 +2286,7 @@ int32_t addExprAndResultField(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, int32_t col setResultColName(name, pItem, cvtFunc.originFuncId, &t, true); if (setExprInfoForFunctions(pCmd, pQueryInfo, &pSchema[index.columnIndex], cvtFunc, name, colIndex, &index, finalResult) != 0) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } colIndex++; } @@ -2308,21 +2305,21 @@ int32_t addExprAndResultField(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, int32_t col // 1. valid the number of parameters if (pItem->pNode->pParam == NULL || taosArrayGetSize(pItem->pNode->pParam) != 2) { /* no parameters or more than one parameter for function */ - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); } tSqlExprItem* pParamElem = taosArrayGet(pItem->pNode->pParam, 0); if (pParamElem->pNode->tokenId != TK_ID) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); } SColumnIndex index = COLUMN_INDEX_INITIALIZER; if (getColumnIndexByName(pCmd, &pParamElem->pNode->colInfo, pQueryInfo, &index) != TSDB_CODE_SUCCESS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg3); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); } if (index.columnIndex == TSDB_TBNAME_COLUMN_INDEX) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg6); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg6); } pTableMetaInfo = tscGetMetaInfo(pQueryInfo, index.tableIndex); @@ -2330,17 +2327,17 @@ int32_t addExprAndResultField(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, int32_t col // functions can not be applied to tags if (index.columnIndex >= tscGetNumOfColumns(pTableMetaInfo->pTableMeta)) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg6); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg6); } // 2. valid the column type if (!IS_NUMERIC_TYPE(pSchema->type)) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } // 3. valid the parameters if (pParamElem[1].pNode->tokenId == TK_ID) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); } tVariant* pVariant = &pParamElem[1].pNode->value; @@ -2356,7 +2353,7 @@ int32_t addExprAndResultField(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, int32_t col double dp = GET_DOUBLE_VAL(val); if (dp < 0 || dp > TOP_BOTTOM_QUERY_LIMIT) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg5); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg5); } resultSize = sizeof(double); @@ -2370,20 +2367,20 @@ int32_t addExprAndResultField(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, int32_t col tscInsertPrimaryTsSourceColumn(pQueryInfo, pTableMetaInfo->pTableMeta->id.uid); colIndex += 1; // the first column is ts - pExpr = tscSqlExprAppend(pQueryInfo, functionId, &index, resultType, resultSize, getNewResColId(pQueryInfo), resultSize, false); - addExprParams(&pExpr->base, val, TSDB_DATA_TYPE_DOUBLE, sizeof(double)); + pExpr = tscExprAppend(pQueryInfo, functionId, &index, resultType, resultSize, getNewResColId(pCmd), resultSize, false); + tscExprAddParams(&pExpr->base, val, TSDB_DATA_TYPE_DOUBLE, sizeof(double)); } else { tVariantDump(pVariant, val, TSDB_DATA_TYPE_BIGINT, true); int64_t nTop = GET_INT32_VAL(val); if (nTop <= 0 || nTop > 100) { // todo use macro - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg5); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg5); } // todo REFACTOR // set the first column ts for top/bottom query SColumnIndex index1 = {index.tableIndex, PRIMARYKEY_TIMESTAMP_COL_INDEX}; - pExpr = tscSqlExprAppend(pQueryInfo, TSDB_FUNC_TS, &index1, TSDB_DATA_TYPE_TIMESTAMP, TSDB_KEYSIZE, getNewResColId(pQueryInfo), + pExpr = tscExprAppend(pQueryInfo, TSDB_FUNC_TS, &index1, TSDB_DATA_TYPE_TIMESTAMP, TSDB_KEYSIZE, getNewResColId(pCmd), TSDB_KEYSIZE, false); tstrncpy(pExpr->base.aliasName, aAggs[TSDB_FUNC_TS].name, sizeof(pExpr->base.aliasName)); @@ -2394,12 +2391,12 @@ int32_t addExprAndResultField(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, int32_t col colIndex += 1; // the first column is ts - pExpr = tscSqlExprAppend(pQueryInfo, functionId, &index, resultType, resultSize, getNewResColId(pQueryInfo), resultSize, false); - addExprParams(&pExpr->base, val, TSDB_DATA_TYPE_BIGINT, sizeof(int64_t)); + pExpr = tscExprAppend(pQueryInfo, functionId, &index, resultType, resultSize, getNewResColId(pCmd), resultSize, false); + tscExprAddParams(&pExpr->base, val, TSDB_DATA_TYPE_BIGINT, sizeof(int64_t)); } memset(pExpr->base.aliasName, 0, tListLen(pExpr->base.aliasName)); - getColumnName(pItem, pExpr->base.aliasName, sizeof(pExpr->base.aliasName) - 1); + getColumnName(pItem, pExpr->base.aliasName, pExpr->base.token,sizeof(pExpr->base.aliasName) - 1); // todo refactor: tscColumnListInsert part SColumnList ids = createColumnList(1, index.tableIndex, index.columnIndex); @@ -2417,12 +2414,12 @@ int32_t addExprAndResultField(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, int32_t col case TSDB_FUNC_TID_TAG: { pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0); if (UTIL_TABLE_IS_NORMAL_TABLE(pTableMetaInfo)) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg7); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg7); } // no parameters or more than one parameter for function if (pItem->pNode->pParam == NULL || taosArrayGetSize(pItem->pNode->pParam) != 1) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); } tSqlExprItem* pParamItem = taosArrayGet(pItem->pNode->pParam, 0); @@ -2430,7 +2427,7 @@ int32_t addExprAndResultField(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, int32_t col SColumnIndex index = COLUMN_INDEX_INITIALIZER; if (getColumnIndexByName(pCmd, &pParam->colInfo, pQueryInfo, &index) != TSDB_CODE_SUCCESS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg3); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); } pTableMetaInfo = tscGetMetaInfo(pQueryInfo, index.tableIndex); @@ -2439,7 +2436,7 @@ int32_t addExprAndResultField(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, int32_t col // functions can not be applied to normal columns int32_t numOfCols = tscGetNumOfColumns(pTableMetaInfo->pTableMeta); if (index.columnIndex < numOfCols && index.columnIndex != TSDB_TBNAME_COLUMN_INDEX) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg6); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg6); } if (index.columnIndex > 0) { @@ -2455,7 +2452,7 @@ int32_t addExprAndResultField(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, int32_t col } if (colType == TSDB_DATA_TYPE_BOOL) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } tscColumnListInsert(pTableMetaInfo->tagColList, index.columnIndex, pTableMetaInfo->pTableMeta->id.uid, @@ -2480,14 +2477,14 @@ int32_t addExprAndResultField(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, int32_t col s.bytes = bytes; TSDB_QUERY_SET_TYPE(pQueryInfo->type, TSDB_QUERY_TYPE_TAG_FILTER_QUERY); - tscAddFuncInSelectClause(pQueryInfo, 0, TSDB_FUNC_TID_TAG, &index, &s, TSDB_COL_TAG); + tscAddFuncInSelectClause(pQueryInfo, 0, TSDB_FUNC_TID_TAG, &index, &s, TSDB_COL_TAG, getNewResColId(pCmd)); return TSDB_CODE_SUCCESS; } case TSDB_FUNC_BLKINFO: { // no parameters or more than one parameter for function if (pItem->pNode->pParam != NULL && taosArrayGetSize(pItem->pNode->pParam) != 0) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); } SColumnIndex index = {.tableIndex = 0, .columnIndex = TSDB_BLOCK_DIST_COLUMN_INDEX,}; @@ -2501,7 +2498,7 @@ int32_t addExprAndResultField(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, int32_t col s.bytes = bytes; s.type = (uint8_t)resType; - SExprInfo* pExpr = tscAddFuncInSelectClause(pQueryInfo, 0, TSDB_FUNC_BLKINFO, &index, &s, TSDB_COL_TAG); + SExprInfo* pExpr = tscAddFuncInSelectClause(pQueryInfo, 0, TSDB_FUNC_BLKINFO, &index, &s, TSDB_COL_TAG, getNewResColId(pCmd)); pExpr->base.numOfParams = 1; pExpr->base.param[0].i64 = pTableMetaInfo->pTableMeta->tableInfo.rowSize; pExpr->base.param[0].nType = TSDB_DATA_TYPE_BIGINT; @@ -2510,7 +2507,7 @@ int32_t addExprAndResultField(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, int32_t col } default: - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } } @@ -2529,12 +2526,14 @@ static SColumnList createColumnList(int32_t num, int16_t tableIndex, int32_t col return columnList; } -void getColumnName(tSqlExprItem* pItem, char* resultFieldName, int32_t nameLength) { +void getColumnName(tSqlExprItem* pItem, char* resultFieldName, char* rawName, int32_t nameLength) { + int32_t len = ((int32_t)pItem->pNode->token.n < nameLength) ? (int32_t)pItem->pNode->token.n : nameLength; + strncpy(rawName, pItem->pNode->token.z, len); + if (pItem->aliasName != NULL) { - strncpy(resultFieldName, pItem->aliasName, nameLength); + strncpy(resultFieldName, pItem->aliasName, len); } else { - int32_t len = ((int32_t)pItem->pNode->token.n < nameLength) ? (int32_t)pItem->pNode->token.n : nameLength; - strncpy(resultFieldName, pItem->pNode->token.z, len); + strncpy(resultFieldName, rawName, len); } } @@ -2595,7 +2594,7 @@ int32_t doGetColumnIndexByName(SSqlCmd* pCmd, SStrToken* pToken, SQueryInfo* pQu if (colIndex != COLUMN_INDEX_INITIAL_VAL) { if (pIndex->columnIndex != COLUMN_INDEX_INITIAL_VAL) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg0); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg0); } else { pIndex->tableIndex = i; pIndex->columnIndex = colIndex; @@ -2610,14 +2609,14 @@ int32_t doGetColumnIndexByName(SSqlCmd* pCmd, SStrToken* pToken, SQueryInfo* pQu } if (pIndex->columnIndex == COLUMN_INDEX_INITIAL_VAL) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } } if (COLUMN_INDEX_VALIDE(*pIndex)) { return TSDB_CODE_SUCCESS; } else { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } } @@ -2643,7 +2642,7 @@ int32_t getTableIndexImpl(SStrToken* pTableToken, SQueryInfo* pQueryInfo, SColum } if (pIndex->tableIndex < 0) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } return TSDB_CODE_SUCCESS; @@ -2654,7 +2653,7 @@ int32_t getTableIndexByName(SStrToken* pToken, SQueryInfo* pQueryInfo, SColumnIn extractTableNameFromToken(pToken, &tableToken); if (getTableIndexImpl(&tableToken, pQueryInfo, pIndex) != TSDB_CODE_SUCCESS) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } return TSDB_CODE_SUCCESS; @@ -2662,13 +2661,13 @@ int32_t getTableIndexByName(SStrToken* pToken, SQueryInfo* pQueryInfo, SColumnIn int32_t getColumnIndexByName(SSqlCmd* pCmd, const SStrToken* pToken, SQueryInfo* pQueryInfo, SColumnIndex* pIndex) { if (pQueryInfo->pTableMetaInfo == NULL || pQueryInfo->numOfTables == 0) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } SStrToken tmpToken = *pToken; if (getTableIndexByName(&tmpToken, pQueryInfo, pIndex) != TSDB_CODE_SUCCESS) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } return doGetColumnIndexByName(pCmd, &tmpToken, pQueryInfo, pIndex); @@ -2676,8 +2675,7 @@ int32_t getColumnIndexByName(SSqlCmd* pCmd, const SStrToken* pToken, SQueryInfo* int32_t setShowInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) { SSqlCmd* pCmd = &pSql->cmd; - STableMetaInfo* pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, pCmd->clauseIndex, 0); - assert(pCmd->numOfClause == 1); + STableMetaInfo* pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, 0); pCmd->command = TSDB_SQL_SHOW; @@ -2700,20 +2698,20 @@ int32_t setShowInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) { if (pDbPrefixToken->type != 0) { if (pDbPrefixToken->n >= TSDB_DB_NAME_LEN) { // db name is too long - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg3); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); } if (pDbPrefixToken->n <= 0) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg5); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg5); } if (tscValidateName(pDbPrefixToken) != TSDB_CODE_SUCCESS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } int32_t ret = tNameSetDbName(&pTableMetaInfo->name, getAccountId(pSql), pDbPrefixToken); if (ret != TSDB_CODE_SUCCESS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } } @@ -2723,26 +2721,26 @@ int32_t setShowInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) { pPattern->n = strdequote(pPattern->z); if (pPattern->n <= 0) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg6); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg6); } if (!tscValidateTableNameLength(pCmd->payloadLen)) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); } } } else if (showType == TSDB_MGMT_TABLE_VNODES) { if (pShowInfo->prefix.type == 0) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), "No specified ip of dnode"); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), "No specified ip of dnode"); } // show vnodes may be ip addr of dnode in payload SStrToken* pDnodeIp = &pShowInfo->prefix; if (pDnodeIp->n >= TSDB_IPv4ADDR_LEN) { // ip addr is too long - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg3); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); } if (!validateIpAddress(pDnodeIp->z, pDnodeIp->n)) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg4); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg4); } } return TSDB_CODE_SUCCESS; @@ -2758,7 +2756,7 @@ int32_t setKillInfo(SSqlObj* pSql, struct SSqlInfo* pInfo, int32_t killType) { SStrToken* idStr = &(pInfo->pMiscInfo->id); if (idStr->n > TSDB_KILL_MSG_LEN) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } strncpy(pCmd->payload, idStr->z, idStr->n); @@ -2770,7 +2768,7 @@ int32_t setKillInfo(SSqlObj* pSql, struct SSqlInfo* pInfo, int32_t killType) { int32_t connId = (int32_t)strtol(connIdStr, NULL, 10); if (connId <= 0) { memset(pCmd->payload, 0, strlen(pCmd->payload)); - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } if (killType == TSDB_SQL_KILL_CONNECTION) { @@ -2781,9 +2779,9 @@ int32_t setKillInfo(SSqlObj* pSql, struct SSqlInfo* pInfo, int32_t killType) { if (queryId <= 0) { memset(pCmd->payload, 0, strlen(pCmd->payload)); if (killType == TSDB_SQL_KILL_QUERY) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); } else { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg3); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); } } @@ -2805,7 +2803,7 @@ int32_t tscTansformFuncForSTableQuery(SQueryInfo* pQueryInfo) { STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0); if (pTableMetaInfo->pTableMeta == NULL || !UTIL_TABLE_IS_SUPER_TABLE(pTableMetaInfo)) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } assert(tscGetNumOfTags(pTableMetaInfo->pTableMeta) >= 0); @@ -2814,9 +2812,9 @@ int32_t tscTansformFuncForSTableQuery(SQueryInfo* pQueryInfo) { int16_t type = 0; int32_t interBytes = 0; - size_t size = tscSqlExprNumOfExprs(pQueryInfo); + size_t size = tscNumOfExprs(pQueryInfo); for (int32_t k = 0; k < size; ++k) { - SExprInfo* pExpr = tscSqlExprGet(pQueryInfo, k); + SExprInfo* pExpr = tscExprGet(pQueryInfo, k); int16_t functionId = aAggs[pExpr->base.functionId].stableFuncId; int32_t colIndex = pExpr->base.colInfo.colIndex; @@ -2827,10 +2825,10 @@ int32_t tscTansformFuncForSTableQuery(SQueryInfo* pQueryInfo) { (functionId >= TSDB_FUNC_RATE && functionId <= TSDB_FUNC_AVG_IRATE)) { if (getResultDataInfo(pSrcSchema->type, pSrcSchema->bytes, functionId, (int32_t)pExpr->base.param[0].i64, &type, &bytes, &interBytes, 0, true) != TSDB_CODE_SUCCESS) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } - tscSqlExprUpdate(pQueryInfo, k, functionId, pExpr->base.colInfo.colIndex, TSDB_DATA_TYPE_BINARY, bytes); + tscExprUpdate(pQueryInfo, k, functionId, pExpr->base.colInfo.colIndex, TSDB_DATA_TYPE_BINARY, bytes); // todo refactor pExpr->base.interBytes = interBytes; } @@ -2847,9 +2845,9 @@ void tscRestoreFuncForSTableQuery(SQueryInfo* pQueryInfo) { return; } - size_t size = tscSqlExprNumOfExprs(pQueryInfo); + size_t size = tscNumOfExprs(pQueryInfo); for (int32_t i = 0; i < size; ++i) { - SExprInfo* pExpr = tscSqlExprGet(pQueryInfo, i); + SExprInfo* pExpr = tscExprGet(pQueryInfo, i); SSchema* pSchema = tscGetTableColumnSchema(pTableMetaInfo->pTableMeta, pExpr->base.colInfo.colIndex); // the final result size and type in the same as query on single table. @@ -2880,28 +2878,28 @@ bool hasUnsupportFunctionsForSTableQuery(SSqlCmd* pCmd, SQueryInfo* pQueryInfo) const char* msg3 = "function not support for super table query"; // filter sql function not supported by metric query yet. - size_t size = tscSqlExprNumOfExprs(pQueryInfo); + size_t size = tscNumOfExprs(pQueryInfo); for (int32_t i = 0; i < size; ++i) { - int32_t functionId = tscSqlExprGet(pQueryInfo, i)->base.functionId; + int32_t functionId = tscExprGet(pQueryInfo, i)->base.functionId; if ((aAggs[functionId].status & TSDB_FUNCSTATE_STABLE) == 0) { - invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg3); + invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); return true; } } if (tscIsTWAQuery(pQueryInfo)) { if (pQueryInfo->groupbyExpr.numOfGroupCols == 0) { - invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); return true; } if (pQueryInfo->groupbyExpr.numOfGroupCols != 1) { - invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); return true; } else { SColIndex* pColIndex = taosArrayGet(pQueryInfo->groupbyExpr.columnInfo, 0); if (pColIndex->colIndex != TSDB_TBNAME_COLUMN_INDEX) { - invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); return true; } } @@ -2930,10 +2928,10 @@ static bool groupbyTagsOrNull(SQueryInfo* pQueryInfo) { static bool functionCompatibleCheck(SQueryInfo* pQueryInfo, bool joinQuery, bool twQuery) { int32_t startIdx = 0; - size_t numOfExpr = tscSqlExprNumOfExprs(pQueryInfo); + size_t numOfExpr = tscNumOfExprs(pQueryInfo); assert(numOfExpr > 0); - SExprInfo* pExpr = tscSqlExprGet(pQueryInfo, startIdx); + SExprInfo* pExpr = tscExprGet(pQueryInfo, startIdx); // ts function can be simultaneously used with any other functions. int32_t functionID = pExpr->base.functionId; @@ -2941,18 +2939,18 @@ static bool functionCompatibleCheck(SQueryInfo* pQueryInfo, bool joinQuery, bool startIdx++; } - int32_t factor = functionCompatList[tscSqlExprGet(pQueryInfo, startIdx)->base.functionId]; + int32_t factor = functionCompatList[tscExprGet(pQueryInfo, startIdx)->base.functionId]; - if (tscSqlExprGet(pQueryInfo, 0)->base.functionId == TSDB_FUNC_LAST_ROW && (joinQuery || twQuery || !groupbyTagsOrNull(pQueryInfo))) { + if (tscExprGet(pQueryInfo, 0)->base.functionId == TSDB_FUNC_LAST_ROW && (joinQuery || twQuery || !groupbyTagsOrNull(pQueryInfo))) { return false; } // diff function cannot be executed with other function // arithmetic function can be executed with other arithmetic functions - size_t size = tscSqlExprNumOfExprs(pQueryInfo); + size_t size = tscNumOfExprs(pQueryInfo); for (int32_t i = startIdx + 1; i < size; ++i) { - SExprInfo* pExpr1 = tscSqlExprGet(pQueryInfo, i); + SExprInfo* pExpr1 = tscExprGet(pQueryInfo, i); int16_t functionId = pExpr1->base.functionId; if (functionId == TSDB_FUNC_TAGPRJ || functionId == TSDB_FUNC_TAG || functionId == TSDB_FUNC_TS) { @@ -3001,11 +2999,11 @@ int32_t validateGroupbyNode(SQueryInfo* pQueryInfo, SArray* pList, SSqlCmd* pCmd pQueryInfo->groupbyExpr.numOfGroupCols = (int16_t)taosArrayGetSize(pList); if (pQueryInfo->groupbyExpr.numOfGroupCols > TSDB_MAX_TAGS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } if (pQueryInfo->numOfTables > 1) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg4); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg4); } STableMeta* pTableMeta = NULL; @@ -3022,13 +3020,13 @@ int32_t validateGroupbyNode(SQueryInfo* pQueryInfo, SArray* pList, SSqlCmd* pCmd SColumnIndex index = COLUMN_INDEX_INITIALIZER; if (getColumnIndexByName(pCmd, &token, pQueryInfo, &index) != TSDB_CODE_SUCCESS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); } if (tableIndex == COLUMN_INDEX_INITIAL_VAL) { tableIndex = index.tableIndex; } else if (tableIndex != index.tableIndex) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg3); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); } pTableMetaInfo = tscGetMetaInfo(pQueryInfo, index.tableIndex); @@ -3046,14 +3044,14 @@ int32_t validateGroupbyNode(SQueryInfo* pQueryInfo, SArray* pList, SSqlCmd* pCmd groupTag = true; } - SSqlGroupbyExpr* pGroupExpr = &pQueryInfo->groupbyExpr; + SGroupbyExpr* pGroupExpr = &pQueryInfo->groupbyExpr; if (pGroupExpr->columnInfo == NULL) { pGroupExpr->columnInfo = taosArrayInit(4, sizeof(SColIndex)); } if (groupTag) { if (!UTIL_TABLE_IS_SUPER_TABLE(pTableMetaInfo)) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg9); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg9); } int32_t relIndex = index.columnIndex; @@ -3062,6 +3060,7 @@ int32_t validateGroupbyNode(SQueryInfo* pQueryInfo, SArray* pList, SSqlCmd* pCmd } SColIndex colIndex = { .colIndex = relIndex, .flag = TSDB_COL_TAG, .colId = pSchema->colId, }; + strncpy(colIndex.name, pSchema->name, tListLen(colIndex.name)); taosArrayPush(pGroupExpr->columnInfo, &colIndex); index.columnIndex = relIndex; @@ -3069,17 +3068,19 @@ int32_t validateGroupbyNode(SQueryInfo* pQueryInfo, SArray* pList, SSqlCmd* pCmd } else { // check if the column type is valid, here only support the bool/tinyint/smallint/bigint group by if (pSchema->type == TSDB_DATA_TYPE_TIMESTAMP || pSchema->type == TSDB_DATA_TYPE_FLOAT || pSchema->type == TSDB_DATA_TYPE_DOUBLE) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg8); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg8); } tscColumnListInsert(pQueryInfo->colList, index.columnIndex, pTableMeta->id.uid, pSchema); SColIndex colIndex = { .colIndex = index.columnIndex, .flag = TSDB_COL_NORMAL, .colId = pSchema->colId }; + strncpy(colIndex.name, pSchema->name, tListLen(colIndex.name)); + taosArrayPush(pGroupExpr->columnInfo, &colIndex); pQueryInfo->groupbyExpr.orderType = TSDB_ORDER_ASC; if (i == 0 && num > 1) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg7); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg7); } } } @@ -3158,7 +3159,7 @@ static int32_t doExtractColumnFilterInfo(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, } if (retVal != TSDB_CODE_SUCCESS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg); } switch (pExpr->tokenId) { @@ -3190,7 +3191,7 @@ static int32_t doExtractColumnFilterInfo(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, pColumnFilter->lowerRelOptr = TSDB_RELATION_NOTNULL; break; default: - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg); } return TSDB_CODE_SUCCESS; @@ -3218,7 +3219,7 @@ static int32_t tablenameListToString(tSqlExpr* pExpr, SStringBuilder* sb) { int32_t size = (int32_t) taosArrayGetSize(pList); if (size <= 0) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } if (size > 0) { @@ -3236,7 +3237,7 @@ static int32_t tablenameListToString(tSqlExpr* pExpr, SStringBuilder* sb) { } if (pVar->nLen <= 0 || !tscValidateTableNameLength(pVar->nLen)) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } } @@ -3292,7 +3293,7 @@ static int32_t extractColumnFilterInfo(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SC return TSDB_CODE_TSC_OUT_OF_MEMORY; } } else { // error; - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } pColFilter->filterstr = @@ -3305,17 +3306,17 @@ static int32_t extractColumnFilterInfo(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SC && pExpr->tokenId != TK_NOTNULL && pExpr->tokenId != TK_LIKE ) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); } } else { if (pExpr->tokenId == TK_LIKE) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } if (pSchema->type == TSDB_DATA_TYPE_BOOL) { int32_t t = pExpr->tokenId; if (t != TK_EQ && t != TK_NE && t != TK_NOTNULL && t != TK_ISNULL) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg3); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); } } } @@ -3339,7 +3340,7 @@ static int32_t getTablenameCond(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, tSqlExpr* tSqlExpr* pRight = pTableCond->pRight; if (!isTablenameToken(&pLeft->colInfo)) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } int32_t ret = TSDB_CODE_SUCCESS; @@ -3348,14 +3349,14 @@ static int32_t getTablenameCond(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, tSqlExpr* ret = tablenameListToString(pRight, sb); } else if (pTableCond->tokenId == TK_LIKE) { if (pRight->tokenId != TK_STRING) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } ret = tablenameCondToString(pRight, sb); } if (ret != TSDB_CODE_SUCCESS) { - invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg0); + invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg0); } return ret; @@ -3376,7 +3377,7 @@ static int32_t getColumnQueryCondInfo(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, tSq } else { // handle leaf node SColumnIndex index = COLUMN_INDEX_INITIALIZER; if (getColumnIndexByName(pCmd, &pExpr->pLeft->colInfo, pQueryInfo, &index) != TSDB_CODE_SUCCESS) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } return extractColumnFilterInfo(pCmd, pQueryInfo, &index, pExpr, relOptr); @@ -3405,7 +3406,7 @@ static int32_t checkAndSetJoinCondInfo(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, tS SColumnIndex index = COLUMN_INDEX_INITIALIZER; if (getColumnIndexByName(pCmd, &pExpr->pLeft->colInfo, pQueryInfo, &index) != TSDB_CODE_SUCCESS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg4); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg4); } STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, index.tableIndex); @@ -3415,7 +3416,7 @@ static int32_t checkAndSetJoinCondInfo(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, tS SJoinNode **leftNode = &pQueryInfo->tagCond.joinInfo.joinTables[index.tableIndex]; if (*leftNode == NULL) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } (*leftNode)->uid = pTableMetaInfo->pTableMeta->id.uid; @@ -3429,7 +3430,7 @@ static int32_t checkAndSetJoinCondInfo(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, tS tscColumnListInsert(pTableMetaInfo->tagColList, index.columnIndex, pTableMeta->id.uid, pTagSchema1); if (taosArrayGetSize(pTableMetaInfo->tagColList) > 1) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); } } } @@ -3438,7 +3439,7 @@ static int32_t checkAndSetJoinCondInfo(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, tS index = (SColumnIndex)COLUMN_INDEX_INITIALIZER; if (getColumnIndexByName(pCmd, &pExpr->pRight->colInfo, pQueryInfo, &index) != TSDB_CODE_SUCCESS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg4); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg4); } pTableMetaInfo = tscGetMetaInfo(pQueryInfo, index.tableIndex); @@ -3448,7 +3449,7 @@ static int32_t checkAndSetJoinCondInfo(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, tS SJoinNode **rightNode = &pQueryInfo->tagCond.joinInfo.joinTables[index.tableIndex]; if (*rightNode == NULL) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } (*rightNode)->uid = pTableMetaInfo->pTableMeta->id.uid; @@ -3461,7 +3462,7 @@ static int32_t checkAndSetJoinCondInfo(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, tS tscColumnListInsert(pTableMetaInfo->tagColList, index.columnIndex, pTableMeta->id.uid, pTagSchema2); if (taosArrayGetSize(pTableMetaInfo->tagColList) > 1) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); } } } @@ -3469,7 +3470,7 @@ static int32_t checkAndSetJoinCondInfo(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, tS int16_t rightIdx = index.tableIndex; if (pTagSchema1->type != pTagSchema2->type) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg3); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); } if ((*leftNode)->tagJoin == NULL) { @@ -3503,12 +3504,12 @@ static int32_t validateSQLExpr(SSqlCmd* pCmd, tSqlExpr* pExpr, SQueryInfo* pQuer if (*type == NON_ARITHMEIC_EXPR) { *type = NORMAL_ARITHMETIC; } else if (*type == AGG_ARIGHTMEIC) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } SColumnIndex index = COLUMN_INDEX_INITIALIZER; if (getColumnIndexByName(pCmd, &pExpr->colInfo, pQueryInfo, &index) != TSDB_CODE_SUCCESS) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } // if column is timestamp, bool, binary, nchar, not support arithmetic, so return invalid sql @@ -3517,21 +3518,21 @@ static int32_t validateSQLExpr(SSqlCmd* pCmd, tSqlExpr* pExpr, SQueryInfo* pQuer if ((pSchema->type == TSDB_DATA_TYPE_TIMESTAMP) || (pSchema->type == TSDB_DATA_TYPE_BOOL) || (pSchema->type == TSDB_DATA_TYPE_BINARY) || (pSchema->type == TSDB_DATA_TYPE_NCHAR)) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } pList->ids[pList->num++] = index; } else if ((pExpr->tokenId == TK_FLOAT && (isnan(pExpr->value.dKey) || isinf(pExpr->value.dKey))) || pExpr->tokenId == TK_NULL) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } else if (pExpr->type == SQL_NODE_SQLFUNCTION) { if (*type == NON_ARITHMEIC_EXPR) { *type = AGG_ARIGHTMEIC; } else if (*type == NORMAL_ARITHMETIC) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } - int32_t outputIndex = (int32_t)tscSqlExprNumOfExprs(pQueryInfo); + int32_t outputIndex = (int32_t)tscNumOfExprs(pQueryInfo); tSqlExprItem item = {.pNode = pExpr, .aliasName = NULL}; @@ -3539,32 +3540,32 @@ static int32_t validateSQLExpr(SSqlCmd* pCmd, tSqlExpr* pExpr, SQueryInfo* pQuer // Append the sqlExpr into exprList of pQueryInfo structure sequentially pExpr->functionId = isValidFunction(pExpr->operand.z, pExpr->operand.n); if (pExpr->functionId < 0) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } if (addExprAndResultField(pCmd, pQueryInfo, outputIndex, &item, false) != TSDB_CODE_SUCCESS) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } // It is invalid in case of more than one sqlExpr, such as first(ts, k) - last(ts, k) - int32_t inc = (int32_t) tscSqlExprNumOfExprs(pQueryInfo) - outputIndex; + int32_t inc = (int32_t) tscNumOfExprs(pQueryInfo) - outputIndex; if (inc > 1) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } // Not supported data type in arithmetic expression uint64_t id = -1; for(int32_t i = 0; i < inc; ++i) { - SExprInfo* p1 = tscSqlExprGet(pQueryInfo, i + outputIndex); + SExprInfo* p1 = tscExprGet(pQueryInfo, i + outputIndex); int16_t t = p1->base.resType; if (t == TSDB_DATA_TYPE_BINARY || t == TSDB_DATA_TYPE_NCHAR || t == TSDB_DATA_TYPE_BOOL || t == TSDB_DATA_TYPE_TIMESTAMP) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } if (i == 0) { id = p1->base.uid; } else if (id != p1->base.uid) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } } @@ -3609,7 +3610,7 @@ static int32_t validateArithmeticSQLExpr(SSqlCmd* pCmd, tSqlExpr* pExpr, SQueryI // the expression not from the same table, return error if (uidLeft != uidRight && uidLeft != 0 && uidRight != 0) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } } @@ -3693,14 +3694,14 @@ static bool validateJoinExprNode(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, tSqlExpr } if (pExpr->tokenId != TK_EQ) { - invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); return false; } SColumnIndex rightIndex = COLUMN_INDEX_INITIALIZER; if (getColumnIndexByName(pCmd, &pRight->colInfo, pQueryInfo, &rightIndex) != TSDB_CODE_SUCCESS) { - invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); return false; } @@ -3709,21 +3710,25 @@ static bool validateJoinExprNode(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, tSqlExpr SSchema* pLeftSchema = tscGetTableSchema(pLeftMeterMeta->pTableMeta); int16_t leftType = pLeftSchema[pLeftIndex->columnIndex].type; + tscColumnListInsert(pQueryInfo->colList, pLeftIndex->columnIndex, pLeftMeterMeta->pTableMeta->id.uid, &pLeftSchema[pLeftIndex->columnIndex]); + STableMetaInfo* pRightMeterMeta = tscGetMetaInfo(pQueryInfo, rightIndex.tableIndex); SSchema* pRightSchema = tscGetTableSchema(pRightMeterMeta->pTableMeta); int16_t rightType = pRightSchema[rightIndex.columnIndex].type; + tscColumnListInsert(pQueryInfo->colList, rightIndex.columnIndex, pRightMeterMeta->pTableMeta->id.uid, &pRightSchema[rightIndex.columnIndex]); + if (leftType != rightType) { - invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg3); + invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); return false; } else if (pLeftIndex->tableIndex == rightIndex.tableIndex) { - invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg4); + invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg4); return false; } // table to table/ super table to super table are allowed if (UTIL_TABLE_IS_SUPER_TABLE(pLeftMeterMeta) != UTIL_TABLE_IS_SUPER_TABLE(pRightMeterMeta)) { - invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg5); + invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg5); return false; } @@ -3745,7 +3750,7 @@ static bool validTableNameOptr(tSqlExpr* pExpr) { static int32_t setExprToCond(tSqlExpr** parent, tSqlExpr* pExpr, const char* msg, int32_t parentOptr, char* msgBuf) { if (*parent != NULL) { if (parentOptr == TK_OR && msg != NULL) { - return invalidSqlErrMsg(msgBuf, msg); + return invalidOperationMsg(msgBuf, msg); } *parent = tSqlExprCreate((*parent), pExpr, parentOptr); @@ -3761,7 +3766,7 @@ static int32_t validateNullExpr(tSqlExpr* pExpr, char* msgBuf) { tSqlExpr* pRight = pExpr->pRight; if (pRight->tokenId == TK_NULL && (!(pExpr->tokenId == TK_ISNULL || pExpr->tokenId == TK_NOTNULL))) { - return invalidSqlErrMsg(msgBuf, msg); + return invalidOperationMsg(msgBuf, msg); } return TSDB_CODE_SUCCESS; @@ -3777,12 +3782,12 @@ static int32_t validateLikeExpr(tSqlExpr* pExpr, STableMeta* pTableMeta, int32_t if (pExpr->tokenId == TK_LIKE) { if (pRight->value.nLen > TSDB_PATTERN_STRING_MAX_LEN) { - return invalidSqlErrMsg(msgBuf, msg1); + return invalidOperationMsg(msgBuf, msg1); } SSchema* pSchema = tscGetTableSchema(pTableMeta); if ((!isTablenameToken(&pLeft->colInfo)) && !IS_VAR_DATA_TYPE(pSchema[index].type)) { - return invalidSqlErrMsg(msgBuf, msg2); + return invalidOperationMsg(msgBuf, msg2); } } @@ -3806,7 +3811,7 @@ static int32_t handleExprInQueryCond(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, tSql SColumnIndex index = COLUMN_INDEX_INITIALIZER; if (getColumnIndexByName(pCmd, &pLeft->colInfo, pQueryInfo, &index) != TSDB_CODE_SUCCESS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); } assert(tSqlExprIsParentOfLeaf(*pExpr)); @@ -3828,7 +3833,7 @@ static int32_t handleExprInQueryCond(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, tSql if (index.columnIndex == PRIMARYKEY_TIMESTAMP_COL_INDEX) { // query on time range if (!validateJoinExprNode(pCmd, pQueryInfo, *pExpr, &index)) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } // set join query condition @@ -3848,11 +3853,11 @@ static int32_t handleExprInQueryCond(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, tSql int16_t leftIdx = index.tableIndex; if (getColumnIndexByName(pCmd, &pRight->colInfo, pQueryInfo, &index) != TSDB_CODE_SUCCESS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); } if (index.tableIndex < 0 || index.tableIndex >= TSDB_MAX_JOIN_TABLE_NUM) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg4); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg4); } SJoinNode **rightNode = &pQueryInfo->tagCond.joinInfo.joinTables[index.tableIndex]; @@ -3877,7 +3882,7 @@ static int32_t handleExprInQueryCond(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, tSql taosArrayPush((*rightNode)->tsJoin, &leftIdx); /* - * to release expression, e.g., m1.ts = m2.ts, + * To release expression, e.g., m1.ts = m2.ts, * since this expression is used to set the join query type */ tSqlExprDestroy(*pExpr); @@ -3890,17 +3895,17 @@ static int32_t handleExprInQueryCond(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, tSql } else if (index.columnIndex >= tscGetNumOfColumns(pTableMeta) || index.columnIndex == TSDB_TBNAME_COLUMN_INDEX) { // query on tags, check for tag query condition if (UTIL_TABLE_IS_NORMAL_TABLE(pTableMetaInfo)) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } // in case of in operator, keep it in a seprate attribute if (index.columnIndex == TSDB_TBNAME_COLUMN_INDEX) { if (!validTableNameOptr(*pExpr)) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg7); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg7); } if (!UTIL_TABLE_IS_SUPER_TABLE(pTableMetaInfo)) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } if (pCondExpr->pTableCond == NULL) { @@ -3908,7 +3913,7 @@ static int32_t handleExprInQueryCond(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, tSql pCondExpr->relType = parentOptr; pCondExpr->tableCondIndex = index.tableIndex; } else { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg6); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg6); } *type = TSQL_EXPR_TBNAME; @@ -3916,7 +3921,7 @@ static int32_t handleExprInQueryCond(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, tSql } else { if (pRight != NULL && pRight->tokenId == TK_ID) { // join on tag columns for stable query if (!validateJoinExprNode(pCmd, pQueryInfo, *pExpr, &index)) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } pQueryInfo->type |= TSDB_QUERY_TYPE_JOIN_QUERY; @@ -3935,7 +3940,7 @@ static int32_t handleExprInQueryCond(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, tSql *type = TSQL_EXPR_COLUMN; if (pRight->tokenId == TK_ID) { // other column cannot be served as the join column - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg5); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg5); } ret = setExprToCond(&pCondExpr->pColumnCond, *pExpr, NULL, parentOptr, pQueryInfo->msg); @@ -3954,14 +3959,14 @@ int32_t getQueryCondExpr(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, tSqlExpr** pExpr const char* msg1 = "query condition between different columns must use 'AND'"; if ((*pExpr)->flags & (1 << EXPR_FLAG_TS_ERROR)) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } tSqlExpr* pLeft = (*pExpr)->pLeft; tSqlExpr* pRight = (*pExpr)->pRight; if (!isValidExpr(pLeft, pRight, (*pExpr)->tokenId)) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } int32_t leftType = -1; @@ -3984,7 +3989,7 @@ int32_t getQueryCondExpr(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, tSqlExpr** pExpr */ if (leftType != rightType) { if ((*pExpr)->tokenId == TK_OR && (leftType + rightType != TSQL_EXPR_TBNAME + TSQL_EXPR_TAG)) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } } @@ -3995,11 +4000,11 @@ int32_t getQueryCondExpr(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, tSqlExpr** pExpr exchangeExpr(*pExpr); if (pLeft->tokenId == TK_ID && pRight->tokenId == TK_TIMESTAMP && (pRight->flags & (1 << EXPR_FLAG_TIMESTAMP_VAR))) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } if ((pLeft->flags & (1 << EXPR_FLAG_TS_ERROR)) || (pRight->flags & (1 << EXPR_FLAG_TS_ERROR))) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } return handleExprInQueryCond(pCmd, pQueryInfo, pExpr, pCondExpr, type, parentOptr); @@ -4111,7 +4116,7 @@ static int32_t setTableCondForSTableQuery(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, taosStringBuilderDestroy(&sb1); tfree(segments); - invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg); + invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg); return ret; } @@ -4166,7 +4171,7 @@ static int32_t getTimeRangeFromExpr(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, tSqlE if (!tSqlExprIsParentOfLeaf(pExpr)) { if (pExpr->tokenId == TK_OR) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } code = getTimeRangeFromExpr(pCmd, pQueryInfo, pExpr->pLeft); @@ -4178,7 +4183,7 @@ static int32_t getTimeRangeFromExpr(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, tSqlE } else { SColumnIndex index = COLUMN_INDEX_INITIALIZER; if (getColumnIndexByName(pCmd, &pExpr->pLeft->colInfo, pQueryInfo, &index) != TSDB_CODE_SUCCESS) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, index.tableIndex); @@ -4188,7 +4193,7 @@ static int32_t getTimeRangeFromExpr(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, tSqlE STimeWindow win = {.skey = INT64_MIN, .ekey = INT64_MAX}; if (getTimeRange(&win, pRight, pExpr->tokenId, tinfo.precision) != TSDB_CODE_SUCCESS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg0); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg0); } // update the timestamp query range @@ -4213,7 +4218,7 @@ static int32_t validateJoinExpr(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SCondExpr if (pQueryInfo->numOfTables == 1) { return TSDB_CODE_SUCCESS; } else { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg3); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); } } @@ -4221,12 +4226,12 @@ static int32_t validateJoinExpr(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SCondExpr if (UTIL_TABLE_IS_SUPER_TABLE(pTableMetaInfo)) { // for stable join, tag columns // must be present for join if (pCondExpr->pJoinExpr == NULL) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } } if (!pCondExpr->tsJoin) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); } return TSDB_CODE_SUCCESS; @@ -4297,7 +4302,7 @@ static int32_t validateTagCondExpr(SSqlCmd* pCmd, tExprNode *p) { } if (IS_ARITHMETIC_OPTR(p->_node.optr)) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } if (!IS_RELATION_OPTR(p->_node.optr)) { @@ -4355,7 +4360,7 @@ static int32_t validateTagCondExpr(SSqlCmd* pCmd, tExprNode *p) { } if (retVal != TSDB_CODE_SUCCESS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); } } while (0); @@ -4416,7 +4421,7 @@ static int32_t getTagQueryCondExpr(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SCondE taosArrayDestroy(colList); if (pQueryInfo->tagCond.pCond != NULL && taosArrayGetSize(pQueryInfo->tagCond.pCond) > 0 && !UTIL_TABLE_IS_SUPER_TABLE(pTableMetaInfo)) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), "filter on tag not supported for normal table"); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), "filter on tag not supported for normal table"); } if (ret) { @@ -4436,7 +4441,7 @@ int32_t validateJoinNodes(SQueryInfo* pQueryInfo, SSqlObj* pSql) { SJoinNode *node = pQueryInfo->tagCond.joinInfo.joinTables[i]; if (node == NULL || node->tsJoin == NULL || taosArrayGetSize(node->tsJoin) <= 0) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(&pSql->cmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(&pSql->cmd), msg1); } } @@ -4446,7 +4451,7 @@ int32_t validateJoinNodes(SQueryInfo* pQueryInfo, SSqlObj* pSql) { SJoinNode *node = pQueryInfo->tagCond.joinInfo.joinTables[i]; if (node == NULL || node->tagJoin == NULL || taosArrayGetSize(node->tagJoin) <= 0) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(&pSql->cmd), msg2); + return invalidOperationMsg(tscGetErrorMsgPayload(&pSql->cmd), msg2); } } } @@ -4494,7 +4499,7 @@ int32_t mergeJoinNodes(SQueryInfo* pQueryInfo, SSqlObj* pSql) { } if (taosArrayGetSize(pQueryInfo->tagCond.joinInfo.joinTables[0]->tsJoin) != pQueryInfo->numOfTables) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(&pSql->cmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(&pSql->cmd), msg1); } STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0); @@ -4515,7 +4520,7 @@ int32_t mergeJoinNodes(SQueryInfo* pQueryInfo, SSqlObj* pSql) { } if (taosArrayGetSize(pQueryInfo->tagCond.joinInfo.joinTables[0]->tagJoin) != pQueryInfo->numOfTables) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(&pSql->cmd), msg2); + return invalidOperationMsg(tscGetErrorMsgPayload(&pSql->cmd), msg2); } } @@ -4539,7 +4544,7 @@ int32_t validateWhereNode(SQueryInfo* pQueryInfo, tSqlExpr** pExpr, SSqlObj* pSq SCondExpr condExpr = {0}; if ((*pExpr)->pLeft == NULL || (*pExpr)->pRight == NULL) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(&pSql->cmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(&pSql->cmd), msg1); } int32_t type = 0; @@ -4592,7 +4597,7 @@ int32_t validateWhereNode(SQueryInfo* pQueryInfo, tSqlExpr** pExpr, SSqlObj* pSq } if (!validateFilterExpr(pQueryInfo)) { - ret = invalidSqlErrMsg(tscGetErrorMsgPayload(&pSql->cmd), msg2); + ret = invalidOperationMsg(tscGetErrorMsgPayload(&pSql->cmd), msg2); goto PARSE_WHERE_EXIT; } @@ -4626,7 +4631,7 @@ int32_t getTimeRange(STimeWindow* win, tSqlExpr* pRight, int32_t optr, int16_t t * where ts in ('2015-12-12 4:8:12') */ if (pRight->tokenId == TK_SET || optr == TK_IN) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } int64_t val = 0; @@ -4639,14 +4644,14 @@ int32_t getTimeRange(STimeWindow* win, tSqlExpr* pRight, int32_t optr, int16_t t if (taosParseTime(pRight->value.pz, &val, pRight->value.nLen, TSDB_TIME_PRECISION_MICRO, tsDaylight) == TSDB_CODE_SUCCESS) { parsed = true; } else { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } } else { SStrToken token = {.z = pRight->value.pz, .n = pRight->value.nLen, .type = TK_ID}; int32_t len = tGetToken(pRight->value.pz, &token.type); if ((token.type != TK_INTEGER && token.type != TK_FLOAT) || len != pRight->value.nLen) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } } } else if (pRight->tokenId == TK_INTEGER && timePrecision == TSDB_TIME_PRECISION_MILLI) { @@ -4727,7 +4732,7 @@ int32_t tsRewriteFieldNameIfNecessary(SSqlCmd* pCmd, SQueryInfo* pQueryInfo) { for (int32_t j = i + 1; j < pQueryInfo->fieldsInfo.numOfOutput; ++j) { if (strncasecmp(fieldName, tscFieldInfoGetField(&pQueryInfo->fieldsInfo, j)->name, (TSDB_COL_NAME_LEN - 1)) == 0) { const char* msg = "duplicated column name in new table"; - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg); } } } @@ -4752,7 +4757,7 @@ int32_t validateFillNode(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SSqlNode* pSqlNo const char* msg5 = "fill only available for interval query"; if ((!isTimeWindowQuery(pQueryInfo)) && (!tscIsPointInterpQuery(pQueryInfo))) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg5); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg5); } /* @@ -4760,12 +4765,12 @@ int32_t validateFillNode(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SSqlNode* pSqlNo * the columns may be increased due to group by operation */ if (checkQueryRangeForFill(pCmd, pQueryInfo) != TSDB_CODE_SUCCESS) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } if (pItem->pVar.nType != TSDB_DATA_TYPE_BINARY) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); } size_t numOfFields = tscNumOfFields(pQueryInfo); @@ -4796,7 +4801,7 @@ int32_t validateFillNode(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SSqlNode* pSqlNo size_t num = taosArrayGetSize(pFillToken); if (num == 1) { // no actual value, return with error code - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } int32_t startPos = 1; @@ -4826,7 +4831,7 @@ int32_t validateFillNode(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SSqlNode* pSqlNo tVariant* p = taosArrayGet(pFillToken, j); int32_t ret = tVariantDump(p, (char*)&pQueryInfo->fillVal[i], pField->type, true); if (ret != TSDB_CODE_SUCCESS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg4); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg4); } } @@ -4844,14 +4849,14 @@ int32_t validateFillNode(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SSqlNode* pSqlNo } } } else { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); } - size_t numOfExprs = tscSqlExprNumOfExprs(pQueryInfo); + size_t numOfExprs = tscNumOfExprs(pQueryInfo); for(int32_t i = 0; i < numOfExprs; ++i) { - SExprInfo* pExpr = tscSqlExprGet(pQueryInfo, i); + SExprInfo* pExpr = tscExprGet(pQueryInfo, i); if (pExpr->base.functionId == TSDB_FUNC_TOP || pExpr->base.functionId == TSDB_FUNC_BOTTOM) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg3); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); } } @@ -4905,11 +4910,11 @@ int32_t validateOrderbyNode(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SSqlNode* pSq size_t size = taosArrayGetSize(pSortorder); if (UTIL_TABLE_IS_NORMAL_TABLE(pTableMetaInfo)) { if (size > 1) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg0); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg0); } } else { if (size > 2) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg3); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); } } @@ -4926,7 +4931,7 @@ int32_t validateOrderbyNode(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SSqlNode* pSq if (UTIL_TABLE_IS_SUPER_TABLE(pTableMetaInfo)) { // super table query if (getColumnIndexByName(pCmd, &columnName, pQueryInfo, &index) != TSDB_CODE_SUCCESS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } bool orderByTags = false; @@ -4937,7 +4942,7 @@ int32_t validateOrderbyNode(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SSqlNode* pSq // it is a tag column if (pQueryInfo->groupbyExpr.columnInfo == NULL) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); } SColIndex* pColIndex = taosArrayGet(pQueryInfo->groupbyExpr.columnInfo, 0); if (relTagIndex == pColIndex->colIndex) { @@ -4952,7 +4957,7 @@ int32_t validateOrderbyNode(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SSqlNode* pSq } if (!(orderByTags || orderByTS) && !isTopBottomQuery(pQueryInfo)) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg3); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); } else { // order by top/bottom result value column is not supported in case of interval query. assert(!(orderByTags && orderByTS)); } @@ -4966,12 +4971,12 @@ int32_t validateOrderbyNode(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SSqlNode* pSq pQueryInfo->groupbyExpr.orderType = p1->sortOrder; } else if (isTopBottomQuery(pQueryInfo)) { /* order of top/bottom query in interval is not valid */ - SExprInfo* pExpr = tscSqlExprGet(pQueryInfo, 0); + SExprInfo* pExpr = tscExprGet(pQueryInfo, 0); assert(pExpr->base.functionId == TSDB_FUNC_TS); - pExpr = tscSqlExprGet(pQueryInfo, 1); + pExpr = tscExprGet(pQueryInfo, 1); if (pExpr->base.colInfo.colIndex != index.columnIndex && index.columnIndex != PRIMARYKEY_TIMESTAMP_COL_INDEX) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); } tVariantListItem* p1 = taosArrayGet(pSqlNode->pSortOrder, 0); @@ -4986,7 +4991,7 @@ int32_t validateOrderbyNode(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SSqlNode* pSq // orderby ts query on super table if (tscOrderedProjectionQueryOnSTable(pQueryInfo, 0)) { - addPrimaryTsColIntoResult(pQueryInfo); + addPrimaryTsColIntoResult(pQueryInfo, pCmd); } } } @@ -5005,11 +5010,11 @@ int32_t validateOrderbyNode(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SSqlNode* pSq tVariant* pVar2 = &pItem->pVar; SStrToken cname = {pVar2->nLen, pVar2->nType, pVar2->pz}; if (getColumnIndexByName(pCmd, &cname, pQueryInfo, &index) != TSDB_CODE_SUCCESS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } if (index.columnIndex != PRIMARYKEY_TIMESTAMP_COL_INDEX) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); } else { tVariantListItem* p1 = taosArrayGet(pSortorder, 1); pQueryInfo->order.order = p1->sortOrder; @@ -5019,21 +5024,21 @@ int32_t validateOrderbyNode(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SSqlNode* pSq } else { // meter query if (getColumnIndexByName(pCmd, &columnName, pQueryInfo, &index) != TSDB_CODE_SUCCESS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } if (index.columnIndex != PRIMARYKEY_TIMESTAMP_COL_INDEX && !isTopBottomQuery(pQueryInfo)) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); } if (isTopBottomQuery(pQueryInfo)) { /* order of top/bottom query in interval is not valid */ - SExprInfo* pExpr = tscSqlExprGet(pQueryInfo, 0); + SExprInfo* pExpr = tscExprGet(pQueryInfo, 0); assert(pExpr->base.functionId == TSDB_FUNC_TS); - pExpr = tscSqlExprGet(pQueryInfo, 1); + pExpr = tscExprGet(pQueryInfo, 1); if (pExpr->base.colInfo.colIndex != index.columnIndex && index.columnIndex != PRIMARYKEY_TIMESTAMP_COL_INDEX) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); } tVariantListItem* pItem = taosArrayGet(pSqlNode->pSortOrder, 0); @@ -5079,15 +5084,15 @@ int32_t setAlterTableInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) { SSqlCmd* pCmd = &pSql->cmd; SAlterTableInfo* pAlterSQL = pInfo->pAlterInfo; - SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd, 0); + SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd); STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, DEFAULT_TABLE_INDEX); if (tscValidateName(&(pAlterSQL->name)) != TSDB_CODE_SUCCESS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } - code = tscSetTableFullName(pTableMetaInfo, &(pAlterSQL->name), pSql); + code = tscSetTableFullName(&pTableMetaInfo->name, &(pAlterSQL->name), pSql); if (code != TSDB_CODE_SUCCESS) { return code; } @@ -5100,60 +5105,60 @@ int32_t setAlterTableInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) { STableMeta* pTableMeta = pTableMetaInfo->pTableMeta; if (pAlterSQL->tableType == TSDB_SUPER_TABLE && !(UTIL_TABLE_IS_SUPER_TABLE(pTableMetaInfo))) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg20); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg20); } if (pAlterSQL->type == TSDB_ALTER_TABLE_ADD_TAG_COLUMN || pAlterSQL->type == TSDB_ALTER_TABLE_DROP_TAG_COLUMN || pAlterSQL->type == TSDB_ALTER_TABLE_CHANGE_TAG_COLUMN) { if (UTIL_TABLE_IS_NORMAL_TABLE(pTableMetaInfo)) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg3); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); } } else if ((pAlterSQL->type == TSDB_ALTER_TABLE_UPDATE_TAG_VAL) && (UTIL_TABLE_IS_SUPER_TABLE(pTableMetaInfo))) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg4); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg4); } else if ((pAlterSQL->type == TSDB_ALTER_TABLE_ADD_COLUMN || pAlterSQL->type == TSDB_ALTER_TABLE_DROP_COLUMN) && UTIL_TABLE_IS_CHILD_TABLE(pTableMetaInfo)) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg6); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg6); } if (pAlterSQL->type == TSDB_ALTER_TABLE_ADD_TAG_COLUMN) { SArray* pFieldList = pAlterSQL->pAddColumns; if (taosArrayGetSize(pFieldList) > 1) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg5); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg5); } TAOS_FIELD* p = taosArrayGet(pFieldList, 0); if (!validateOneTags(pCmd, p)) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } tscFieldInfoAppend(&pQueryInfo->fieldsInfo, p); } else if (pAlterSQL->type == TSDB_ALTER_TABLE_DROP_TAG_COLUMN) { if (tscGetNumOfTags(pTableMeta) == 1) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg7); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg7); } // numOfTags == 1 if (taosArrayGetSize(pAlterSQL->varList) > 1) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg8); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg8); } tVariantListItem* pItem = taosArrayGet(pAlterSQL->varList, 0); if (pItem->pVar.nLen >= TSDB_COL_NAME_LEN) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg9); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg9); } SColumnIndex index = COLUMN_INDEX_INITIALIZER; SStrToken name = {.z = pItem->pVar.pz, .n = pItem->pVar.nLen, .type = TK_STRING}; if (getColumnIndexByName(pCmd, &name, pQueryInfo, &index) != TSDB_CODE_SUCCESS) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } int32_t numOfCols = tscGetNumOfColumns(pTableMeta); if (index.columnIndex < numOfCols) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg10); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg10); } else if (index.columnIndex == numOfCols) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg11); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg11); } char name1[128] = {0}; @@ -5164,18 +5169,18 @@ int32_t setAlterTableInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) { } else if (pAlterSQL->type == TSDB_ALTER_TABLE_CHANGE_TAG_COLUMN) { SArray* pVarList = pAlterSQL->varList; if (taosArrayGetSize(pVarList) > 2) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } tVariantListItem* pSrcItem = taosArrayGet(pAlterSQL->varList, 0); tVariantListItem* pDstItem = taosArrayGet(pAlterSQL->varList, 1); if (pSrcItem->pVar.nLen >= TSDB_COL_NAME_LEN || pDstItem->pVar.nLen >= TSDB_COL_NAME_LEN) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg9); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg9); } if (pSrcItem->pVar.nType != TSDB_DATA_TYPE_BINARY || pDstItem->pVar.nType != TSDB_DATA_TYPE_BINARY) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg10); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg10); } SColumnIndex srcIndex = COLUMN_INDEX_INITIALIZER; @@ -5183,12 +5188,12 @@ int32_t setAlterTableInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) { SStrToken srcToken = {.z = pSrcItem->pVar.pz, .n = pSrcItem->pVar.nLen, .type = TK_STRING}; if (getColumnIndexByName(pCmd, &srcToken, pQueryInfo, &srcIndex) != TSDB_CODE_SUCCESS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg17); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg17); } SStrToken destToken = {.z = pDstItem->pVar.pz, .n = pDstItem->pVar.nLen, .type = TK_STRING}; if (getColumnIndexByName(pCmd, &destToken, pQueryInfo, &destIndex) == TSDB_CODE_SUCCESS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg19); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg19); } tVariantListItem* pItem = taosArrayGet(pVarList, 0); @@ -5216,11 +5221,11 @@ int32_t setAlterTableInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) { SColumnIndex columnIndex = COLUMN_INDEX_INITIALIZER; SStrToken name = {.type = TK_STRING, .z = item->pVar.pz, .n = item->pVar.nLen}; if (getColumnIndexByName(pCmd, &name, pQueryInfo, &columnIndex) != TSDB_CODE_SUCCESS) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } if (columnIndex.columnIndex < tscGetNumOfColumns(pTableMeta)) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg12); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg12); } tVariantListItem* pItem = taosArrayGet(pVarList, 1); @@ -5228,7 +5233,7 @@ int32_t setAlterTableInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) { pAlterSQL->tagData.data = calloc(1, pTagsSchema->bytes * TSDB_NCHAR_SIZE + VARSTR_HEADER_SIZE); if (tVariantDump(&pItem->pVar, pAlterSQL->tagData.data, pTagsSchema->type, true) != TSDB_CODE_SUCCESS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg13); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg13); } pAlterSQL->tagData.dataLen = pTagsSchema->bytes; @@ -5236,7 +5241,7 @@ int32_t setAlterTableInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) { // validate the length of binary if ((pTagsSchema->type == TSDB_DATA_TYPE_BINARY || pTagsSchema->type == TSDB_DATA_TYPE_NCHAR) && varDataTLen(pAlterSQL->tagData.data) > pTagsSchema->bytes) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg14); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg14); } int32_t schemaLen = sizeof(STColumn) * numOfTags; @@ -5291,23 +5296,23 @@ int32_t setAlterTableInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) { SArray* pFieldList = pAlterSQL->pAddColumns; if (taosArrayGetSize(pFieldList) > 1) { const char* msg = "only support add one column"; - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg); } TAOS_FIELD* p = taosArrayGet(pFieldList, 0); if (!validateOneColumn(pCmd, p)) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } tscFieldInfoAppend(&pQueryInfo->fieldsInfo, p); } else if (pAlterSQL->type == TSDB_ALTER_TABLE_DROP_COLUMN) { if (tscGetNumOfColumns(pTableMeta) == TSDB_MIN_COLUMNS) { // - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg15); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg15); } size_t size = taosArrayGetSize(pAlterSQL->varList); if (size > 1) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg16); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg16); } tVariantListItem* pItem = taosArrayGet(pAlterSQL->varList, 0); @@ -5315,11 +5320,11 @@ int32_t setAlterTableInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) { SColumnIndex columnIndex = COLUMN_INDEX_INITIALIZER; SStrToken name = {.type = TK_STRING, .z = pItem->pVar.pz, .n = pItem->pVar.nLen}; if (getColumnIndexByName(pCmd, &name, pQueryInfo, &columnIndex) != TSDB_CODE_SUCCESS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg17); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg17); } if (columnIndex.columnIndex == PRIMARYKEY_TIMESTAMP_COL_INDEX) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg18); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg18); } char name1[TSDB_COL_NAME_LEN] = {0}; @@ -5338,14 +5343,14 @@ int32_t validateSqlFunctionInStreamSql(SSqlCmd* pCmd, SQueryInfo* pQueryInfo) { if (pQueryInfo->interval.interval != 0 && pQueryInfo->interval.interval < 10 && pQueryInfo->interval.intervalUnit != 'n' && pQueryInfo->interval.intervalUnit != 'y') { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg0); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg0); } size_t size = taosArrayGetSize(pQueryInfo->exprList); for (int32_t i = 0; i < size; ++i) { - int32_t functId = tscSqlExprGet(pQueryInfo, i)->base.functionId; + int32_t functId = tscExprGet(pQueryInfo, i)->base.functionId; if (!IS_STREAM_QUERY_VALID(aAggs[functId].status)) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } } @@ -5360,13 +5365,13 @@ int32_t validateFunctionsInIntervalOrGroupbyQuery(SSqlCmd* pCmd, SQueryInfo* pQu size_t size = taosArrayGetSize(pQueryInfo->exprList); for (int32_t k = 0; k < size; ++k) { - SExprInfo* pExpr = tscSqlExprGet(pQueryInfo, k); + SExprInfo* pExpr = tscExprGet(pQueryInfo, k); // projection query on primary timestamp, the selectivity function needs to be present. if (pExpr->base.functionId == TSDB_FUNC_PRJ && pExpr->base.colInfo.colId == PRIMARYKEY_TIMESTAMP_COL_INDEX) { bool hasSelectivity = false; for (int32_t j = 0; j < size; ++j) { - SExprInfo* pEx = tscSqlExprGet(pQueryInfo, j); + SExprInfo* pEx = tscExprGet(pQueryInfo, j); if ((aAggs[pEx->base.functionId].status & TSDB_FUNCSTATE_SELECTIVITY) == TSDB_FUNCSTATE_SELECTIVITY) { hasSelectivity = true; break; @@ -5385,10 +5390,10 @@ int32_t validateFunctionsInIntervalOrGroupbyQuery(SSqlCmd* pCmd, SQueryInfo* pQu } if (isProjectionFunction) { - invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } - return isProjectionFunction == true ? TSDB_CODE_TSC_INVALID_SQL : TSDB_CODE_SUCCESS; + return isProjectionFunction == true ? TSDB_CODE_TSC_INVALID_OPERATION : TSDB_CODE_SUCCESS; } typedef struct SDNodeDynConfOption { @@ -5405,12 +5410,12 @@ int32_t validateEp(char* ep) { if (NULL == pos) { int32_t val = strtol(ep, NULL, 10); if (val <= 0 || val > 65536) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } } else { uint16_t port = atoi(pos + 1); if (0 == port) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } } @@ -5421,7 +5426,7 @@ int32_t validateDNodeConfig(SMiscInfo* pOptions) { int32_t numOfToken = (int32_t) taosArrayGetSize(pOptions->a); if (numOfToken < 2 || numOfToken > 3) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } const int tokenLogEnd = 2; @@ -5456,7 +5461,7 @@ int32_t validateDNodeConfig(SMiscInfo* pOptions) { strdequote(pValToken->z); bool parseOk = taosCheckBalanceCfgOptions(pValToken->z, &vnodeId, &dnodeId); if (!parseOk) { - return TSDB_CODE_TSC_INVALID_SQL; // options value is invalid + return TSDB_CODE_TSC_INVALID_OPERATION; // options value is invalid } return TSDB_CODE_SUCCESS; } else if ((strncasecmp(cfgOptions[tokenMonitor].name, pOptionToken->z, pOptionToken->n) == 0) && @@ -5464,7 +5469,7 @@ int32_t validateDNodeConfig(SMiscInfo* pOptions) { SStrToken* pValToken = taosArrayGet(pOptions->a, 2); int32_t val = strtol(pValToken->z, NULL, 10); if (val != 0 && val != 1) { - return TSDB_CODE_TSC_INVALID_SQL; // options value is invalid + return TSDB_CODE_TSC_INVALID_OPERATION; // options value is invalid } return TSDB_CODE_SUCCESS; } else { @@ -5473,7 +5478,7 @@ int32_t validateDNodeConfig(SMiscInfo* pOptions) { int32_t val = strtol(pValToken->z, NULL, 10); if (val < 0 || val > 256) { /* options value is out of valid range */ - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } for (int32_t i = tokenDebugFlag; i < tokenDebugFlagEnd; ++i) { @@ -5486,13 +5491,13 @@ int32_t validateDNodeConfig(SMiscInfo* pOptions) { } } - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } int32_t validateLocalConfig(SMiscInfo* pOptions) { int32_t numOfToken = (int32_t) taosArrayGetSize(pOptions->a); if (numOfToken < 1 || numOfToken > 2) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } SDNodeDynConfOption LOCAL_DYNAMIC_CFG_OPTIONS[6] = {{"resetLog", 8}, {"rpcDebugFlag", 12}, {"tmrDebugFlag", 12}, @@ -5515,7 +5520,7 @@ int32_t validateLocalConfig(SMiscInfo* pOptions) { int32_t val = strtol(pValToken->z, NULL, 10); if (!validateDebugFlag(val)) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } for (int32_t i = 1; i < tListLen(LOCAL_DYNAMIC_CFG_OPTIONS); ++i) { @@ -5526,20 +5531,20 @@ int32_t validateLocalConfig(SMiscInfo* pOptions) { } } } - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } int32_t validateColumnName(char* name) { bool ret = taosIsKeyWordToken(name, (int32_t)strlen(name)); if (ret) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } SStrToken token = {.z = name}; token.n = tGetToken(name, &token.type); if (token.type != TK_STRING && token.type != TK_ID) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } if (token.type == TK_STRING) { @@ -5549,13 +5554,13 @@ int32_t validateColumnName(char* name) { int32_t k = tGetToken(token.z, &token.type); if (k != token.n) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } return validateColumnName(token.z); } else { if (isNumber(&token)) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } } @@ -5574,7 +5579,7 @@ bool hasTimestampForPointInterpQuery(SQueryInfo* pQueryInfo) { return !(pQueryInfo->window.skey != pQueryInfo->window.ekey && pQueryInfo->interval.interval == 0); } -int32_t validateLimitNode(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, int32_t clauseIndex, SSqlNode* pSqlNode, SSqlObj* pSql) { +int32_t validateLimitNode(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SSqlNode* pSqlNode, SSqlObj* pSql) { STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0); const char* msg0 = "soffset/offset can not be less than 0"; @@ -5590,7 +5595,7 @@ int32_t validateLimitNode(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, int32_t clauseI pQueryInfo->limit.limit, pQueryInfo->limit.offset, pQueryInfo->slimit.limit, pQueryInfo->slimit.offset); if (pQueryInfo->slimit.offset < 0 || pQueryInfo->limit.offset < 0) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg0); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg0); } if (pQueryInfo->limit.limit == 0) { @@ -5604,7 +5609,7 @@ int32_t validateLimitNode(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, int32_t clauseI if (!tscQueryTags(pQueryInfo)) { // local handle the super table tag query if (tscIsProjectionQueryOnSTable(pQueryInfo, 0)) { if (pQueryInfo->slimit.limit > 0 || pQueryInfo->slimit.offset > 0) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); } // for projection query on super table, all queries are subqueries @@ -5627,10 +5632,7 @@ int32_t validateLimitNode(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, int32_t clauseI * And then launching multiple async-queries against all qualified virtual nodes, during the first-stage * query operation. */ - int32_t code = tscGetSTableVgroupInfo(pSql, clauseIndex); - if (code != TSDB_CODE_SUCCESS) { - return code; - } +// assert(allVgroupInfoRetrieved(pQueryInfo)); // No tables included. No results generated. Query results are empty. if (pTableMetaInfo->vgroupList->numOfVgroups == 0) { @@ -5660,7 +5662,7 @@ int32_t validateLimitNode(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, int32_t clauseI } } else { if (pQueryInfo->slimit.limit != -1 || pQueryInfo->slimit.offset != 0) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } } @@ -5698,7 +5700,7 @@ static int32_t setKeepOption(SSqlCmd* pCmd, SCreateDbMsg* pMsg, SCreateDbInfo* p pMsg->daysToKeep2 = htonl((int32_t)p2->pVar.i64); break; } - default: { return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg); } + default: { return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg); } } } @@ -5722,7 +5724,7 @@ static int32_t setTimePrecision(SSqlCmd* pCmd, SCreateDbMsg* pMsg, SCreateDbInfo strlen(TSDB_TIME_PRECISION_MICRO_STR) == pToken->n) { pMsg->precision = TSDB_TIME_PRECISION_MICRO; } else { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg); } } @@ -5754,30 +5756,30 @@ int32_t parseCreateDBOptions(SSqlCmd* pCmd, SCreateDbInfo* pCreateDbSql) { setCreateDBOption(pMsg, pCreateDbSql); if (setKeepOption(pCmd, pMsg, pCreateDbSql) != TSDB_CODE_SUCCESS) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } if (setTimePrecision(pCmd, pMsg, pCreateDbSql) != TSDB_CODE_SUCCESS) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } if (tscCheckCreateDbParams(pCmd, pMsg) != TSDB_CODE_SUCCESS) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } return TSDB_CODE_SUCCESS; } void addGroupInfoForSubquery(SSqlObj* pParentObj, SSqlObj* pSql, int32_t subClauseIndex, int32_t tableIndex) { - SQueryInfo* pParentQueryInfo = tscGetQueryInfo(&pParentObj->cmd, subClauseIndex); + SQueryInfo* pParentQueryInfo = tscGetQueryInfo(&pParentObj->cmd); if (pParentQueryInfo->groupbyExpr.numOfGroupCols > 0) { - SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd, subClauseIndex); + SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd); SExprInfo* pExpr = NULL; size_t size = taosArrayGetSize(pQueryInfo->exprList); if (size > 0) { - pExpr = tscSqlExprGet(pQueryInfo, (int32_t)size - 1); + pExpr = tscExprGet(pQueryInfo, (int32_t)size - 1); } if (pExpr == NULL || pExpr->base.functionId != TSDB_FUNC_TAG) { @@ -5794,7 +5796,7 @@ void addGroupInfoForSubquery(SSqlObj* pParentObj, SSqlObj* pSql, int32_t subClau int16_t type = pTagSchema->type; int16_t bytes = pTagSchema->bytes; - pExpr = tscSqlExprAppend(pQueryInfo, TSDB_FUNC_TAG, &index, type, bytes, getNewResColId(pQueryInfo), bytes, true); + pExpr = tscExprAppend(pQueryInfo, TSDB_FUNC_TAG, &index, type, bytes, getNewResColId(&pSql->cmd), bytes, true); pExpr->base.colInfo.flag = TSDB_COL_TAG; // NOTE: tag column does not add to source column list @@ -5819,16 +5821,19 @@ static void doLimitOutputNormalColOfGroupby(SExprInfo* pExpr) { pExpr->base.numOfParams = 1; } -void doAddGroupColumnForSubquery(SQueryInfo* pQueryInfo, int32_t tagIndex) { +void doAddGroupColumnForSubquery(SQueryInfo* pQueryInfo, int32_t tagIndex, SSqlCmd* pCmd) { SColIndex* pColIndex = taosArrayGet(pQueryInfo->groupbyExpr.columnInfo, tagIndex); - size_t size = tscSqlExprNumOfExprs(pQueryInfo); + size_t size = tscNumOfExprs(pQueryInfo); STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0); SSchema* pSchema = tscGetTableColumnSchema(pTableMetaInfo->pTableMeta, pColIndex->colIndex); SColumnIndex colIndex = {.tableIndex = 0, .columnIndex = pColIndex->colIndex}; - tscAddFuncInSelectClause(pQueryInfo, (int32_t)size, TSDB_FUNC_PRJ, &colIndex, pSchema, TSDB_COL_NORMAL); + SExprInfo* pExprInfo = tscAddFuncInSelectClause(pQueryInfo, (int32_t)size, TSDB_FUNC_PRJ, &colIndex, pSchema, + TSDB_COL_NORMAL, getNewResColId(pCmd)); + + strncpy(pExprInfo->base.token, pExprInfo->base.colInfo.name, tListLen(pExprInfo->base.token)); int32_t numOfFields = tscNumOfFields(pQueryInfo); SInternalField* pInfo = tscFieldInfoGetInternalField(&pQueryInfo->fieldsInfo, numOfFields - 1); @@ -5846,7 +5851,7 @@ static void doUpdateSqlFunctionForTagPrj(SQueryInfo* pQueryInfo) { bool isSTable = UTIL_TABLE_IS_SUPER_TABLE(pTableMetaInfo); for (int32_t i = 0; i < size; ++i) { - SExprInfo* pExpr = tscSqlExprGet(pQueryInfo, i); + SExprInfo* pExpr = tscExprGet(pQueryInfo, i); if (pExpr->base.functionId == TSDB_FUNC_TAGPRJ || pExpr->base.functionId == TSDB_FUNC_TAG) { pExpr->base.functionId = TSDB_FUNC_TAG_DUMMY; tagLength += pExpr->base.resBytes; @@ -5859,7 +5864,7 @@ static void doUpdateSqlFunctionForTagPrj(SQueryInfo* pQueryInfo) { SSchema* pSchema = tscGetTableSchema(pTableMetaInfo->pTableMeta); for (int32_t i = 0; i < size; ++i) { - SExprInfo* pExpr = tscSqlExprGet(pQueryInfo, i); + SExprInfo* pExpr = tscExprGet(pQueryInfo, i); if ((pExpr->base.functionId != TSDB_FUNC_TAG_DUMMY && pExpr->base.functionId != TSDB_FUNC_TS_DUMMY) && !(pExpr->base.functionId == TSDB_FUNC_PRJ && TSDB_COL_IS_UD_COL(pExpr->base.colInfo.flag))) { SSchema* pColSchema = &pSchema[pExpr->base.colInfo.colIndex]; @@ -5873,7 +5878,7 @@ static int32_t doUpdateSqlFunctionForColPrj(SQueryInfo* pQueryInfo) { size_t size = taosArrayGetSize(pQueryInfo->exprList); for (int32_t i = 0; i < size; ++i) { - SExprInfo* pExpr = tscSqlExprGet(pQueryInfo, i); + SExprInfo* pExpr = tscExprGet(pQueryInfo, i); if (pExpr->base.functionId == TSDB_FUNC_PRJ && (!TSDB_COL_IS_UD_COL(pExpr->base.colInfo.flag) && (pExpr->base.colInfo.colId != PRIMARYKEY_TIMESTAMP_COL_INDEX))) { bool qualifiedCol = false; @@ -5890,7 +5895,7 @@ static int32_t doUpdateSqlFunctionForColPrj(SQueryInfo* pQueryInfo) { // it is not a tag column/tbname column/user-defined column, return error if (!qualifiedCol) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } } } @@ -5898,7 +5903,7 @@ static int32_t doUpdateSqlFunctionForColPrj(SQueryInfo* pQueryInfo) { return TSDB_CODE_SUCCESS; } -static bool tagColumnInGroupby(SSqlGroupbyExpr* pGroupbyExpr, int16_t columnId) { +static bool tagColumnInGroupby(SGroupbyExpr* pGroupbyExpr, int16_t columnId) { for (int32_t j = 0; j < pGroupbyExpr->numOfGroupCols; ++j) { SColIndex* pColIndex = taosArrayGet(pGroupbyExpr->columnInfo, j); @@ -5916,7 +5921,7 @@ static bool onlyTagPrjFunction(SQueryInfo* pQueryInfo) { size_t size = taosArrayGetSize(pQueryInfo->exprList); for (int32_t i = 0; i < size; ++i) { - SExprInfo* pExpr = tscSqlExprGet(pQueryInfo, i); + SExprInfo* pExpr = tscExprGet(pQueryInfo, i); if (pExpr->base.functionId == TSDB_FUNC_PRJ) { hasColumnPrj = true; } else if (pExpr->base.functionId == TSDB_FUNC_TAGPRJ) { @@ -5931,9 +5936,9 @@ static bool onlyTagPrjFunction(SQueryInfo* pQueryInfo) { static bool allTagPrjInGroupby(SQueryInfo* pQueryInfo) { bool allInGroupby = true; - size_t size = tscSqlExprNumOfExprs(pQueryInfo); + size_t size = tscNumOfExprs(pQueryInfo); for (int32_t i = 0; i < size; ++i) { - SExprInfo* pExpr = tscSqlExprGet(pQueryInfo, i); + SExprInfo* pExpr = tscExprGet(pQueryInfo, i); if (pExpr->base.functionId != TSDB_FUNC_TAGPRJ) { continue; } @@ -5952,7 +5957,7 @@ static void updateTagPrjFunction(SQueryInfo* pQueryInfo) { size_t size = taosArrayGetSize(pQueryInfo->exprList); for (int32_t i = 0; i < size; ++i) { - SExprInfo* pExpr = tscSqlExprGet(pQueryInfo, i); + SExprInfo* pExpr = tscExprGet(pQueryInfo, i); if (pExpr->base.functionId == TSDB_FUNC_TAGPRJ) { pExpr->base.functionId = TSDB_FUNC_TAG; } @@ -6003,7 +6008,7 @@ static int32_t checkUpdateTagPrjFunctions(SQueryInfo* pQueryInfo, SSqlCmd* pCmd) // When the tag projection function on tag column that is not in the group by clause, aggregation function and // selectivity function exist in select clause is not allowed. if (numOfAggregation > 0) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } /* @@ -6022,7 +6027,7 @@ static int32_t checkUpdateTagPrjFunctions(SQueryInfo* pQueryInfo, SSqlCmd* pCmd) * Otherwise, return with error code. */ for (int32_t i = 0; i < numOfExprs; ++i) { - SExprInfo* pExpr = tscSqlExprGet(pQueryInfo, i); + SExprInfo* pExpr = tscExprGet(pQueryInfo, i); int16_t functionId = pExpr->base.functionId; if (functionId == TSDB_FUNC_TAGPRJ || (aAggs[functionId].status & TSDB_FUNCSTATE_SELECTIVITY) == 0) { continue; @@ -6032,7 +6037,7 @@ static int32_t checkUpdateTagPrjFunctions(SQueryInfo* pQueryInfo, SSqlCmd* pCmd) (functionId == TSDB_FUNC_LAST_DST && (pExpr->base.colInfo.flag & TSDB_COL_NULL) != 0)) { // do nothing } else { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } } @@ -6045,7 +6050,7 @@ static int32_t checkUpdateTagPrjFunctions(SQueryInfo* pQueryInfo, SSqlCmd* pCmd) } else { if ((pQueryInfo->type & TSDB_QUERY_TYPE_PROJECTION_QUERY) != 0) { if (numOfAggregation > 0 && pQueryInfo->groupbyExpr.numOfGroupCols == 0) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg3); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); } if (numOfAggregation > 0 || numOfSelectivity > 0) { @@ -6090,15 +6095,16 @@ static int32_t doAddGroupbyColumnsOnDemand(SSqlCmd* pCmd, SQueryInfo* pQueryInfo } } - size_t size = tscSqlExprNumOfExprs(pQueryInfo); + size_t size = tscNumOfExprs(pQueryInfo); if (TSDB_COL_IS_TAG(pColIndex->flag)) { SColumnIndex index = {.tableIndex = pQueryInfo->groupbyExpr.tableIndex, .columnIndex = colIndex}; - SExprInfo* pExpr = tscSqlExprAppend(pQueryInfo, TSDB_FUNC_TAG, &index, s->type, s->bytes, - getNewResColId(pQueryInfo), s->bytes, true); + SExprInfo* pExpr = tscExprAppend(pQueryInfo, TSDB_FUNC_TAG, &index, s->type, s->bytes, + getNewResColId(pCmd), s->bytes, true); memset(pExpr->base.aliasName, 0, sizeof(pExpr->base.aliasName)); tstrncpy(pExpr->base.aliasName, s->name, sizeof(pExpr->base.aliasName)); + tstrncpy(pExpr->base.token, s->name, sizeof(pExpr->base.aliasName)); pExpr->base.colInfo.flag = TSDB_COL_TAG; @@ -6108,12 +6114,12 @@ static int32_t doAddGroupbyColumnsOnDemand(SSqlCmd* pCmd, SQueryInfo* pQueryInfo } else { // if this query is "group by" normal column, time window query is not allowed if (isTimeWindowQuery(pQueryInfo)) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } bool hasGroupColumn = false; for (int32_t j = 0; j < size; ++j) { - SExprInfo* pExpr = tscSqlExprGet(pQueryInfo, j); + SExprInfo* pExpr = tscExprGet(pQueryInfo, j); if ((pExpr->base.functionId == TSDB_FUNC_PRJ) && pExpr->base.colInfo.colId == pColIndex->colId) { hasGroupColumn = true; break; @@ -6122,7 +6128,7 @@ static int32_t doAddGroupbyColumnsOnDemand(SSqlCmd* pCmd, SQueryInfo* pQueryInfo //if the group by column does not required by user, add an invisible column into the final result set. if (!hasGroupColumn) { - doAddGroupColumnForSubquery(pQueryInfo, i); + doAddGroupColumnForSubquery(pQueryInfo, i, pCmd); } } } @@ -6134,10 +6140,10 @@ static int32_t doTagFunctionCheck(SQueryInfo* pQueryInfo) { bool tagProjection = false; bool tableCounting = false; - int32_t numOfCols = (int32_t) tscSqlExprNumOfExprs(pQueryInfo); + int32_t numOfCols = (int32_t) tscNumOfExprs(pQueryInfo); for (int32_t i = 0; i < numOfCols; ++i) { - SExprInfo* pExpr = tscSqlExprGet(pQueryInfo, i); + SExprInfo* pExpr = tscExprGet(pQueryInfo, i); int32_t functionId = pExpr->base.functionId; if (functionId == TSDB_FUNC_TAGPRJ) { @@ -6164,11 +6170,11 @@ int32_t doFunctionsCompatibleCheck(SSqlCmd* pCmd, SQueryInfo* pQueryInfo) { // only retrieve tags, group by is not supportted if (tscQueryTags(pQueryInfo)) { if (doTagFunctionCheck(pQueryInfo) != TSDB_CODE_SUCCESS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg5); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg5); } if (pQueryInfo->groupbyExpr.numOfGroupCols > 0 || isTimeWindowQuery(pQueryInfo)) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg4); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg4); } else { return TSDB_CODE_SUCCESS; } @@ -6183,9 +6189,9 @@ int32_t doFunctionsCompatibleCheck(SSqlCmd* pCmd, SQueryInfo* pQueryInfo) { } // check all query functions in selection clause, multi-output functions are not allowed - size_t size = tscSqlExprNumOfExprs(pQueryInfo); + size_t size = tscNumOfExprs(pQueryInfo); for (int32_t i = 0; i < size; ++i) { - SExprInfo* pExpr = tscSqlExprGet(pQueryInfo, i); + SExprInfo* pExpr = tscExprGet(pQueryInfo, i); int32_t functId = pExpr->base.functionId; /* @@ -6203,31 +6209,31 @@ int32_t doFunctionsCompatibleCheck(SSqlCmd* pCmd, SQueryInfo* pQueryInfo) { } if (!qualified) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); } } if (IS_MULTIOUTPUT(aAggs[functId].status) && functId != TSDB_FUNC_TOP && functId != TSDB_FUNC_BOTTOM && functId != TSDB_FUNC_TAGPRJ && functId != TSDB_FUNC_PRJ) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } if (functId == TSDB_FUNC_COUNT && pExpr->base.colInfo.colIndex == TSDB_TBNAME_COLUMN_INDEX) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } } if (checkUpdateTagPrjFunctions(pQueryInfo, pCmd) != TSDB_CODE_SUCCESS) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } if (doAddGroupbyColumnsOnDemand(pCmd, pQueryInfo) != TSDB_CODE_SUCCESS) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } // projection query on super table does not compatible with "group by" syntax if (tscIsProjectionQuery(pQueryInfo)) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg3); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); } return TSDB_CODE_SUCCESS; @@ -6243,7 +6249,7 @@ int32_t doLocalQueryProcess(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SSqlNode* pSq SArray* pExprList = pSqlNode->pSelNodeList; size_t size = taosArrayGetSize(pExprList); if (size != 1) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } bool server_status = false; @@ -6254,7 +6260,7 @@ int32_t doLocalQueryProcess(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SSqlNode* pSq if (pExpr->token.n == 1 && 0 == strncasecmp(pExpr->token.z, "1", 1)) { server_status = true; } else { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); } } // TODO redefine the function @@ -6288,12 +6294,12 @@ int32_t doLocalQueryProcess(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SSqlNode* pSq pQueryInfo->command = TSDB_SQL_CLI_VERSION;break; case 4: pQueryInfo->command = TSDB_SQL_CURRENT_USER;break; - default: { return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg3); } + default: { return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); } } SColumnIndex ind = {0}; - SExprInfo* pExpr1 = tscSqlExprAppend(pQueryInfo, TSDB_FUNC_TAG_DUMMY, &ind, TSDB_DATA_TYPE_INT, - tDataTypes[TSDB_DATA_TYPE_INT].bytes, getNewResColId(pQueryInfo), tDataTypes[TSDB_DATA_TYPE_INT].bytes, false); + SExprInfo* pExpr1 = tscExprAppend(pQueryInfo, TSDB_FUNC_TAG_DUMMY, &ind, TSDB_DATA_TYPE_INT, + tDataTypes[TSDB_DATA_TYPE_INT].bytes, getNewResColId(pCmd), tDataTypes[TSDB_DATA_TYPE_INT].bytes, false); tSqlExprItem* item = taosArrayGet(pExprList, 0); const char* name = (item->aliasName != NULL)? item->aliasName:functionsInfo[index].name; @@ -6308,69 +6314,69 @@ int32_t tscCheckCreateDbParams(SSqlCmd* pCmd, SCreateDbMsg* pCreate) { if (pCreate->walLevel != -1 && (pCreate->walLevel < TSDB_MIN_WAL_LEVEL || pCreate->walLevel > TSDB_MAX_WAL_LEVEL)) { snprintf(msg, tListLen(msg), "invalid db option walLevel: %d, only 1-2 allowed", pCreate->walLevel); - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg); } if (pCreate->replications != -1 && (pCreate->replications < TSDB_MIN_DB_REPLICA_OPTION || pCreate->replications > TSDB_MAX_DB_REPLICA_OPTION)) { snprintf(msg, tListLen(msg), "invalid db option replications: %d valid range: [%d, %d]", pCreate->replications, TSDB_MIN_DB_REPLICA_OPTION, TSDB_MAX_DB_REPLICA_OPTION); - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg); } if (pCreate->quorum != -1 && (pCreate->quorum < TSDB_MIN_DB_QUORUM_OPTION || pCreate->quorum > TSDB_MAX_DB_QUORUM_OPTION)) { snprintf(msg, tListLen(msg), "invalid db option quorum: %d valid range: [%d, %d]", pCreate->quorum, TSDB_MIN_DB_QUORUM_OPTION, TSDB_MAX_DB_QUORUM_OPTION); - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg); } int32_t val = htonl(pCreate->daysPerFile); if (val != -1 && (val < TSDB_MIN_DAYS_PER_FILE || val > TSDB_MAX_DAYS_PER_FILE)) { snprintf(msg, tListLen(msg), "invalid db option daysPerFile: %d valid range: [%d, %d]", val, TSDB_MIN_DAYS_PER_FILE, TSDB_MAX_DAYS_PER_FILE); - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg); } val = htonl(pCreate->cacheBlockSize); if (val != -1 && (val < TSDB_MIN_CACHE_BLOCK_SIZE || val > TSDB_MAX_CACHE_BLOCK_SIZE)) { snprintf(msg, tListLen(msg), "invalid db option cacheBlockSize: %d valid range: [%d, %d]", val, TSDB_MIN_CACHE_BLOCK_SIZE, TSDB_MAX_CACHE_BLOCK_SIZE); - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg); } val = htonl(pCreate->maxTables); if (val != -1 && (val < TSDB_MIN_TABLES || val > TSDB_MAX_TABLES)) { snprintf(msg, tListLen(msg), "invalid db option maxSessions: %d valid range: [%d, %d]", val, TSDB_MIN_TABLES, TSDB_MAX_TABLES); - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg); } if (pCreate->precision != TSDB_TIME_PRECISION_MILLI && pCreate->precision != TSDB_TIME_PRECISION_MICRO) { snprintf(msg, tListLen(msg), "invalid db option timePrecision: %d valid value: [%d, %d]", pCreate->precision, TSDB_TIME_PRECISION_MILLI, TSDB_TIME_PRECISION_MICRO); - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg); } val = htonl(pCreate->commitTime); if (val != -1 && (val < TSDB_MIN_COMMIT_TIME || val > TSDB_MAX_COMMIT_TIME)) { snprintf(msg, tListLen(msg), "invalid db option commitTime: %d valid range: [%d, %d]", val, TSDB_MIN_COMMIT_TIME, TSDB_MAX_COMMIT_TIME); - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg); } val = htonl(pCreate->fsyncPeriod); if (val != -1 && (val < TSDB_MIN_FSYNC_PERIOD || val > TSDB_MAX_FSYNC_PERIOD)) { snprintf(msg, tListLen(msg), "invalid db option fsyncPeriod: %d valid range: [%d, %d]", val, TSDB_MIN_FSYNC_PERIOD, TSDB_MAX_FSYNC_PERIOD); - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg); } if (pCreate->compression != -1 && (pCreate->compression < TSDB_MIN_COMP_LEVEL || pCreate->compression > TSDB_MAX_COMP_LEVEL)) { snprintf(msg, tListLen(msg), "invalid db option compression: %d valid range: [%d, %d]", pCreate->compression, TSDB_MIN_COMP_LEVEL, TSDB_MAX_COMP_LEVEL); - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg); } val = (int16_t)htons(pCreate->partitions); @@ -6378,7 +6384,7 @@ int32_t tscCheckCreateDbParams(SSqlCmd* pCmd, SCreateDbMsg* pCreate) { (val < TSDB_MIN_DB_PARTITON_OPTION || val > TSDB_MAX_DB_PARTITON_OPTION)) { snprintf(msg, tListLen(msg), "invalid topic option partition: %d valid range: [%d, %d]", val, TSDB_MIN_DB_PARTITON_OPTION, TSDB_MAX_DB_PARTITON_OPTION); - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg); } @@ -6387,9 +6393,9 @@ int32_t tscCheckCreateDbParams(SSqlCmd* pCmd, SCreateDbMsg* pCreate) { // for debug purpose void tscPrintSelNodeList(SSqlObj* pSql, int32_t subClauseIndex) { - SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd, subClauseIndex); + SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd); - int32_t size = (int32_t)tscSqlExprNumOfExprs(pQueryInfo); + int32_t size = (int32_t)tscNumOfExprs(pQueryInfo); if (size == 0) { return; } @@ -6401,7 +6407,7 @@ void tscPrintSelNodeList(SSqlObj* pSql, int32_t subClauseIndex) { offset += sprintf(str, "num:%d [", size); for (int32_t i = 0; i < size; ++i) { - SExprInfo* pExpr = tscSqlExprGet(pQueryInfo, i); + SExprInfo* pExpr = tscExprGet(pQueryInfo, i); char tmpBuf[1024] = {0}; int32_t tmpLen = 0; @@ -6429,7 +6435,7 @@ int32_t doCheckForCreateTable(SSqlObj* pSql, int32_t subClauseIndex, SSqlInfo* p const char* msg1 = "invalid table name"; SSqlCmd* pCmd = &pSql->cmd; - SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd, subClauseIndex); + SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd); STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0); SCreateTableSql* pCreateTable = pInfo->pCreateTableInfo; @@ -6443,17 +6449,17 @@ int32_t doCheckForCreateTable(SSqlObj* pSql, int32_t subClauseIndex, SSqlInfo* p SStrToken* pzTableName = &(pCreateTable->name); if (tscValidateName(pzTableName) != TSDB_CODE_SUCCESS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } - int32_t code = tscSetTableFullName(pTableMetaInfo, pzTableName, pSql); + int32_t code = tscSetTableFullName(&pTableMetaInfo->name, pzTableName, pSql); if(code != TSDB_CODE_SUCCESS) { return code; } if (!validateTableColumnInfo(pFieldList, pCmd) || (pTagList != NULL && !validateTagParams(pTagList, pFieldList, pCmd))) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } int32_t col = 0; @@ -6488,7 +6494,7 @@ int32_t doCheckForCreateFromStable(SSqlObj* pSql, SSqlInfo* pInfo) { SSqlCmd* pCmd = &pSql->cmd; SCreateTableSql* pCreateTable = pInfo->pCreateTableInfo; - SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd, 0); + SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd); // two table: the first one is for current table, and the secondary is for the super table. if (pQueryInfo->numOfTables < 2) { @@ -6507,10 +6513,10 @@ int32_t doCheckForCreateFromStable(SSqlObj* pSql, SSqlInfo* pInfo) { SStrToken* pToken = &pCreateTableInfo->stableName; if (tscValidateName(pToken) != TSDB_CODE_SUCCESS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } - int32_t code = tscSetTableFullName(pStableMetaInfo, pToken, pSql); + int32_t code = tscSetTableFullName(&pStableMetaInfo->name, pToken, pSql); if (code != TSDB_CODE_SUCCESS) { return code; } @@ -6546,12 +6552,12 @@ int32_t doCheckForCreateFromStable(SSqlObj* pSql, SSqlInfo* pInfo) { if (valSize != nameSize) { tdDestroyKVRowBuilder(&kvRowBuilder); - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg5); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg5); } if (schemaSize < valSize) { tdDestroyKVRowBuilder(&kvRowBuilder); - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg5); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg5); } bool findColumnIndex = false; @@ -6575,7 +6581,7 @@ int32_t doCheckForCreateFromStable(SSqlObj* pSql, SSqlInfo* pInfo) { if (pSchema->type == TSDB_DATA_TYPE_BINARY || pSchema->type == TSDB_DATA_TYPE_NCHAR) { if (pItem->pVar.nLen > pSchema->bytes) { tdDestroyKVRowBuilder(&kvRowBuilder); - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg3); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); } } @@ -6586,13 +6592,13 @@ int32_t doCheckForCreateFromStable(SSqlObj* pSql, SSqlInfo* pInfo) { int16_t len = varDataTLen(tagVal); if (len > pSchema->bytes) { tdDestroyKVRowBuilder(&kvRowBuilder); - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg3); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); } } if (ret != TSDB_CODE_SUCCESS) { tdDestroyKVRowBuilder(&kvRowBuilder); - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg4); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg4); } tdAddColToKVRow(&kvRowBuilder, pSchema->colId, pSchema->type, tagVal); @@ -6610,7 +6616,7 @@ int32_t doCheckForCreateFromStable(SSqlObj* pSql, SSqlInfo* pInfo) { } else { if (schemaSize != valSize) { tdDestroyKVRowBuilder(&kvRowBuilder); - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg5); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg5); } for (int32_t i = 0; i < valSize; ++i) { @@ -6621,7 +6627,7 @@ int32_t doCheckForCreateFromStable(SSqlObj* pSql, SSqlInfo* pInfo) { if (pSchema->type == TSDB_DATA_TYPE_BINARY || pSchema->type == TSDB_DATA_TYPE_NCHAR) { if (pItem->pVar.nLen > pSchema->bytes) { tdDestroyKVRowBuilder(&kvRowBuilder); - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg3); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); } } @@ -6632,13 +6638,13 @@ int32_t doCheckForCreateFromStable(SSqlObj* pSql, SSqlInfo* pInfo) { int16_t len = varDataTLen(tagVal); if (len > pSchema->bytes) { tdDestroyKVRowBuilder(&kvRowBuilder); - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg3); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); } } if (ret != TSDB_CODE_SUCCESS) { tdDestroyKVRowBuilder(&kvRowBuilder); - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg4); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg4); } tdAddColToKVRow(&kvRowBuilder, pSchema->colId, pSchema->type, tagVal); @@ -6662,11 +6668,11 @@ int32_t doCheckForCreateFromStable(SSqlObj* pSql, SSqlInfo* pInfo) { // table name if (tscValidateName(&(pCreateTableInfo->name)) != TSDB_CODE_SUCCESS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, TABLE_INDEX); - ret = tscSetTableFullName(pTableMetaInfo, &pCreateTableInfo->name, pSql); + ret = tscSetTableFullName(&pTableMetaInfo->name, &pCreateTableInfo->name, pSql); if (ret != TSDB_CODE_SUCCESS) { return ret; } @@ -6674,7 +6680,7 @@ int32_t doCheckForCreateFromStable(SSqlObj* pSql, SSqlInfo* pInfo) { pCreateTableInfo->fullname = calloc(1, tNameLen(&pTableMetaInfo->name) + 1); ret = tNameExtractFullName(&pTableMetaInfo->name, pCreateTableInfo->fullname); if (ret != TSDB_CODE_SUCCESS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } } @@ -6691,7 +6697,7 @@ int32_t doCheckForStream(SSqlObj* pSql, SSqlInfo* pInfo) { const char* msg7 = "time interval is required"; SSqlCmd* pCmd = &pSql->cmd; - SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd, 0); + SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd); assert(pQueryInfo->numOfTables == 1); SCreateTableSql* pCreateTable = pInfo->pCreateTableInfo; @@ -6702,21 +6708,21 @@ int32_t doCheckForStream(SSqlObj* pSql, SSqlInfo* pInfo) { SSqlNode* pSqlNode = pCreateTable->pSelect; if (tscValidateName(pName) != TSDB_CODE_SUCCESS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } SRelationInfo* pFromInfo = pInfo->pCreateTableInfo->pSelect->from; if (pFromInfo == NULL || taosArrayGetSize(pFromInfo->list) == 0) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg6); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg6); } - STableNamePair* p1 = taosArrayGet(pFromInfo->list, 0); - SStrToken srcToken = {.z = p1->name.z, .n = p1->name.n, .type = TK_STRING}; + SRelElementPair* p1 = taosArrayGet(pFromInfo->list, 0); + SStrToken srcToken = {.z = p1->tableName.z, .n = p1->tableName.n, .type = TK_STRING}; if (tscValidateName(&srcToken) != TSDB_CODE_SUCCESS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } - int32_t code = tscSetTableFullName(pTableMetaInfo, &srcToken, pSql); + int32_t code = tscSetTableFullName(&pTableMetaInfo->name, &srcToken, pSql); if (code != TSDB_CODE_SUCCESS) { return code; } @@ -6728,46 +6734,46 @@ int32_t doCheckForStream(SSqlObj* pSql, SSqlInfo* pInfo) { bool isSTable = UTIL_TABLE_IS_SUPER_TABLE(pTableMetaInfo); if (validateSelectNodeList(&pSql->cmd, pQueryInfo, pSqlNode->pSelNodeList, isSTable, false, false) != TSDB_CODE_SUCCESS) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } if (pSqlNode->pWhere != NULL) { // query condition in stream computing if (validateWhereNode(pQueryInfo, &pSqlNode->pWhere, pSql) != TSDB_CODE_SUCCESS) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } } // set interval value if (validateIntervalNode(pSql, pQueryInfo, pSqlNode) != TSDB_CODE_SUCCESS) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } if (isTimeWindowQuery(pQueryInfo) && (validateFunctionsInIntervalOrGroupbyQuery(pCmd, pQueryInfo) != TSDB_CODE_SUCCESS)) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); } if (!tscIsProjectionQuery(pQueryInfo) && pQueryInfo->interval.interval == 0) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg7); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg7); } // set the created table[stream] name - code = tscSetTableFullName(pTableMetaInfo, pName, pSql); + code = tscSetTableFullName(&pTableMetaInfo->name, pName, pSql); if (code != TSDB_CODE_SUCCESS) { return code; } if (pSqlNode->sqlstr.n > TSDB_MAX_SAVED_SQL_LEN) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg5); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg5); } if (tsRewriteFieldNameIfNecessary(pCmd, pQueryInfo) != TSDB_CODE_SUCCESS) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } pCmd->numOfCols = pQueryInfo->fieldsInfo.numOfOutput; if (validateSqlFunctionInStreamSql(pCmd, pQueryInfo) != TSDB_CODE_SUCCESS) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } /* @@ -6776,14 +6782,14 @@ int32_t doCheckForStream(SSqlObj* pSql, SSqlInfo* pInfo) { */ if (pSqlNode->fillType != NULL) { if (pQueryInfo->interval.interval == 0) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg3); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); } tVariantListItem* pItem = taosArrayGet(pSqlNode->fillType, 0); if (pItem->pVar.nType == TSDB_DATA_TYPE_BINARY) { if (!((strncmp(pItem->pVar.pz, "none", 4) == 0 && pItem->pVar.nLen == 4) || (strncmp(pItem->pVar.pz, "null", 4) == 0 && pItem->pVar.nLen == 4))) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg4); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg4); } } } @@ -6802,7 +6808,7 @@ int32_t checkQueryRangeForFill(SSqlCmd* pCmd, SQueryInfo* pQueryInfo) { bool initialWindows = TSWINDOW_IS_EQUAL(pQueryInfo->window, TSWINDOW_INITIALIZER); if (initialWindows) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg3); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); } int64_t timeRange = ABS(pQueryInfo->window.skey - pQueryInfo->window.ekey); @@ -6822,7 +6828,7 @@ int32_t checkQueryRangeForFill(SSqlCmd* pCmd, SQueryInfo* pQueryInfo) { } // number of result is not greater than 10,000,000 if ((timeRange == 0) || (timeRange / intervalRange) >= MAX_INTERVAL_TIME_WINDOW) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg3); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); } return TSDB_CODE_SUCCESS; @@ -6849,9 +6855,9 @@ int32_t tscGetExprFilters(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SArray* pSelect getColumnIndexByName(pCmd, pToken, pQueryInfo, &index); } - size_t numOfNodeInSel = tscSqlExprNumOfExprs(pQueryInfo); + size_t numOfNodeInSel = tscNumOfExprs(pQueryInfo); for(int32_t k = 0; k < numOfNodeInSel; ++k) { - SExprInfo* pExpr1 = tscSqlExprGet(pQueryInfo, k); + SExprInfo* pExpr1 = tscExprGet(pQueryInfo, k); if (pExpr1->base.functionId != functionId) { continue; @@ -6873,17 +6879,17 @@ int32_t tscGetExprFilters(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SArray* pSelect tSqlExprItem item = {.pNode = pSqlExpr, .aliasName = NULL, .distinct = false}; - int32_t outputIndex = (int32_t)tscSqlExprNumOfExprs(pQueryInfo); + int32_t outputIndex = (int32_t)tscNumOfExprs(pQueryInfo); // ADD TRUE FOR TEST if (addExprAndResultField(pCmd, pQueryInfo, outputIndex, &item, true) != TSDB_CODE_SUCCESS) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } ++pQueryInfo->havingFieldNum; - size_t n = tscSqlExprNumOfExprs(pQueryInfo); - *pExpr = tscSqlExprGet(pQueryInfo, (int32_t)n - 1); + size_t n = tscNumOfExprs(pQueryInfo); + *pExpr = tscExprGet(pQueryInfo, (int32_t)n - 1); SInternalField* pField = taosArrayGet(pQueryInfo->fieldsInfo.internalField, n - 1); pField->visible = false; @@ -6932,7 +6938,7 @@ static int32_t handleExprInHavingClause(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, S return TSDB_CODE_TSC_OUT_OF_MEMORY; } } else { // error; - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } pColFilter->filterstr = @@ -6945,16 +6951,16 @@ static int32_t handleExprInHavingClause(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, S && pExpr->tokenId != TK_NOTNULL && pExpr->tokenId != TK_LIKE ) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); } } else { if (pExpr->tokenId == TK_LIKE) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } if (expr->base.resType == TSDB_DATA_TYPE_BOOL) { if (pExpr->tokenId != TK_EQ && pExpr->tokenId != TK_NE) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg3); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); } } } @@ -6991,11 +6997,11 @@ int32_t getHavingExpr(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SArray* pSelectNode } if (pLeft == NULL || pRight == NULL) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } if (pLeft->type == pRight->type) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } exchangeExpr(pExpr); @@ -7003,15 +7009,15 @@ int32_t getHavingExpr(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SArray* pSelectNode pLeft = pExpr->pLeft; pRight = pExpr->pRight; if (pLeft->type != SQL_NODE_SQLFUNCTION) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } if (pRight->type != SQL_NODE_VALUE) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } if (pExpr->tokenId >= TK_BITAND) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } if (pLeft->pParam) { @@ -7025,17 +7031,17 @@ int32_t getHavingExpr(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SArray* pSelectNode pExpr1->tokenId != TK_STRING && pExpr1->tokenId != TK_INTEGER && pExpr1->tokenId != TK_FLOAT) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } if (pExpr1->tokenId == TK_ID && (pExpr1->colInfo.z == NULL && pExpr1->colInfo.n == 0)) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } if (pExpr1->tokenId == TK_ID) { SColumnIndex index = COLUMN_INDEX_INITIALIZER; if ((getColumnIndexByName(pCmd, &pExpr1->colInfo, pQueryInfo, &index) != TSDB_CODE_SUCCESS)) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, index.tableIndex); @@ -7043,7 +7049,7 @@ int32_t getHavingExpr(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SArray* pSelectNode if (index.columnIndex <= 0 || index.columnIndex >= tscGetNumOfColumns(pTableMeta)) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } } } @@ -7051,7 +7057,7 @@ int32_t getHavingExpr(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SArray* pSelectNode pLeft->functionId = isValidFunction(pLeft->operand.z, pLeft->operand.n); if (pLeft->functionId < 0) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } return handleExprInHavingClause(pCmd, pQueryInfo, pSelectNodeList, pExpr, parentOptr); @@ -7068,11 +7074,11 @@ int32_t validateHavingClause(SQueryInfo* pQueryInfo, tSqlExpr* pExpr, SSqlCmd* p } if (pQueryInfo->groupbyExpr.numOfGroupCols <= 0) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } if (pExpr->pLeft == NULL || pExpr->pRight == NULL) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg3); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); } if (pQueryInfo->colList == NULL) { @@ -7087,41 +7093,211 @@ int32_t validateHavingClause(SQueryInfo* pQueryInfo, tSqlExpr* pExpr, SSqlCmd* p //REDO function check if (!functionCompatibleCheck(pQueryInfo, joinQuery, timeWindowQuery)) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); + } + + return TSDB_CODE_SUCCESS; +} + +static int32_t getTableNameFromSqlNode(SSqlNode* pSqlNode, SArray* tableNameList, char* msgBuf, SSqlObj* pSql) { + const char* msg1 = "invalid table name"; + + int32_t numOfTables = (int32_t) taosArrayGetSize(pSqlNode->from->list); + assert(pSqlNode->from->type == SQL_NODE_FROM_TABLELIST); + + for(int32_t j = 0; j < numOfTables; ++j) { + SRelElementPair* item = taosArrayGet(pSqlNode->from->list, j); + + SStrToken* t = &item->tableName; + if (t->type == TK_INTEGER || t->type == TK_FLOAT) { + return invalidOperationMsg(msgBuf, msg1); + } + + tscDequoteAndTrimToken(t); + if (tscValidateName(t) != TSDB_CODE_SUCCESS) { + return invalidOperationMsg(msgBuf, msg1); + } + + SName name = {0}; + if (tscSetTableFullName(&name, t, pSql) != TSDB_CODE_SUCCESS) { + return invalidOperationMsg(msgBuf, msg1); + } + + taosArrayPush(tableNameList, &name); } return TSDB_CODE_SUCCESS; } -static int32_t doLoadAllTableMeta(SSqlObj* pSql, int32_t index, SSqlNode* pSqlNode, int32_t numOfTables) { +static int32_t getTableNameFromSubquery(SSqlNode* pSqlNode, SArray* tableNameList, char* msgBuf, SSqlObj* pSql) { + int32_t numOfSub = (int32_t) taosArrayGetSize(pSqlNode->from->list); + + for(int32_t j = 0; j < numOfSub; ++j) { + SRelElementPair* sub = taosArrayGet(pSqlNode->from->list, j); + + int32_t num = (int32_t)taosArrayGetSize(sub->pSubquery); + for (int32_t i = 0; i < num; ++i) { + SSqlNode* p = taosArrayGetP(sub->pSubquery, i); + if (p->from->type == SQL_NODE_FROM_TABLELIST) { + int32_t code = getTableNameFromSqlNode(p, tableNameList, msgBuf, pSql); + if (code != TSDB_CODE_SUCCESS) { + return code; + } + } else { + getTableNameFromSubquery(p, tableNameList, msgBuf, pSql); + } + } + } + + return TSDB_CODE_SUCCESS; +} + +void tscTableMetaCallBack(void *param, TAOS_RES *res, int code); +static void freeElem(void* p) { + tfree(*(char**)p); +} + +int32_t loadAllTableMeta(SSqlObj* pSql, struct SSqlInfo* pInfo) { + SSqlCmd* pCmd = &pSql->cmd; + + // the table meta has already been loaded from local buffer or mnode already + if (pCmd->pTableMetaMap != NULL) { + return TSDB_CODE_SUCCESS; + } + + int32_t code = TSDB_CODE_SUCCESS; + + SArray* tableNameList = NULL; + SArray* pVgroupList = NULL; + SArray* plist = NULL; + STableMeta* pTableMeta = NULL; + + pCmd->pTableMetaMap = taosHashInit(4, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY), false, HASH_NO_LOCK); + + tableNameList = taosArrayInit(4, sizeof(SName)); + size_t size = taosArrayGetSize(pInfo->list); + for (int32_t i = 0; i < size; ++i) { + SSqlNode* pSqlNode = taosArrayGetP(pInfo->list, i); + if (pSqlNode->from == NULL) { + goto _end; + } + + // load the table meta in the from clause + if (pSqlNode->from->type == SQL_NODE_FROM_TABLELIST) { + code = getTableNameFromSqlNode(pSqlNode, tableNameList, tscGetErrorMsgPayload(pCmd), pSql); + if (code != TSDB_CODE_SUCCESS) { + goto _end; + } + } else { + code = getTableNameFromSubquery(pSqlNode, tableNameList, tscGetErrorMsgPayload(pCmd), pSql); + if (code != TSDB_CODE_SUCCESS) { + goto _end; + } + } + } + + uint32_t maxSize = tscGetTableMetaMaxSize(); + char name[TSDB_TABLE_FNAME_LEN] = {0}; + + char buf[80 * 1024] = {0}; + assert(maxSize < 80 * 1024); + pTableMeta = calloc(1, maxSize); + + plist = taosArrayInit(4, POINTER_BYTES); + pVgroupList = taosArrayInit(4, POINTER_BYTES); + + size_t numOfTables = taosArrayGetSize(tableNameList); + for (int32_t i = 0; i < numOfTables; ++i) { + SName* pname = taosArrayGet(tableNameList, i); + tNameExtractFullName(pname, name); + + size_t len = strlen(name); + memset(pTableMeta, 0, maxSize); + taosHashGetClone(tscTableMetaInfo, name, len, NULL, pTableMeta, -1); + + if (pTableMeta->id.uid > 0) { + if (pTableMeta->tableType == TSDB_CHILD_TABLE) { + code = tscCreateTableMetaFromSTableMeta(pTableMeta, name, buf); + + // create the child table meta from super table failed, try load it from mnode + if (code != TSDB_CODE_SUCCESS) { + char* t = strdup(name); + taosArrayPush(plist, &t); + continue; + } + } else if (pTableMeta->tableType == TSDB_SUPER_TABLE) { + // the vgroup list of a super table is not kept in local buffer, so here need retrieve it + // from the mnode each time + char* t = strdup(name); + taosArrayPush(pVgroupList, &t); + } + + STableMeta* pMeta = tscTableMetaDup(pTableMeta); + STableMetaVgroupInfo p = { .pTableMeta = pMeta }; + + const char* px = tNameGetTableName(pname); + taosHashPut(pCmd->pTableMetaMap, px, strlen(px), &p, sizeof(STableMetaVgroupInfo)); + } else { // add to the retrieve table meta array list. + char* t = strdup(name); + taosArrayPush(plist, &t); + } + } + + // load the table meta for a given table name list + if (taosArrayGetSize(plist) > 0 || taosArrayGetSize(pVgroupList) > 0) { + code = getMultiTableMetaFromMnode(pSql, plist, pVgroupList, tscTableMetaCallBack); + } + +_end: + if (plist != NULL) { + taosArrayDestroyEx(plist, freeElem); + } + + if (pVgroupList != NULL) { + taosArrayDestroyEx(pVgroupList, freeElem); + } + + if (tableNameList != NULL) { + taosArrayDestroy(tableNameList); + } + + tfree(pTableMeta); + + return code; +} + +static int32_t doLoadAllTableMeta(SSqlObj* pSql, SQueryInfo* pQueryInfo, SSqlNode* pSqlNode, int32_t numOfTables) { const char* msg1 = "invalid table name"; const char* msg2 = "invalid table alias name"; const char* msg3 = "alias name too long"; + const char* msg4 = "self join not allowed"; int32_t code = TSDB_CODE_SUCCESS; - SSqlCmd* pCmd = &pSql->cmd; - SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd, index); + + if (numOfTables > taosHashGetSize(pCmd->pTableMetaMap)) { + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg4); + } for (int32_t i = 0; i < numOfTables; ++i) { if (pQueryInfo->numOfTables <= i) { // more than one table tscAddEmptyMetaInfo(pQueryInfo); } - STableNamePair *item = taosArrayGet(pSqlNode->from->list, i); - SStrToken *oriName = &item->name; + SRelElementPair *item = taosArrayGet(pSqlNode->from->list, i); + SStrToken *oriName = &item->tableName; if (oriName->type == TK_INTEGER || oriName->type == TK_FLOAT) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } tscDequoteAndTrimToken(oriName); if (tscValidateName(oriName) != TSDB_CODE_SUCCESS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, i); - code = tscSetTableFullName(pTableMetaInfo, oriName, pSql); + code = tscSetTableFullName(&pTableMetaInfo->name, oriName, pSql); if (code != TSDB_CODE_SUCCESS) { return code; } @@ -7129,25 +7305,29 @@ static int32_t doLoadAllTableMeta(SSqlObj* pSql, int32_t index, SSqlNode* pSqlNo SStrToken* aliasName = &item->aliasName; if (TPARSER_HAS_TOKEN(*aliasName)) { if (aliasName->type == TK_INTEGER || aliasName->type == TK_FLOAT) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); } tscDequoteAndTrimToken(aliasName); - if (tscValidateName(aliasName) != TSDB_CODE_SUCCESS) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg3); - } - - if (aliasName->n >= TSDB_TABLE_NAME_LEN) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg3); + if (tscValidateName(aliasName) != TSDB_CODE_SUCCESS || aliasName->n >= TSDB_TABLE_NAME_LEN) { + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); } strncpy(pTableMetaInfo->aliasName, aliasName->z, aliasName->n); } else { - strncpy(pTableMetaInfo->aliasName, tNameGetTableName(&pTableMetaInfo->name), - tListLen(pTableMetaInfo->aliasName)); + strncpy(pTableMetaInfo->aliasName, tNameGetTableName(&pTableMetaInfo->name), tListLen(pTableMetaInfo->aliasName)); + } + + const char* name = tNameGetTableName(&pTableMetaInfo->name); + STableMetaVgroupInfo* p = taosHashGet(pCmd->pTableMetaMap, name, strlen(name)); + + pTableMetaInfo->pTableMeta = tscTableMetaDup(p->pTableMeta); + assert(pTableMetaInfo->pTableMeta != NULL); + + if (p->pVgroupInfo != NULL) { + pTableMetaInfo->vgroupList = tscVgroupsInfoDup(p->pVgroupInfo); } - code = tscGetTableMeta(pSql, pTableMetaInfo); if (code != TSDB_CODE_SUCCESS) { return code; } @@ -7156,7 +7336,7 @@ static int32_t doLoadAllTableMeta(SSqlObj* pSql, int32_t index, SSqlNode* pSqlNo return TSDB_CODE_SUCCESS; } -static STableMeta* extractTempTableMetaFromNestQuery(SQueryInfo* pUpstream) { +static STableMeta* extractTempTableMetaFromSubquery(SQueryInfo* pUpstream) { int32_t numOfColumns = pUpstream->fieldsInfo.numOfOutput; STableMeta* meta = calloc(1, sizeof(STableMeta) + sizeof(SSchema) * numOfColumns); @@ -7181,31 +7361,84 @@ static STableMeta* extractTempTableMetaFromNestQuery(SQueryInfo* pUpstream) { return meta; } -int32_t validateSqlNode(SSqlObj* pSql, SSqlNode* pSqlNode, int32_t index) { +static int32_t doValidateSubquery(SSqlNode* pSqlNode, int32_t index, SSqlObj* pSql, SQueryInfo* pQueryInfo, char* msgBuf) { + SRelElementPair* subInfo = taosArrayGet(pSqlNode->from->list, index); + + // union all is not support currently + SSqlNode* p = taosArrayGetP(subInfo->pSubquery, 0); + + SQueryInfo* pSub = calloc(1, sizeof(SQueryInfo)); + tscInitQueryInfo(pSub); + + int32_t code = validateSqlNode(pSql, p, pSub); + assert(code != TSDB_CODE_TSC_ACTION_IN_PROGRESS); + if (code != TSDB_CODE_SUCCESS) { + return code; + } + + pSub->pDownstream = pQueryInfo; + + // create dummy table meta info + STableMetaInfo* pTableMetaInfo1 = calloc(1, sizeof(STableMetaInfo)); + pTableMetaInfo1->pTableMeta = extractTempTableMetaFromSubquery(pSub); + + if (subInfo->aliasName.n > 0) { + if (subInfo->aliasName.n >= TSDB_TABLE_FNAME_LEN) { + return invalidOperationMsg(msgBuf, "subquery alias name too long"); + } + + strncpy(pTableMetaInfo1->aliasName, subInfo->aliasName.z, subInfo->aliasName.n); + } + + taosArrayPush(pQueryInfo->pUpstream, &pSub); + + // NOTE: order mix up in subquery not support yet. + pQueryInfo->order = pSub->order; + + STableMetaInfo** tmp = realloc(pQueryInfo->pTableMetaInfo, (pQueryInfo->numOfTables + 1) * POINTER_BYTES); + if (tmp == NULL) { + return TSDB_CODE_TSC_OUT_OF_MEMORY; + } + + pQueryInfo->pTableMetaInfo = tmp; + + pQueryInfo->pTableMetaInfo[pQueryInfo->numOfTables] = pTableMetaInfo1; + pQueryInfo->numOfTables += 1; + + // all columns are added into the table column list + STableMeta* pMeta = pTableMetaInfo1->pTableMeta; + int32_t startOffset = (int32_t) taosArrayGetSize(pQueryInfo->colList); + + for(int32_t i = 0; i < pMeta->tableInfo.numOfColumns; ++i) { + tscColumnListInsert(pQueryInfo->colList, i + startOffset, pMeta->id.uid, &pMeta->schema[i]); + } + + return TSDB_CODE_SUCCESS; +} + +int32_t validateSqlNode(SSqlObj* pSql, SSqlNode* pSqlNode, SQueryInfo* pQueryInfo) { assert(pSqlNode != NULL && (pSqlNode->from == NULL || taosArrayGetSize(pSqlNode->from->list) > 0)); const char* msg1 = "point interpolation query needs timestamp"; const char* msg2 = "too many tables in from clause"; const char* msg3 = "start(end) time of query range required or time range too large"; + const char* msg9 = "only tag query not compatible with normal column filter"; int32_t code = TSDB_CODE_SUCCESS; SSqlCmd* pCmd = &pSql->cmd; - SQueryInfo *pQueryInfo = tscGetQueryInfo(pCmd, index); STableMetaInfo *pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0); if (pTableMetaInfo == NULL) { pTableMetaInfo = tscAddEmptyMetaInfo(pQueryInfo); } - assert(pCmd->clauseIndex == index); - /* * handle the sql expression without from subclause - * select current_database(); + * select server_status(); * select server_version(); * select client_version(); - * select server_state(); + * select current_database(); */ if (pSqlNode->from == NULL) { assert(pSqlNode->fillType == NULL && pSqlNode->pGroupby == NULL && pSqlNode->pWhere == NULL && @@ -7214,71 +7447,55 @@ int32_t validateSqlNode(SSqlObj* pSql, SSqlNode* pSqlNode, int32_t index) { } if (pSqlNode->from->type == SQL_NODE_FROM_SUBQUERY) { - // parse the subquery in the first place - SArray* list = taosArrayGetP(pSqlNode->from->list, 0); - SSqlNode* p = taosArrayGetP(list, 0); + clearAllTableMetaInfo(pQueryInfo, false); + pQueryInfo->numOfTables = 0; - code = validateSqlNode(pSql, p, 0); - if (code == TSDB_CODE_TSC_ACTION_IN_PROGRESS) { - return code; + // parse the subquery in the first place + int32_t numOfSub = (int32_t) taosArrayGetSize(pSqlNode->from->list); + for(int32_t i = 0; i < numOfSub; ++i) { + code = doValidateSubquery(pSqlNode, i, pSql, pQueryInfo, tscGetErrorMsgPayload(pCmd)); + if (code != TSDB_CODE_SUCCESS) { + return code; + } } - if (code != TSDB_CODE_SUCCESS) { - return code; + if (validateSelectNodeList(pCmd, pQueryInfo, pSqlNode->pSelNodeList, false, false, false) != TSDB_CODE_SUCCESS) { + return TSDB_CODE_TSC_INVALID_OPERATION; } - pQueryInfo = pCmd->pQueryInfo[0]; - - SQueryInfo* current = calloc(1, sizeof(SQueryInfo)); - - tscInitQueryInfo(current); - taosArrayPush(current->pUpstream, &pQueryInfo); - - STableMeta* pTableMeta = extractTempTableMetaFromNestQuery(pQueryInfo); - STableMetaInfo* pTableMetaInfo1 = calloc(1, sizeof(STableMetaInfo)); - pTableMetaInfo1->pTableMeta = pTableMeta; - - current->pTableMetaInfo = calloc(1, POINTER_BYTES); - current->pTableMetaInfo[0] = pTableMetaInfo1; - current->numOfTables = 1; - current->order = pQueryInfo->order; - - pCmd->pQueryInfo[0] = current; - pQueryInfo->pDownstream = current; + if (pSqlNode->pWhere != NULL) { + if (validateWhereNode(pQueryInfo, &pSqlNode->pWhere, pSql) != TSDB_CODE_SUCCESS) { + return TSDB_CODE_TSC_INVALID_OPERATION; + } - if (validateSelectNodeList(pCmd, current, pSqlNode->pSelNodeList, false, false, false) != TSDB_CODE_SUCCESS) { - return TSDB_CODE_TSC_INVALID_SQL; + STableMeta* pTableMeta = tscGetMetaInfo(pQueryInfo, 0)->pTableMeta; + if (pTableMeta->tableInfo.precision == TSDB_TIME_PRECISION_MILLI) { + pQueryInfo->window.skey = pQueryInfo->window.skey / 1000; + pQueryInfo->window.ekey = pQueryInfo->window.ekey / 1000; + } } - } else { pQueryInfo->command = TSDB_SQL_SELECT; - size_t fromSize = taosArrayGetSize(pSqlNode->from->list); - if (fromSize > TSDB_MAX_JOIN_TABLE_NUM) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2); + size_t numOfTables = taosArrayGetSize(pSqlNode->from->list); + if (numOfTables > TSDB_MAX_JOIN_TABLE_NUM) { + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); } // set all query tables, which are maybe more than one. - code = doLoadAllTableMeta(pSql, index, pSqlNode, (int32_t) fromSize); + code = doLoadAllTableMeta(pSql, pQueryInfo, pSqlNode, (int32_t) numOfTables); if (code != TSDB_CODE_SUCCESS) { return code; } bool isSTable = UTIL_TABLE_IS_SUPER_TABLE(pTableMetaInfo); - if (isSTable) { - code = tscGetSTableVgroupInfo(pSql, index); // TODO refactor: getTablemeta along with vgroupInfo - if (code != TSDB_CODE_SUCCESS) { - return code; - } - TSDB_QUERY_SET_TYPE(pQueryInfo->type, TSDB_QUERY_TYPE_STABLE_QUERY); - } else { - TSDB_QUERY_SET_TYPE(pQueryInfo->type, TSDB_QUERY_TYPE_TABLE_QUERY); - } + int32_t type = isSTable? TSDB_QUERY_TYPE_STABLE_QUERY:TSDB_QUERY_TYPE_TABLE_QUERY; + TSDB_QUERY_SET_TYPE(pQueryInfo->type, type); // parse the group by clause in the first place if (validateGroupbyNode(pQueryInfo, pSqlNode->pGroupby, pCmd) != TSDB_CODE_SUCCESS) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } // set where info @@ -7286,7 +7503,7 @@ int32_t validateSqlNode(SSqlObj* pSql, SSqlNode* pSqlNode, int32_t index) { if (pSqlNode->pWhere != NULL) { if (validateWhereNode(pQueryInfo, &pSqlNode->pWhere, pSql) != TSDB_CODE_SUCCESS) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } pSqlNode->pWhere = NULL; @@ -7294,10 +7511,9 @@ int32_t validateSqlNode(SSqlObj* pSql, SSqlNode* pSqlNode, int32_t index) { pQueryInfo->window.skey = pQueryInfo->window.skey / 1000; pQueryInfo->window.ekey = pQueryInfo->window.ekey / 1000; } - } else { // set the time rang - if (taosArrayGetSize(pSqlNode->from->list) > 1) { - // If it is a join query, no where clause is not allowed. - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), "condition missing for join query "); + } else { + if (taosArrayGetSize(pSqlNode->from->list) > 1) { // Cross join not allowed yet + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), "cross join not supported yet"); } } @@ -7307,29 +7523,43 @@ int32_t validateSqlNode(SSqlObj* pSql, SSqlNode* pSqlNode, int32_t index) { if (validateSelectNodeList(pCmd, pQueryInfo, pSqlNode->pSelNodeList, isSTable, joinQuery, timeWindowQuery) != TSDB_CODE_SUCCESS) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } // set order by info if (validateOrderbyNode(pCmd, pQueryInfo, pSqlNode, tscGetTableSchema(pTableMetaInfo->pTableMeta)) != TSDB_CODE_SUCCESS) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } // set interval value if (validateIntervalNode(pSql, pQueryInfo, pSqlNode) != TSDB_CODE_SUCCESS) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } else { if (isTimeWindowQuery(pQueryInfo) && (validateFunctionsInIntervalOrGroupbyQuery(pCmd, pQueryInfo) != TSDB_CODE_SUCCESS)) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; + } + } + + if (tscQueryTags(pQueryInfo)) { + SExprInfo* pExpr1 = tscExprGet(pQueryInfo, 0); + + if (pExpr1->base.functionId != TSDB_FUNC_TID_TAG) { + int32_t numOfCols = (int32_t)taosArrayGetSize(pQueryInfo->colList); + for (int32_t i = 0; i < numOfCols; ++i) { + SColumn* pCols = taosArrayGetP(pQueryInfo->colList, i); + if (pCols->info.flist.numOfFilters > 0) { + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg9); + } + } } } // parse the having clause in the first place if (validateHavingClause(pQueryInfo, pSqlNode->pHaving, pCmd, pSqlNode->pSelNodeList, joinQuery, timeWindowQuery) != TSDB_CODE_SUCCESS) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } /* @@ -7338,14 +7568,13 @@ int32_t validateSqlNode(SSqlObj* pSql, SSqlNode* pSqlNode, int32_t index) { */ if (isSTable) { tscTansformFuncForSTableQuery(pQueryInfo); - if (hasUnsupportFunctionsForSTableQuery(pCmd, pQueryInfo)) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } } if (validateSessionNode(pCmd, pQueryInfo, pSqlNode) != TSDB_CODE_SUCCESS) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } // no result due to invalid query time range @@ -7355,18 +7584,18 @@ int32_t validateSqlNode(SSqlObj* pSql, SSqlNode* pSqlNode, int32_t index) { } if (!hasTimestampForPointInterpQuery(pQueryInfo)) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); } // in case of join query, time range is required. if (QUERY_IS_JOIN_QUERY(pQueryInfo->type)) { int64_t timeRange = ABS(pQueryInfo->window.skey - pQueryInfo->window.ekey); if (timeRange == 0 && pQueryInfo->window.skey == 0) { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg3); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); } } - if ((code = validateLimitNode(pCmd, pQueryInfo, index, pSqlNode, pSql)) != TSDB_CODE_SUCCESS) { + if ((code = validateLimitNode(pCmd, pQueryInfo, pSqlNode, pSql)) != TSDB_CODE_SUCCESS) { return code; } @@ -7382,6 +7611,34 @@ int32_t validateSqlNode(SSqlObj* pSql, SSqlNode* pSqlNode, int32_t index) { } } + { // set the query info + pQueryInfo->projectionQuery = tscIsProjectionQuery(pQueryInfo); + pQueryInfo->hasFilter = tscHasColumnFilter(pQueryInfo); + pQueryInfo->simpleAgg = isSimpleAggregateRv(pQueryInfo); + pQueryInfo->onlyTagQuery = onlyTagPrjFunction(pQueryInfo); + pQueryInfo->groupbyColumn = tscGroupbyColumn(pQueryInfo); + + pQueryInfo->arithmeticOnAgg = tsIsArithmeticQueryOnAggResult(pQueryInfo); + + SExprInfo** p = NULL; + int32_t numOfExpr = 0; + code = createProjectionExpr(pQueryInfo, pTableMetaInfo, &p, &numOfExpr); + + if (pQueryInfo->exprList1 == NULL) { + pQueryInfo->exprList1 = taosArrayInit(4, POINTER_BYTES); + } + + taosArrayAddBatch(pQueryInfo->exprList1, (void*) p, numOfExpr); + } + +#if 0 + SQueryNode* p = qCreateQueryPlan(pQueryInfo); + char* s = queryPlanToString(p); + tfree(s); + + qDestroyQueryPlan(p); +#endif + return TSDB_CODE_SUCCESS; // Does not build query message here } @@ -7473,7 +7730,7 @@ int32_t exprTreeFromSqlExpr(SSqlCmd* pCmd, tExprNode **pExpr, const tSqlExpr* pS return TSDB_CODE_SUCCESS; } else { - return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), "not support filter expression"); + return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), "not support filter expression"); } } else { @@ -7493,9 +7750,9 @@ int32_t exprTreeFromSqlExpr(SSqlCmd* pCmd, tExprNode **pExpr, const tSqlExpr* pS if ((*pExpr)->_node.optr == TSDB_BINARY_OP_DIVIDE) { if (pRight->nodeType == TSQL_NODE_VALUE) { if (pRight->pVal->nType == TSDB_DATA_TYPE_INT && pRight->pVal->i64 == 0) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } else if (pRight->pVal->nType == TSDB_DATA_TYPE_FLOAT && pRight->pVal->dKey == 0) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } } } @@ -7504,7 +7761,7 @@ int32_t exprTreeFromSqlExpr(SSqlCmd* pCmd, tExprNode **pExpr, const tSqlExpr* pS if ((*pExpr)->_node.optr != TSDB_RELATION_EQUAL && (*pExpr)->_node.optr != TSDB_RELATION_NOT_EQUAL) { if (pRight != NULL && pRight->nodeType == TSQL_NODE_VALUE) { if (pRight->pVal->nType == TSDB_DATA_TYPE_BOOL) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } } } diff --git a/src/client/src/tscServer.c b/src/client/src/tscServer.c index 9b7d5c0c7fa79ef6937239d8604d40d4cedd6278..a3032a342e1d97278e38033092f05441209c3eb1 100644 --- a/src/client/src/tscServer.c +++ b/src/client/src/tscServer.c @@ -116,7 +116,7 @@ static void tscDumpEpSetFromVgroupInfo(SRpcEpSet *pEpSet, SNewVgroupInfo *pVgrou static void tscUpdateVgroupInfo(SSqlObj *pSql, SRpcEpSet *pEpSet) { SSqlCmd *pCmd = &pSql->cmd; - STableMetaInfo *pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, pCmd->clauseIndex, 0); + STableMetaInfo *pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, 0); if (pTableMetaInfo == NULL || pTableMetaInfo->pTableMeta == NULL) { return; } @@ -335,7 +335,7 @@ void tscProcessMsgFromServer(SRpcMsg *rpcMsg, SRpcEpSet *pEpSet) { return; } - SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd, 0); + SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd); if (pQueryInfo != NULL && pQueryInfo->type == TSDB_QUERY_TYPE_FREE_RESOURCE) { tscDebug("0x%"PRIx64" sqlObj needs to be released or DB connection is closed, cmd:%d type:%d, pObj:%p signature:%p", pSql->self, pCmd->command, pQueryInfo->type, pObj, pObj->signature); @@ -360,7 +360,7 @@ void tscProcessMsgFromServer(SRpcMsg *rpcMsg, SRpcEpSet *pEpSet) { // set the flag to denote that sql string needs to be re-parsed and build submit block with table schema if (cmd == TSDB_SQL_INSERT && rpcMsg->code == TSDB_CODE_TDB_TABLE_RECONFIGURE) { - pSql->cmd.submitSchema = 1; + pSql->cmd.insertParam.schemaAttached = 1; } if ((cmd == TSDB_SQL_SELECT || cmd == TSDB_SQL_UPDATE_TAGS_VAL) && @@ -477,7 +477,7 @@ int doBuildAndSendMsg(SSqlObj *pSql) { pCmd->command == TSDB_SQL_INSERT || pCmd->command == TSDB_SQL_CONNECT || pCmd->command == TSDB_SQL_HB || - pCmd->command == TSDB_SQL_META || +// pCmd->command == TSDB_SQL_META || pCmd->command == TSDB_SQL_STABLEVGROUP) { pRes->code = tscBuildMsg[pCmd->command](pSql, NULL); } @@ -506,7 +506,7 @@ int tscBuildAndSendRequest(SSqlObj *pSql, SQueryInfo* pQueryInfo) { uint32_t type = 0; if (pQueryInfo == NULL) { - pQueryInfo = tscGetQueryInfo(pCmd, pCmd->clauseIndex); + pQueryInfo = tscGetQueryInfo(pCmd); } STableMetaInfo *pTableMetaInfo = NULL; @@ -539,7 +539,7 @@ int tscBuildAndSendRequest(SSqlObj *pSql, SQueryInfo* pQueryInfo) { int tscBuildFetchMsg(SSqlObj *pSql, SSqlInfo *pInfo) { SRetrieveTableMsg *pRetrieveMsg = (SRetrieveTableMsg *) pSql->cmd.payload; - SQueryInfo *pQueryInfo = tscGetActiveQueryInfo(&pSql->cmd); + SQueryInfo *pQueryInfo = tscGetQueryInfo(&pSql->cmd); pRetrieveMsg->free = htons(pQueryInfo->type); pRetrieveMsg->qId = htobe64(pSql->res.qId); @@ -581,25 +581,8 @@ int tscBuildFetchMsg(SSqlObj *pSql, SSqlInfo *pInfo) { } int tscBuildSubmitMsg(SSqlObj *pSql, SSqlInfo *pInfo) { - SQueryInfo *pQueryInfo = tscGetQueryInfo(&pSql->cmd, 0); + SQueryInfo *pQueryInfo = tscGetQueryInfo(&pSql->cmd); STableMeta* pTableMeta = tscGetMetaInfo(pQueryInfo, 0)->pTableMeta; - - char* pMsg = pSql->cmd.payload; - - // NOTE: shell message size should not include SMsgDesc - int32_t size = pSql->cmd.payloadLen - sizeof(SMsgDesc); - - SMsgDesc* pMsgDesc = (SMsgDesc*) pMsg; - pMsgDesc->numOfVnodes = htonl(1); // always one vnode - - pMsg += sizeof(SMsgDesc); - SSubmitMsg *pShellMsg = (SSubmitMsg *)pMsg; - - pShellMsg->header.vgId = htonl(pTableMeta->vgId); - pShellMsg->header.contLen = htonl(size); // the length not includes the size of SMsgDesc - pShellMsg->length = pShellMsg->header.contLen; - - pShellMsg->numOfBlocks = htonl(pSql->cmd.numOfTablesInSubmit); // number of tables to be inserted // pSql->cmd.payloadLen is set during copying data into payload pSql->cmd.msgType = TSDB_MSG_TYPE_SUBMIT; @@ -608,29 +591,28 @@ int tscBuildSubmitMsg(SSqlObj *pSql, SSqlInfo *pInfo) { taosHashGetClone(tscVgroupMap, &pTableMeta->vgId, sizeof(pTableMeta->vgId), NULL, &vgroupInfo, sizeof(SNewVgroupInfo)); tscDumpEpSetFromVgroupInfo(&pSql->epSet, &vgroupInfo); - tscDebug("0x%"PRIx64" build submit msg, vgId:%d numOfTables:%d numberOfEP:%d", pSql->self, pTableMeta->vgId, pSql->cmd.numOfTablesInSubmit, - pSql->epSet.numOfEps); + tscDebug("0x%"PRIx64" submit msg built, numberOfEP:%d", pSql->self, pSql->epSet.numOfEps); + return TSDB_CODE_SUCCESS; } /* * for table query, simply return the size <= 1k */ -static int32_t tscEstimateQueryMsgSize(SSqlObj *pSql, int32_t clauseIndex) { +static int32_t tscEstimateQueryMsgSize(SSqlObj *pSql) { const static int32_t MIN_QUERY_MSG_PKT_SIZE = TSDB_MAX_BYTES_PER_ROW * 5; SSqlCmd* pCmd = &pSql->cmd; - SQueryInfo *pQueryInfo = tscGetQueryInfo(pCmd, clauseIndex); + SQueryInfo *pQueryInfo = tscGetQueryInfo(pCmd); int32_t srcColListSize = (int32_t)(taosArrayGetSize(pQueryInfo->colList) * sizeof(SColumnInfo)); - size_t numOfExprs = tscSqlExprNumOfExprs(pQueryInfo); + size_t numOfExprs = tscNumOfExprs(pQueryInfo); int32_t exprSize = (int32_t)(sizeof(SSqlExpr) * numOfExprs * 2); int32_t tsBufSize = (pQueryInfo->tsBuf != NULL) ? pQueryInfo->tsBuf->fileSize : 0; int32_t sqlLen = (int32_t) strlen(pSql->sqlstr) + 1; - int32_t tableSerialize = 0; STableMetaInfo *pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0); if (pTableMetaInfo->pVgroupTables != NULL) { @@ -754,7 +736,7 @@ static int32_t serializeColFilterInfo(SColumnFilterInfo* pColFilters, int16_t nu if (pColFilter->lowerRelOptr == TSDB_RELATION_INVALID && pColFilter->upperRelOptr == TSDB_RELATION_INVALID) { tscError("invalid filter info"); - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } } @@ -773,7 +755,7 @@ static int32_t serializeSqlExpr(SSqlExpr* pExpr, STableMetaInfo* pTableMetaInfo, if (validateColumn && !tscValidateColumnId(pTableMetaInfo, pExpr->colInfo.colId, pExpr->numOfParams)) { tscError("0x%"PRIx64" table schema is not matched with parsed sql", id); - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } assert(pExpr->resColId < 0); @@ -816,14 +798,14 @@ int tscBuildQueryMsg(SSqlObj *pSql, SSqlInfo *pInfo) { SSqlCmd *pCmd = &pSql->cmd; int32_t code = TSDB_CODE_SUCCESS; - int32_t size = tscEstimateQueryMsgSize(pSql, pCmd->clauseIndex); + int32_t size = tscEstimateQueryMsgSize(pSql); if (TSDB_CODE_SUCCESS != tscAllocPayload(pCmd, size)) { tscError("%p failed to malloc for query msg", pSql); - return TSDB_CODE_TSC_INVALID_SQL; // todo add test for this + return TSDB_CODE_TSC_INVALID_OPERATION; // todo add test for this } - SQueryInfo *pQueryInfo = tscGetActiveQueryInfo(pCmd); + SQueryInfo *pQueryInfo = tscGetQueryInfo(pCmd); SQueryAttr query = {{0}}; tscCreateQueryFromQueryInfo(pQueryInfo, &query, pSql); @@ -929,7 +911,7 @@ int tscBuildQueryMsg(SSqlObj *pSql, SSqlInfo *pInfo) { goto _end; } - SSqlGroupbyExpr *pGroupbyExpr = query.pGroupbyExpr; + SGroupbyExpr *pGroupbyExpr = query.pGroupbyExpr; if (pGroupbyExpr->numOfGroupCols > 0) { pQueryMsg->orderByIdx = htons(pGroupbyExpr->orderIndex); pQueryMsg->orderType = htons(pGroupbyExpr->orderType); @@ -1050,8 +1032,8 @@ int32_t tscBuildCreateDbMsg(SSqlObj *pSql, SSqlInfo *pInfo) { SCreateDbMsg *pCreateDbMsg = (SCreateDbMsg *)pCmd->payload; - assert(pCmd->numOfClause == 1); - STableMetaInfo *pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, pCmd->clauseIndex, 0); +// assert(pCmd->numOfClause == 1); + STableMetaInfo *pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, 0); int32_t code = tNameExtractFullName(&pTableMetaInfo->name, pCreateDbMsg->db); assert(code == TSDB_CODE_SUCCESS); @@ -1171,7 +1153,7 @@ int32_t tscBuildDropDbMsg(SSqlObj *pSql, SSqlInfo *pInfo) { SDropDbMsg *pDropDbMsg = (SDropDbMsg*)pCmd->payload; - STableMetaInfo *pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, pCmd->clauseIndex, 0); + STableMetaInfo *pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, 0); int32_t code = tNameExtractFullName(&pTableMetaInfo->name, pDropDbMsg->db); assert(code == TSDB_CODE_SUCCESS && pTableMetaInfo->name.type == TSDB_DB_NAME_T); @@ -1192,9 +1174,10 @@ int32_t tscBuildDropTableMsg(SSqlObj *pSql, SSqlInfo *pInfo) { } SCMDropTableMsg *pDropTableMsg = (SCMDropTableMsg*)pCmd->payload; - STableMetaInfo *pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, pCmd->clauseIndex, 0); + STableMetaInfo *pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, 0); tNameExtractFullName(&pTableMetaInfo->name, pDropTableMsg->name); + pDropTableMsg->supertable = (pInfo->pMiscInfo->tableType == TSDB_SUPER_TABLE)? 1:0; pDropTableMsg->igNotExists = pInfo->pMiscInfo->existsCheck ? 1 : 0; pCmd->msgType = TSDB_MSG_TYPE_CM_DROP_TABLE; return TSDB_CODE_SUCCESS; @@ -1249,7 +1232,7 @@ int32_t tscBuildUseDbMsg(SSqlObj *pSql, SSqlInfo *pInfo) { } SUseDbMsg *pUseDbMsg = (SUseDbMsg *)pCmd->payload; - STableMetaInfo *pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, pCmd->clauseIndex, 0); + STableMetaInfo *pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, 0); tNameExtractFullName(&pTableMetaInfo->name, pUseDbMsg->db); pCmd->msgType = TSDB_MSG_TYPE_CM_USE_DB; @@ -1266,7 +1249,7 @@ int32_t tscBuildSyncDbReplicaMsg(SSqlObj* pSql, SSqlInfo *pInfo) { } SSyncDbMsg *pSyncMsg = (SSyncDbMsg *)pCmd->payload; - STableMetaInfo *pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, pCmd->clauseIndex, 0); + STableMetaInfo *pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, 0); tNameExtractFullName(&pTableMetaInfo->name, pSyncMsg->db); pCmd->msgType = TSDB_MSG_TYPE_CM_SYNC_DB; @@ -1286,7 +1269,7 @@ int32_t tscBuildShowMsg(SSqlObj *pSql, SSqlInfo *pInfo) { SShowMsg *pShowMsg = (SShowMsg *)pCmd->payload; - STableMetaInfo *pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, pCmd->clauseIndex, 0); + STableMetaInfo *pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, 0); if (tNameIsEmpty(&pTableMetaInfo->name)) { pthread_mutex_lock(&pObj->mutex); @@ -1360,7 +1343,7 @@ int tscBuildCreateTableMsg(SSqlObj *pSql, SSqlInfo *pInfo) { SSchema *pSchema; SSqlCmd *pCmd = &pSql->cmd; - SQueryInfo *pQueryInfo = tscGetQueryInfo(pCmd, 0); + SQueryInfo *pQueryInfo = tscGetQueryInfo(pCmd); STableMetaInfo *pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0); // Reallocate the payload size @@ -1449,7 +1432,7 @@ int tscBuildCreateTableMsg(SSqlObj *pSql, SSqlInfo *pInfo) { } int tscEstimateAlterTableMsgLength(SSqlCmd *pCmd) { - SQueryInfo *pQueryInfo = tscGetQueryInfo(pCmd, 0); + SQueryInfo *pQueryInfo = tscGetQueryInfo(pCmd); return minMsgSize() + sizeof(SAlterTableMsg) + sizeof(SSchema) * tscNumOfFields(pQueryInfo) + TSDB_EXTRA_PAYLOAD_SIZE; } @@ -1458,7 +1441,7 @@ int tscBuildAlterTableMsg(SSqlObj *pSql, SSqlInfo *pInfo) { int msgLen = 0; SSqlCmd *pCmd = &pSql->cmd; - SQueryInfo *pQueryInfo = tscGetQueryInfo(pCmd, 0); + SQueryInfo *pQueryInfo = tscGetQueryInfo(pCmd); STableMetaInfo *pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0); @@ -1507,7 +1490,7 @@ int tscBuildUpdateTagMsg(SSqlObj* pSql, SSqlInfo *pInfo) { SUpdateTableTagValMsg* pUpdateMsg = (SUpdateTableTagValMsg*) pCmd->payload; pCmd->payloadLen = htonl(pUpdateMsg->head.contLen); - SQueryInfo *pQueryInfo = tscGetQueryInfo(pCmd, 0); + SQueryInfo *pQueryInfo = tscGetQueryInfo(pCmd); STableMeta *pTableMeta = tscGetMetaInfo(pQueryInfo, 0)->pTableMeta; SNewVgroupInfo vgroupInfo = {.vgId = -1}; @@ -1527,7 +1510,7 @@ int tscAlterDbMsg(SSqlObj *pSql, SSqlInfo *pInfo) { SAlterDbMsg *pAlterDbMsg = (SAlterDbMsg* )pCmd->payload; pAlterDbMsg->dbType = -1; - STableMetaInfo *pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, pCmd->clauseIndex, 0); + STableMetaInfo *pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, 0); tNameExtractFullName(&pTableMetaInfo->name, pAlterDbMsg->db); return TSDB_CODE_SUCCESS; @@ -1543,7 +1526,7 @@ int tscBuildRetrieveFromMgmtMsg(SSqlObj *pSql, SSqlInfo *pInfo) { return TSDB_CODE_TSC_OUT_OF_MEMORY; } - SQueryInfo *pQueryInfo = tscGetQueryInfo(pCmd, 0); + SQueryInfo *pQueryInfo = tscGetQueryInfo(pCmd); SRetrieveTableMsg *pRetrieveMsg = (SRetrieveTableMsg*)pCmd->payload; pRetrieveMsg->qId = htobe64(pSql->res.qId); pRetrieveMsg->free = htons(pQueryInfo->type); @@ -1567,7 +1550,7 @@ static int tscLocalResultCommonBuilder(SSqlObj *pSql, int32_t numOfRes) { pRes->row = 0; pRes->rspType = 1; - SQueryInfo *pQueryInfo = tscGetQueryInfo(pCmd, pCmd->clauseIndex); + SQueryInfo *pQueryInfo = tscGetQueryInfo(pCmd); if (tscCreateResPointerInfo(pRes, pQueryInfo) != TSDB_CODE_SUCCESS) { return pRes->code; } @@ -1591,7 +1574,7 @@ static int tscLocalResultCommonBuilder(SSqlObj *pSql, int32_t numOfRes) { int tscProcessDescribeTableRsp(SSqlObj *pSql) { SSqlCmd * pCmd = &pSql->cmd; - STableMetaInfo *pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, pCmd->clauseIndex, 0); + STableMetaInfo *pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, 0); STableComInfo tinfo = tscGetTableInfo(pTableMetaInfo->pTableMeta); @@ -1615,8 +1598,13 @@ int tscProcessRetrieveLocalMergeRsp(SSqlObj *pSql) { return code; } + if (pRes->pLocalMerger == NULL) { // no result from subquery, so abort here directly. + (*pSql->fp)(pSql->param, pSql, pRes->numOfRows); + return code; + } + // global aggregation may be the upstream for parent query - SQueryInfo *pQueryInfo = tscGetActiveQueryInfo(pCmd); + SQueryInfo *pQueryInfo = tscGetQueryInfo(pCmd); if (pQueryInfo->pQInfo == NULL) { STableGroupInfo tableGroupInfo = {.numOfTables = 1, .pGroupList = taosArrayInit(1, POINTER_BYTES),}; tableGroupInfo.map = taosHashInit(1, taosGetDefaultHashFunction(TSDB_DATA_TYPE_INT), true, HASH_NO_LOCK); @@ -1627,21 +1615,21 @@ int tscProcessRetrieveLocalMergeRsp(SSqlObj *pSql) { taosArrayPush(group, &tableKeyInfo); taosArrayPush(tableGroupInfo.pGroupList, &group); - SExprInfo* list = calloc(tscSqlExprNumOfExprs(pQueryInfo), sizeof(SExprInfo)); - for(int32_t i = 0; i < tscSqlExprNumOfExprs(pQueryInfo); ++i) { - SExprInfo* pExprInfo = tscSqlExprGet(pQueryInfo, i); + // todo remove it + SExprInfo* list = calloc(tscNumOfExprs(pQueryInfo), sizeof(SExprInfo)); + for(int32_t i = 0; i < tscNumOfExprs(pQueryInfo); ++i) { + SExprInfo* pExprInfo = tscExprGet(pQueryInfo, i); list[i] = *pExprInfo; } - pQueryInfo->pQInfo = createQueryInfoFromQueryNode(pQueryInfo, list, &tableGroupInfo, NULL, NULL, pRes->pLocalMerger, MERGE_STAGE); + pQueryInfo->pQInfo = createQInfoFromQueryNode(pQueryInfo, list, &tableGroupInfo, NULL, NULL, pRes->pLocalMerger, MERGE_STAGE); + tfree(list); } uint64_t localQueryId = 0; qTableQuery(pQueryInfo->pQInfo, &localQueryId); convertQueryResult(pRes, pQueryInfo); - handleDownstreamOperator(pRes, pQueryInfo); - code = pRes->code; if (pRes->code == TSDB_CODE_SUCCESS) { (*pSql->fp)(pSql->param, pSql, pRes->numOfRows); @@ -1687,15 +1675,16 @@ int tscBuildConnectMsg(SSqlObj *pSql, SSqlInfo *pInfo) { } int tscBuildTableMetaMsg(SSqlObj *pSql, SSqlInfo *pInfo) { +#if 0 SSqlCmd *pCmd = &pSql->cmd; - SQueryInfo *pQueryInfo = tscGetQueryInfo(&pSql->cmd, 0); + SQueryInfo *pQueryInfo = tscGetQueryInfo(&pSql->cmd); STableMetaInfo *pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0); STableInfoMsg *pInfoMsg = (STableInfoMsg *)pCmd->payload; int32_t code = tNameExtractFullName(&pTableMetaInfo->name, pInfoMsg->tableFname); if (code != TSDB_CODE_SUCCESS) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } pInfoMsg->createFlag = htons(pSql->cmd.autoCreated ? 1 : 0); @@ -1708,64 +1697,40 @@ int tscBuildTableMetaMsg(SSqlObj *pSql, SSqlInfo *pInfo) { pCmd->payloadLen = (int32_t)(pMsg - (char*)pInfoMsg); pCmd->msgType = TSDB_MSG_TYPE_CM_TABLE_META; +#endif return TSDB_CODE_SUCCESS; } /** * multi table meta req pkg format: - * | SMgmtHead | SMultiTableInfoMsg | tableId0 | tableId1 | tableId2 | ...... - * no used 4B + * |SMultiTableInfoMsg | tableId0 | tableId1 | tableId2 | ...... + * 4B **/ -int tscBuildMultiMeterMetaMsg(SSqlObj *pSql, SSqlInfo *pInfo) { -#if 0 +int tscBuildMultiTableMetaMsg(SSqlObj *pSql, SSqlInfo *pInfo) { SSqlCmd *pCmd = &pSql->cmd; - // copy payload content to temp buff - char *tmpData = 0; - if (pCmd->payloadLen > 0) { - if ((tmpData = calloc(1, pCmd->payloadLen + 1)) == NULL) return -1; - memcpy(tmpData, pCmd->payload, pCmd->payloadLen); - } - - // fill head info - SMgmtHead *pMgmt = (SMgmtHead *)(pCmd->payload + tsRpcHeadSize); - memset(pMgmt->db, 0, TSDB_TABLE_FNAME_LEN); // server don't need the db - - SMultiTableInfoMsg *pInfoMsg = (SMultiTableInfoMsg *)(pCmd->payload + tsRpcHeadSize + sizeof(SMgmtHead)); - pInfoMsg->numOfTables = htonl((int32_t)pCmd->count); - - if (pCmd->payloadLen > 0) { - memcpy(pInfoMsg->tableIds, tmpData, pCmd->payloadLen); - } - - tfree(tmpData); - - pCmd->payloadLen += sizeof(SMgmtHead) + sizeof(SMultiTableInfoMsg); pCmd->msgType = TSDB_MSG_TYPE_CM_TABLES_META; - assert(pCmd->payloadLen + minMsgSize() <= pCmd->allocSize); - tscDebug("0x%"PRIx64" build load multi-metermeta msg completed, numOfTables:%d, msg size:%d", pSql->self, pCmd->count, + tscDebug("0x%"PRIx64" build load multi-tablemeta msg completed, numOfTables:%d, msg size:%d", pSql->self, pCmd->count, pCmd->payloadLen); return pCmd->payloadLen; -#endif - return 0; } int tscBuildSTableVgroupMsg(SSqlObj *pSql, SSqlInfo *pInfo) { SSqlCmd *pCmd = &pSql->cmd; char* pMsg = pCmd->payload; - SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd, 0); + SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd); SSTableVgroupMsg *pStableVgroupMsg = (SSTableVgroupMsg *)pMsg; pStableVgroupMsg->numOfTables = htonl(pQueryInfo->numOfTables); pMsg += sizeof(SSTableVgroupMsg); for (int32_t i = 0; i < pQueryInfo->numOfTables; ++i) { - STableMetaInfo *pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, pCmd->clauseIndex, i); + STableMetaInfo *pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, i); int32_t code = tNameExtractFullName(&pTableMetaInfo->name, pMsg); assert(code == TSDB_CODE_SUCCESS); @@ -1825,18 +1790,16 @@ int tscBuildHeartBeatMsg(SSqlObj *pSql, SSqlInfo *pInfo) { return TSDB_CODE_SUCCESS; } -int tscProcessTableMetaRsp(SSqlObj *pSql) { - STableMetaMsg *pMetaMsg = (STableMetaMsg *)pSql->res.pRsp; - - pMetaMsg->tid = htonl(pMetaMsg->tid); - pMetaMsg->sversion = htons(pMetaMsg->sversion); - pMetaMsg->tversion = htons(pMetaMsg->tversion); +static int32_t tableMetaMsgConvert(STableMetaMsg* pMetaMsg) { + pMetaMsg->tid = htonl(pMetaMsg->tid); + pMetaMsg->sversion = htons(pMetaMsg->sversion); + pMetaMsg->tversion = htons(pMetaMsg->tversion); pMetaMsg->vgroup.vgId = htonl(pMetaMsg->vgroup.vgId); - pMetaMsg->uid = htobe64(pMetaMsg->uid); - pMetaMsg->suid = pMetaMsg->suid; - pMetaMsg->contLen = htons(pMetaMsg->contLen); + + pMetaMsg->uid = htobe64(pMetaMsg->uid); +// pMetaMsg->contLen = htonl(pMetaMsg->contLen); pMetaMsg->numOfColumns = htons(pMetaMsg->numOfColumns); - + if ((pMetaMsg->tableType != TSDB_SUPER_TABLE) && (pMetaMsg->tid <= 0 || pMetaMsg->vgroup.vgId < 2 || pMetaMsg->vgroup.numOfEps <= 0)) { tscError("invalid value in table numOfEps:%d, vgId:%d tid:%d, name:%s", pMetaMsg->vgroup.numOfEps, pMetaMsg->vgroup.vgId, @@ -1871,23 +1834,34 @@ int tscProcessTableMetaRsp(SSqlObj *pSql) { pSchema++; } - - STableMetaInfo *pTableMetaInfo = tscGetTableMetaInfoFromCmd(&pSql->cmd, 0, 0); - assert(pTableMetaInfo->pTableMeta == NULL); - STableMeta* pTableMeta = tscCreateTableMetaFromMsg(pMetaMsg); - if (!tIsValidSchema(pTableMeta->schema, pTableMeta->tableInfo.numOfColumns, pTableMeta->tableInfo.numOfTags)) { - tscError("0x%"PRIx64" invalid table meta from mnode, name:%s", pSql->self, tNameGetTableName(&pTableMetaInfo->name)); - return TSDB_CODE_TSC_INVALID_VALUE; - } + return TSDB_CODE_SUCCESS; +} + +// update the vgroupInfo if needed +static void doUpdateVgroupInfo(STableMeta *pTableMeta, SVgroupMsg *pVgroupMsg) { + if (pTableMeta->vgId > 0) { + int32_t vgId = pTableMeta->vgId; + assert(pTableMeta->tableType != TSDB_SUPER_TABLE); - assert(pTableMeta->tableType == TSDB_SUPER_TABLE || pTableMeta->tableType == TSDB_CHILD_TABLE || pTableMeta->tableType == TSDB_NORMAL_TABLE || pTableMeta->tableType == TSDB_STREAM_TABLE); + SNewVgroupInfo vgroupInfo = {.inUse = -1}; + taosHashGetClone(tscVgroupMap, &vgId, sizeof(vgId), NULL, &vgroupInfo, sizeof(SNewVgroupInfo)); + // vgroup info exists, compare with it + if (((vgroupInfo.inUse >= 0) && !vgroupInfoIdentical(&vgroupInfo, pVgroupMsg)) || (vgroupInfo.inUse < 0)) { + vgroupInfo = createNewVgroupInfo(pVgroupMsg); + taosHashPut(tscVgroupMap, &vgId, sizeof(vgId), &vgroupInfo, sizeof(vgroupInfo)); + tscDebug("add new VgroupInfo, vgId:%d, total cached:%d", vgId, (int32_t) taosHashGetSize(tscVgroupMap)); + } + } +} + +static void doAddTableMetaToLocalBuf(STableMeta* pTableMeta, STableMetaMsg* pMetaMsg, bool updateSTable) { if (pTableMeta->tableType == TSDB_CHILD_TABLE) { - // check if super table hashmap or not + // add or update the corresponding super table meta data info int32_t len = (int32_t) strnlen(pTableMeta->sTableName, TSDB_TABLE_FNAME_LEN); - // super tableMeta data alreay exists, create it according to tableMeta and add it to hash map + // The super tableMeta already exists, create it according to tableMeta and add it to hash map STableMeta* pSupTableMeta = createSuperTableMeta(pMetaMsg); uint32_t size = tscGetTableMetaSize(pSupTableMeta); @@ -1897,147 +1871,176 @@ int tscProcessTableMetaRsp(SSqlObj *pSql) { tfree(pSupTableMeta); CChildTableMeta* cMeta = tscCreateChildMeta(pTableMeta); - - char name[TSDB_TABLE_FNAME_LEN] = {0}; - tNameExtractFullName(&pTableMetaInfo->name, name); - - taosHashPut(tscTableMetaInfo, name, strlen(name), cMeta, sizeof(CChildTableMeta)); + taosHashPut(tscTableMetaInfo, pMetaMsg->tableFname, strlen(pMetaMsg->tableFname), cMeta, sizeof(CChildTableMeta)); tfree(cMeta); } else { uint32_t s = tscGetTableMetaSize(pTableMeta); - - char name[TSDB_TABLE_FNAME_LEN] = {0}; - tNameExtractFullName(&pTableMetaInfo->name, name); - - taosHashPut(tscTableMetaInfo, name, strlen(name), pTableMeta, s); + taosHashPut(tscTableMetaInfo, pMetaMsg->tableFname, strlen(pMetaMsg->tableFname), pTableMeta, s); } +} - // update the vgroupInfo if needed - if (pTableMeta->vgId > 0) { - int32_t vgId = pTableMeta->vgId; - assert(pTableMeta->tableType != TSDB_SUPER_TABLE); +int tscProcessTableMetaRsp(SSqlObj *pSql) { + STableMetaMsg *pMetaMsg = (STableMetaMsg *)pSql->res.pRsp; + int32_t code = tableMetaMsgConvert(pMetaMsg); + if (code != TSDB_CODE_SUCCESS) { + return code; + } - SNewVgroupInfo vgroupInfo = {.inUse = -1}; - taosHashGetClone(tscVgroupMap, &vgId, sizeof(vgId), NULL, &vgroupInfo, sizeof(SNewVgroupInfo)); + STableMetaInfo *pTableMetaInfo = tscGetTableMetaInfoFromCmd(&pSql->cmd, 0); + assert(pTableMetaInfo->pTableMeta == NULL); - if (((vgroupInfo.inUse >= 0) && !vgroupInfoIdentical(&vgroupInfo, &pMetaMsg->vgroup)) || - (vgroupInfo.inUse < 0)) { // vgroup info exists, compare with it - vgroupInfo = createNewVgroupInfo(&pMetaMsg->vgroup); - taosHashPut(tscVgroupMap, &vgId, sizeof(vgId), &vgroupInfo, sizeof(vgroupInfo)); - tscDebug("add new VgroupInfo, vgId:%d, total cached:%d", vgId, (int32_t) taosHashGetSize(tscVgroupMap)); - } + STableMeta* pTableMeta = tscCreateTableMetaFromMsg(pMetaMsg); + if (!tIsValidSchema(pTableMeta->schema, pTableMeta->tableInfo.numOfColumns, pTableMeta->tableInfo.numOfTags)) { + tscError("0x%"PRIx64" invalid table meta from mnode, name:%s", pSql->self, tNameGetTableName(&pTableMetaInfo->name)); + return TSDB_CODE_TSC_INVALID_VALUE; } - tscDebug("0x%"PRIx64" recv table meta, uid:%" PRIu64 ", tid:%d, name:%s", pSql->self, pTableMeta->id.uid, pTableMeta->id.tid, - tNameGetTableName(&pTableMetaInfo->name)); + char name[TSDB_TABLE_FNAME_LEN] = {0}; + tNameExtractFullName(&pTableMetaInfo->name, name); + assert(strncmp(pMetaMsg->tableFname, name, tListLen(pMetaMsg->tableFname)) == 0); + + doAddTableMetaToLocalBuf(pTableMeta, pMetaMsg, true); + doUpdateVgroupInfo(pTableMeta, &pMetaMsg->vgroup); + + tscDebug("0x%"PRIx64" recv table meta, uid:%" PRIu64 ", tid:%d, name:%s, numOfCols:%d, numOfTags:%d", pSql->self, + pTableMeta->id.uid, pTableMeta->id.tid, tNameGetTableName(&pTableMetaInfo->name), pTableMeta->tableInfo.numOfColumns, + pTableMeta->tableInfo.numOfTags); free(pTableMeta); return TSDB_CODE_SUCCESS; } -/** - * multi table meta rsp pkg format: - * | STaosRsp | SMultiTableInfoMsg | SMeterMeta0 | SSchema0 | SMeterMeta1 | SSchema1 | SMeterMeta2 | SSchema2 - * |...... 1B 4B - **/ -int tscProcessMultiMeterMetaRsp(SSqlObj *pSql) { -#if 0 +static SVgroupsInfo* createVgroupInfoFromMsg(char* pMsg, int32_t* size, uint64_t id) { + SVgroupsMsg *pVgroupMsg = (SVgroupsMsg *)pMsg; + pVgroupMsg->numOfVgroups = htonl(pVgroupMsg->numOfVgroups); + + *size = (int32_t)(sizeof(SVgroupMsg) * pVgroupMsg->numOfVgroups + sizeof(SVgroupsMsg)); + + size_t vgroupsz = sizeof(SVgroupInfo) * pVgroupMsg->numOfVgroups + sizeof(SVgroupsInfo); + SVgroupsInfo *pVgroupInfo = calloc(1, vgroupsz); + assert(pVgroupInfo != NULL); + + pVgroupInfo->numOfVgroups = pVgroupMsg->numOfVgroups; + if (pVgroupInfo->numOfVgroups <= 0) { + tscDebug("0x%" PRIx64 " empty vgroup info, no corresponding tables for stable", id); + } else { + for (int32_t j = 0; j < pVgroupInfo->numOfVgroups; ++j) { + // just init, no need to lock + SVgroupInfo *pVgroup = &pVgroupInfo->vgroups[j]; + + SVgroupMsg *vmsg = &pVgroupMsg->vgroups[j]; + vmsg->vgId = htonl(vmsg->vgId); + vmsg->numOfEps = vmsg->numOfEps; + for (int32_t k = 0; k < vmsg->numOfEps; ++k) { + vmsg->epAddr[k].port = htons(vmsg->epAddr[k].port); + } + + SNewVgroupInfo newVi = createNewVgroupInfo(vmsg); + pVgroup->numOfEps = newVi.numOfEps; + pVgroup->vgId = newVi.vgId; + for (int32_t k = 0; k < vmsg->numOfEps; ++k) { + pVgroup->epAddr[k].port = newVi.ep[k].port; + pVgroup->epAddr[k].fqdn = strndup(newVi.ep[k].fqdn, TSDB_FQDN_LEN); + } + + // check if current buffer contains the vgroup info. + // If not, add it + SNewVgroupInfo existVgroupInfo = {.inUse = -1}; + taosHashGetClone(tscVgroupMap, &newVi.vgId, sizeof(newVi.vgId), NULL, &existVgroupInfo, sizeof(SNewVgroupInfo)); + + if (((existVgroupInfo.inUse >= 0) && !vgroupInfoIdentical(&existVgroupInfo, vmsg)) || + (existVgroupInfo.inUse < 0)) { // vgroup info exists, compare with it + taosHashPut(tscVgroupMap, &newVi.vgId, sizeof(newVi.vgId), &newVi, sizeof(newVi)); + tscDebug("0x%" PRIx64 " add new VgroupInfo, vgId:%d, total cached:%d", id, newVi.vgId, (int32_t)taosHashGetSize(tscVgroupMap)); + } + } + } + + return pVgroupInfo; +} + +int tscProcessMultiTableMetaRsp(SSqlObj *pSql) { char *rsp = pSql->res.pRsp; - ieType = *rsp; - if (ieType != TSDB_IE_TYPE_META) { - tscError("invalid ie type:%d", ieType); - pSql->res.code = TSDB_CODE_TSC_INVALID_IE; - pSql->res.numOfTotal = 0; - return TSDB_CODE_TSC_APP_ERROR; + SMultiTableMeta *pMultiMeta = (SMultiTableMeta *)rsp; + pMultiMeta->numOfTables = htonl(pMultiMeta->numOfTables); + pMultiMeta->numOfVgroup = htonl(pMultiMeta->numOfVgroup); + + rsp += sizeof(SMultiTableMeta); + + SSqlObj* pParentSql = (SSqlObj*)taosAcquireRef(tscObjRef, (int64_t)pSql->param); + if(pParentSql == NULL) { + return pSql->res.code; } - rsp++; + SSqlCmd *pParentCmd = &pParentSql->cmd; - SMultiTableInfoMsg *pInfo = (SMultiTableInfoMsg *)rsp; - totalNum = htonl(pInfo->numOfTables); - rsp += sizeof(SMultiTableInfoMsg); + SHashObj *pSet = taosHashInit(4, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BIGINT), false, HASH_NO_LOCK); - for (i = 0; i < totalNum; i++) { - SMultiTableMeta *pMultiMeta = (SMultiTableMeta *)rsp; - STableMeta * pMeta = pMultiMeta->metas; + char* pMsg = pMultiMeta->meta; + for (int32_t i = 0; i < pMultiMeta->numOfTables; i++) { + STableMetaMsg *pMetaMsg = (STableMetaMsg *)pMsg; + int32_t code = tableMetaMsgConvert(pMetaMsg); + if (code != TSDB_CODE_SUCCESS) { + taosHashCleanup(pSet); + taosReleaseRef(tscObjRef, pParentSql->self); + return code; + } + + STableMeta* pTableMeta = tscCreateTableMetaFromMsg(pMetaMsg); + if (!tIsValidSchema(pTableMeta->schema, pTableMeta->tableInfo.numOfColumns, pTableMeta->tableInfo.numOfTags)) { + tscError("0x%"PRIx64" invalid table meta from mnode, name:%s", pSql->self, pMetaMsg->tableFname); + taosHashCleanup(pSet); + taosReleaseRef(tscObjRef, pParentSql->self); + return TSDB_CODE_TSC_INVALID_VALUE; + } - pMeta->sid = htonl(pMeta->sid); - pMeta->sversion = htons(pMeta->sversion); - pMeta->vgId = htonl(pMeta->vgId); - pMeta->uid = htobe64(pMeta->uid); + SName sn = {0}; + tNameFromString(&sn, pMetaMsg->tableFname, T_NAME_ACCT | T_NAME_DB | T_NAME_TABLE); - if (pMeta->sid <= 0 || pMeta->vgId < 0) { - tscError("invalid meter vgId:%d, sid%d", pMeta->vgId, pMeta->sid); - pSql->res.code = TSDB_CODE_TSC_INVALID_VALUE; - pSql->res.numOfTotal = i; - return TSDB_CODE_TSC_APP_ERROR; + const char* tableName = tNameGetTableName(&sn); + size_t keyLen = strlen(tableName); + + STableMetaVgroupInfo p = {.pTableMeta = pTableMeta,}; + taosHashPut(pParentCmd->pTableMetaMap, tableName, keyLen, &p, sizeof(STableMetaVgroupInfo)); + + bool addToBuf = false; + if (taosHashGet(pSet, &pMetaMsg->uid, sizeof(pMetaMsg->uid)) == NULL) { + addToBuf = true; + taosHashPut(pSet, &pMetaMsg->uid, sizeof(pMetaMsg->uid), "", 0); + } + + // create the tableMeta and add it into the TableMeta map + doAddTableMetaToLocalBuf(pTableMeta, pMetaMsg, addToBuf); + + // if the vgroup is not updated in current process, update it. + int64_t vgId = pMetaMsg->vgroup.vgId; + if (pTableMeta->tableType != TSDB_SUPER_TABLE && taosHashGet(pSet, &vgId, sizeof(vgId)) == NULL) { + doUpdateVgroupInfo(pTableMeta, &pMetaMsg->vgroup); + taosHashPut(pSet, &vgId, sizeof(vgId), "", 0); } - // pMeta->numOfColumns = htons(pMeta->numOfColumns); - // - // if (pMeta->numOfTags > TSDB_MAX_TAGS || pMeta->numOfTags < 0) { - // tscError("invalid tag value count:%d", pMeta->numOfTags); - // pSql->res.code = TSDB_CODE_TSC_INVALID_VALUE; - // pSql->res.numOfTotal = i; - // return TSDB_CODE_TSC_APP_ERROR; - // } - // - // if (pMeta->numOfTags > TSDB_MAX_TAGS || pMeta->numOfTags < 0) { - // tscError("invalid numOfTags:%d", pMeta->numOfTags); - // pSql->res.code = TSDB_CODE_TSC_INVALID_VALUE; - // pSql->res.numOfTotal = i; - // return TSDB_CODE_TSC_APP_ERROR; - // } - // - // if (pMeta->numOfColumns > TSDB_MAX_COLUMNS || pMeta->numOfColumns < 0) { - // tscError("invalid numOfColumns:%d", pMeta->numOfColumns); - // pSql->res.code = TSDB_CODE_TSC_INVALID_VALUE; - // pSql->res.numOfTotal = i; - // return TSDB_CODE_TSC_APP_ERROR; - // } - // - // for (int j = 0; j < TSDB_REPLICA_MAX_NUM; ++j) { - // pMeta->vpeerDesc[j].vnode = htonl(pMeta->vpeerDesc[j].vnode); - // } - // - // pMeta->rowSize = 0; - // rsp += sizeof(SMultiTableMeta); - // pSchema = (SSchema *)rsp; - // - // int32_t numOfTotalCols = pMeta->numOfColumns + pMeta->numOfTags; - // for (int j = 0; j < numOfTotalCols; ++j) { - // pSchema->bytes = htons(pSchema->bytes); - // pSchema->colId = htons(pSchema->colId); - // - // // ignore the tags length - // if (j < pMeta->numOfColumns) { - // pMeta->rowSize += pSchema->bytes; - // } - // pSchema++; - // } - // - // rsp += numOfTotalCols * sizeof(SSchema); - // - // int32_t tagLen = 0; - // SSchema *pTagsSchema = tscGetTableTagSchema(pMeta); - // - // if (pMeta->tableType == TSDB_CHILD_TABLE) { - // for (int32_t j = 0; j < pMeta->numOfTags; ++j) { - // tagLen += pTagsSchema[j].bytes; - // } - // } - // - // rsp += tagLen; - // int32_t size = (int32_t)(rsp - ((char *)pMeta)); // Consistent with STableMeta in cache - // } + pMsg += pMetaMsg->contLen; } - + + for(int32_t i = 0; i < pMultiMeta->numOfVgroup; ++i) { + char* name = pMsg; + pMsg += TSDB_TABLE_NAME_LEN; + + STableMetaVgroupInfo* p = taosHashGet(pParentCmd->pTableMetaMap, name, strnlen(name, TSDB_TABLE_NAME_LEN)); + assert(p != NULL); + + int32_t size = 0; + p->pVgroupInfo = createVgroupInfoFromMsg(pMsg, &size, pSql->self); + pMsg += size; + } + pSql->res.code = TSDB_CODE_SUCCESS; - pSql->res.numOfTotal = i; - tscDebug("0x%"PRIx64" load multi-metermeta resp from complete num:%d", pSql->self, pSql->res.numOfTotal); -#endif - + pSql->res.numOfTotal = pMultiMeta->numOfTables; + tscDebug("0x%"PRIx64" load multi-tableMeta from mnode, numOfTables:%d", pSql->self, pMultiMeta->numOfTables); + + taosHashCleanup(pSet); + taosReleaseRef(tscObjRef, pParentSql->self); return TSDB_CODE_SUCCESS; } @@ -2049,68 +2052,38 @@ int tscProcessSTableVgroupRsp(SSqlObj *pSql) { } assert(parent->signature == parent && (int64_t)pSql->param == parent->self); - + SSqlRes* pRes = &pSql->res; - + // NOTE: the order of several table must be preserved. SSTableVgroupRspMsg *pStableVgroup = (SSTableVgroupRspMsg *)pRes->pRsp; pStableVgroup->numOfTables = htonl(pStableVgroup->numOfTables); char *pMsg = pRes->pRsp + sizeof(SSTableVgroupRspMsg); - - SSqlCmd* pCmd = &parent->cmd; - for(int32_t i = 0; i < pStableVgroup->numOfTables; ++i) { - STableMetaInfo *pInfo = tscGetTableMetaInfoFromCmd(pCmd, pCmd->clauseIndex, i); - - SVgroupsMsg * pVgroupMsg = (SVgroupsMsg *) pMsg; - pVgroupMsg->numOfVgroups = htonl(pVgroupMsg->numOfVgroups); - - size_t size = sizeof(SVgroupMsg) * pVgroupMsg->numOfVgroups + sizeof(SVgroupsMsg); - size_t vgroupsz = sizeof(SVgroupInfo) * pVgroupMsg->numOfVgroups + sizeof(SVgroupsInfo); - pInfo->vgroupList = calloc(1, vgroupsz); - assert(pInfo->vgroupList != NULL); + SSqlCmd* pCmd = &parent->cmd; + SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd); - pInfo->vgroupList->numOfVgroups = pVgroupMsg->numOfVgroups; - if (pInfo->vgroupList->numOfVgroups <= 0) { - tscDebug("0x%"PRIx64" empty vgroup info, no corresponding tables for stable", pSql->self); - } else { - for (int32_t j = 0; j < pInfo->vgroupList->numOfVgroups; ++j) { - // just init, no need to lock - SVgroupInfo *pVgroup = &pInfo->vgroupList->vgroups[j]; - - SVgroupMsg *vmsg = &pVgroupMsg->vgroups[j]; - vmsg->vgId = htonl(vmsg->vgId); - vmsg->numOfEps = vmsg->numOfEps; - for (int32_t k = 0; k < vmsg->numOfEps; ++k) { - vmsg->epAddr[k].port = htons(vmsg->epAddr[k].port); - } - - SNewVgroupInfo newVi = createNewVgroupInfo(vmsg); - pVgroup->numOfEps = newVi.numOfEps; - pVgroup->vgId = newVi.vgId; - for (int32_t k = 0; k < vmsg->numOfEps; ++k) { - pVgroup->epAddr[k].port = newVi.ep[k].port; - pVgroup->epAddr[k].fqdn = strndup(newVi.ep[k].fqdn, TSDB_FQDN_LEN); - } - - // check if current buffer contains the vgroup info. - // If not, add it - SNewVgroupInfo existVgroupInfo = {.inUse = -1}; - taosHashGetClone(tscVgroupMap, &newVi.vgId, sizeof(newVi.vgId), NULL, &existVgroupInfo, sizeof(SNewVgroupInfo)); - - if (((existVgroupInfo.inUse >= 0) && !vgroupInfoIdentical(&existVgroupInfo, vmsg)) || - (existVgroupInfo.inUse < 0)) { // vgroup info exists, compare with it - taosHashPut(tscVgroupMap, &newVi.vgId, sizeof(newVi.vgId), &newVi, sizeof(newVi)); - tscDebug("add new VgroupInfo, vgId:%d, total cached:%d", newVi.vgId, (int32_t) taosHashGetSize(tscVgroupMap)); - } + for(int32_t i = 0; i < pStableVgroup->numOfTables; ++i) { + char* name = pMsg; + pMsg += TSDB_TABLE_NAME_LEN; + + STableMetaInfo *pInfo = NULL; + for(int32_t j = 0; j < pQueryInfo->numOfTables; ++j) { + STableMetaInfo *pInfo1 = tscGetTableMetaInfoFromCmd(pCmd, j); + if (strcmp(name, tNameGetTableName(&pInfo1->name)) != 0) { + continue; } + + pInfo = pInfo1; + break; } + int32_t size = 0; + pInfo->vgroupList = createVgroupInfoFromMsg(pMsg, &size, pSql->self); pMsg += size; } taosReleaseRef(tscObjRef, parent->self); - return pSql->res.code; } @@ -2122,7 +2095,7 @@ int tscProcessShowRsp(SSqlObj *pSql) { SSqlRes *pRes = &pSql->res; SSqlCmd *pCmd = &pSql->cmd; - SQueryInfo *pQueryInfo = tscGetQueryInfo(pCmd, 0); + SQueryInfo *pQueryInfo = tscGetQueryInfo(pCmd); STableMetaInfo *pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0); @@ -2163,8 +2136,8 @@ int tscProcessShowRsp(SSqlObj *pSql) { TAOS_FIELD f = tscCreateField(pSchema->type, pSchema->name, pSchema->bytes); SInternalField* pInfo = tscFieldInfoAppend(pFieldInfo, &f); - pInfo->pExpr = tscSqlExprAppend(pQueryInfo, TSDB_FUNC_TS_DUMMY, &index, - pTableSchema[i].type, pTableSchema[i].bytes, getNewResColId(pQueryInfo), pTableSchema[i].bytes, false); + pInfo->pExpr = tscExprAppend(pQueryInfo, TSDB_FUNC_TS_DUMMY, &index, + pTableSchema[i].type, pTableSchema[i].bytes, getNewResColId(pCmd), pTableSchema[i].bytes, false); } pCmd->numOfCols = pQueryInfo->fieldsInfo.numOfOutput; @@ -2182,7 +2155,7 @@ static void createHbObj(STscObj* pObj) { pSql->fp = tscProcessHeartBeatRsp; - SQueryInfo *pQueryInfo = tscGetQueryInfoS(&pSql->cmd, 0); + SQueryInfo *pQueryInfo = tscGetQueryInfoS(&pSql->cmd); if (pQueryInfo == NULL) { terrno = TSDB_CODE_TSC_OUT_OF_MEMORY; tfree(pSql); @@ -2248,7 +2221,7 @@ int tscProcessConnectRsp(SSqlObj *pSql) { int tscProcessUseDbRsp(SSqlObj *pSql) { STscObj * pObj = pSql->pTscObj; - STableMetaInfo *pTableMetaInfo = tscGetTableMetaInfoFromCmd(&pSql->cmd, 0, 0); + STableMetaInfo *pTableMetaInfo = tscGetTableMetaInfoFromCmd(&pSql->cmd, 0); pthread_mutex_lock(&pObj->mutex); int ret = tNameExtractFullName(&pTableMetaInfo->name, pObj->db); @@ -2266,7 +2239,7 @@ int tscProcessDropDbRsp(SSqlObj *pSql) { } int tscProcessDropTableRsp(SSqlObj *pSql) { - STableMetaInfo *pTableMetaInfo = tscGetTableMetaInfoFromCmd(&pSql->cmd, 0, 0); + STableMetaInfo *pTableMetaInfo = tscGetTableMetaInfoFromCmd(&pSql->cmd, 0); //The cached tableMeta is expired in this case, so clean it in hash table char name[TSDB_TABLE_FNAME_LEN] = {0}; @@ -2280,7 +2253,7 @@ int tscProcessDropTableRsp(SSqlObj *pSql) { } int tscProcessAlterTableMsgRsp(SSqlObj *pSql) { - STableMetaInfo *pTableMetaInfo = tscGetTableMetaInfoFromCmd(&pSql->cmd, 0, 0); + STableMetaInfo *pTableMetaInfo = tscGetTableMetaInfoFromCmd(&pSql->cmd, 0); char name[TSDB_TABLE_FNAME_LEN] = {0}; tNameExtractFullName(&pTableMetaInfo->name, name); @@ -2340,7 +2313,7 @@ int tscProcessRetrieveRspFromNode(SSqlObj *pSql) { pRes->completed = (pRetrieve->completed == 1); pRes->data = pRetrieve->data; - SQueryInfo* pQueryInfo = tscGetActiveQueryInfo(pCmd); + SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd); if (tscCreateResPointerInfo(pRes, pQueryInfo) != TSDB_CODE_SUCCESS) { return pRes->code; } @@ -2354,8 +2327,6 @@ int tscProcessRetrieveRspFromNode(SSqlObj *pSql) { tscSetResRawPtr(pRes, pQueryInfo); } - handleDownstreamOperator(pRes, pQueryInfo); - if (pSql->pSubscription != NULL) { int32_t numOfCols = pQueryInfo->fieldsInfo.numOfOutput; @@ -2385,53 +2356,130 @@ int tscProcessRetrieveRspFromNode(SSqlObj *pSql) { void tscTableMetaCallBack(void *param, TAOS_RES *res, int code); -static int32_t getTableMetaFromMnode(SSqlObj *pSql, STableMetaInfo *pTableMetaInfo) { +static int32_t getTableMetaFromMnode(SSqlObj *pSql, STableMetaInfo *pTableMetaInfo, bool autocreate) { SSqlObj *pNew = calloc(1, sizeof(SSqlObj)); if (NULL == pNew) { tscError("0x%"PRIx64" malloc failed for new sqlobj to get table meta", pSql->self); return TSDB_CODE_TSC_OUT_OF_MEMORY; } - pNew->pTscObj = pSql->pTscObj; - pNew->signature = pNew; + pNew->pTscObj = pSql->pTscObj; + pNew->signature = pNew; pNew->cmd.command = TSDB_SQL_META; tscAddQueryInfo(&pNew->cmd); - SQueryInfo *pNewQueryInfo = tscGetQueryInfoS(&pNew->cmd, 0); - - pNew->cmd.autoCreated = pSql->cmd.autoCreated; // create table if not exists + SQueryInfo *pNewQueryInfo = tscGetQueryInfoS(&pNew->cmd); if (TSDB_CODE_SUCCESS != tscAllocPayload(&pNew->cmd, TSDB_DEFAULT_PAYLOAD_SIZE + pSql->cmd.payloadLen)) { tscError("0x%"PRIx64" malloc failed for payload to get table meta", pSql->self); + tscFreeSqlObj(pNew); return TSDB_CODE_TSC_OUT_OF_MEMORY; } - STableMetaInfo *pNewMeterMetaInfo = tscAddEmptyMetaInfo(pNewQueryInfo); - assert(pNew->cmd.numOfClause == 1 && pNewQueryInfo->numOfTables == 1); + STableMetaInfo *pNewTableMetaInfo = tscAddEmptyMetaInfo(pNewQueryInfo); + assert(pNewQueryInfo->numOfTables == 1); + + tNameAssign(&pNewTableMetaInfo->name, &pTableMetaInfo->name); - tNameAssign(&pNewMeterMetaInfo->name, &pTableMetaInfo->name); + registerSqlObj(pNew); + + pNew->fp = tscTableMetaCallBack; + pNew->param = (void *)pSql->self; - if (pSql->cmd.autoCreated) { - int32_t code = copyTagData(&pNew->cmd.tagData, &pSql->cmd.tagData); + tscDebug("0x%"PRIx64" new pSqlObj:0x%"PRIx64" to get tableMeta, auto create:%d, metaRid from %"PRId64" to %"PRId64, + pSql->self, pNew->self, autocreate, pSql->metaRid, pNew->self); + pSql->metaRid = pNew->self; + + { + STableInfoMsg *pInfoMsg = (STableInfoMsg *)pNew->cmd.payload; + int32_t code = tNameExtractFullName(&pNewTableMetaInfo->name, pInfoMsg->tableFname); if (code != TSDB_CODE_SUCCESS) { - tscError("0x%"PRIx64" malloc failed for new tag data to get table meta", pSql->self); - tscFreeSqlObj(pNew); - return TSDB_CODE_TSC_OUT_OF_MEMORY; + return TSDB_CODE_TSC_INVALID_OPERATION; } + + pInfoMsg->createFlag = htons(autocreate? 1 : 0); + char *pMsg = (char *)pInfoMsg + sizeof(STableInfoMsg); + + // tag data exists + if (autocreate && pSql->cmd.tagData.dataLen != 0) { + pMsg = serializeTagData(&pSql->cmd.tagData, pMsg); + } + + pNew->cmd.payloadLen = (int32_t)(pMsg - (char*)pInfoMsg); + pNew->cmd.msgType = TSDB_MSG_TYPE_CM_TABLE_META; + } + + int32_t code = tscBuildAndSendRequest(pNew, NULL); + if (code == TSDB_CODE_SUCCESS) { + code = TSDB_CODE_TSC_ACTION_IN_PROGRESS; // notify application that current process needs to be terminated + } + + return code; +} + +int32_t getMultiTableMetaFromMnode(SSqlObj *pSql, SArray* pNameList, SArray* pVgroupNameList, __async_cb_func_t fp) { + SSqlObj *pNew = calloc(1, sizeof(SSqlObj)); + if (NULL == pNew) { + tscError("0x%"PRIx64" failed to allocate sqlobj to get multiple table meta", pSql->self); + return TSDB_CODE_TSC_OUT_OF_MEMORY; + } + + pNew->pTscObj = pSql->pTscObj; + pNew->signature = pNew; + pNew->cmd.command = TSDB_SQL_MULTI_META; + + int32_t numOfTable = (int32_t) taosArrayGetSize(pNameList); + int32_t numOfVgroupList = (int32_t) taosArrayGetSize(pVgroupNameList); + + int32_t size = (numOfTable + numOfVgroupList) * TSDB_TABLE_FNAME_LEN + sizeof(SMultiTableInfoMsg); + if (TSDB_CODE_SUCCESS != tscAllocPayload(&pNew->cmd, size)) { + tscError("0x%"PRIx64" malloc failed for payload to get table meta", pSql->self); + tscFreeSqlObj(pNew); + return TSDB_CODE_TSC_OUT_OF_MEMORY; } + SMultiTableInfoMsg* pInfo = (SMultiTableInfoMsg*) pNew->cmd.payload; + pInfo->numOfTables = htonl((uint32_t) taosArrayGetSize(pNameList)); + pInfo->numOfVgroups = htonl((uint32_t) taosArrayGetSize(pVgroupNameList)); + + char* start = pInfo->tableNames; + int32_t len = 0; + for(int32_t i = 0; i < numOfTable; ++i) { + char* name = taosArrayGetP(pNameList, i); + if (i < numOfTable - 1 || numOfVgroupList > 0) { + len = sprintf(start, "%s,", name); + } else { + len = sprintf(start, "%s", name); + } + + start += len; + } + + for(int32_t i = 0; i < numOfVgroupList; ++i) { + char* name = taosArrayGetP(pVgroupNameList, i); + if (i < numOfVgroupList - 1) { + len = sprintf(start, "%s,", name); + } else { + len = sprintf(start, "%s", name); + } + + start += len; + } + + pNew->cmd.payloadLen = (int32_t) ((start - pInfo->tableNames) + sizeof(SMultiTableInfoMsg)); + pNew->cmd.msgType = TSDB_MSG_TYPE_CM_TABLES_META; + registerSqlObj(pNew); - tscDebug("0x%"PRIx64" new pSqlObj:0x%"PRIx64" to get tableMeta, auto create:%d", pSql->self, pNew->self, - pNew->cmd.autoCreated); + tscDebug("0x%"PRIx64" new pSqlObj:0x%"PRIx64" to get %d tableMeta, vgroupInfo:%d, msg size:%d", pSql->self, + pNew->self, numOfTable, numOfVgroupList, pNew->cmd.payloadLen); - pNew->fp = tscTableMetaCallBack; + pNew->fp = fp; pNew->param = (void *)pSql->self; tscDebug("0x%"PRIx64" metaRid from %" PRId64 " to %" PRId64 , pSql->self, pSql->metaRid, pNew->self); - - pSql->metaRid = pNew->self; + pSql->metaRid = pNew->self; int32_t code = tscBuildAndSendRequest(pNew, NULL); if (code == TSDB_CODE_SUCCESS) { code = TSDB_CODE_TSC_ACTION_IN_PROGRESS; // notify application that current process needs to be terminated @@ -2440,7 +2488,7 @@ static int32_t getTableMetaFromMnode(SSqlObj *pSql, STableMetaInfo *pTableMetaIn return code; } -int32_t tscGetTableMeta(SSqlObj *pSql, STableMetaInfo *pTableMetaInfo) { +int32_t tscGetTableMetaImpl(SSqlObj* pSql, STableMetaInfo *pTableMetaInfo, bool autocreate) { assert(tIsValidName(&pTableMetaInfo->name)); uint32_t size = tscGetTableMetaMaxSize(); @@ -2477,23 +2525,26 @@ int32_t tscGetTableMeta(SSqlObj *pSql, STableMetaInfo *pTableMetaInfo) { if (pMeta->tableType == TSDB_CHILD_TABLE) { int32_t code = tscCreateTableMetaFromSTableMeta(pTableMetaInfo->pTableMeta, name, buf); if (code != TSDB_CODE_SUCCESS) { - return getTableMetaFromMnode(pSql, pTableMetaInfo); + return getTableMetaFromMnode(pSql, pTableMetaInfo, autocreate); } } return TSDB_CODE_SUCCESS; } - return getTableMetaFromMnode(pSql, pTableMetaInfo); + return getTableMetaFromMnode(pSql, pTableMetaInfo, autocreate); +} + +int32_t tscGetTableMeta(SSqlObj *pSql, STableMetaInfo *pTableMetaInfo) { + return tscGetTableMetaImpl(pSql, pTableMetaInfo, false); } int tscGetTableMetaEx(SSqlObj *pSql, STableMetaInfo *pTableMetaInfo, bool createIfNotExists) { - pSql->cmd.autoCreated = createIfNotExists; - return tscGetTableMeta(pSql, pTableMetaInfo); + return tscGetTableMetaImpl(pSql, pTableMetaInfo, createIfNotExists); } /** - * retrieve table meta from mnode, and update the local table meta hashmap. + * retrieve table meta from mnode, and then update the local table meta hashmap. * @param pSql sql object * @param tableIndex table index * @return status code @@ -2501,14 +2552,14 @@ int tscGetTableMetaEx(SSqlObj *pSql, STableMetaInfo *pTableMetaInfo, bool create int tscRenewTableMeta(SSqlObj *pSql, int32_t tableIndex) { SSqlCmd *pCmd = &pSql->cmd; - SQueryInfo *pQueryInfo = tscGetQueryInfo(pCmd, 0); + SQueryInfo *pQueryInfo = tscGetQueryInfo(pCmd); STableMetaInfo *pTableMetaInfo = tscGetMetaInfo(pQueryInfo, tableIndex); char name[TSDB_TABLE_FNAME_LEN] = {0}; int32_t code = tNameExtractFullName(&pTableMetaInfo->name, name); if (code != TSDB_CODE_SUCCESS) { tscError("0x%"PRIx64" failed to generate the table full name", pSql->self); - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } STableMeta* pTableMeta = pTableMetaInfo->pTableMeta; @@ -2521,11 +2572,10 @@ int tscRenewTableMeta(SSqlObj *pSql, int32_t tableIndex) { size_t len = strlen(name); taosHashRemove(tscTableMetaInfo, name, len); - return getTableMetaFromMnode(pSql, pTableMetaInfo); + return getTableMetaFromMnode(pSql, pTableMetaInfo, false); } -static bool allVgroupInfoRetrieved(SSqlCmd* pCmd, int32_t clauseIndex) { - SQueryInfo *pQueryInfo = tscGetQueryInfo(pCmd, clauseIndex); +static bool allVgroupInfoRetrieved(SQueryInfo* pQueryInfo) { for (int32_t i = 0; i < pQueryInfo->numOfTables; ++i) { STableMetaInfo *pTableMetaInfo = tscGetMetaInfo(pQueryInfo, i); if (pTableMetaInfo->vgroupList == NULL) { @@ -2537,11 +2587,9 @@ static bool allVgroupInfoRetrieved(SSqlCmd* pCmd, int32_t clauseIndex) { return true; } -int tscGetSTableVgroupInfo(SSqlObj *pSql, int32_t clauseIndex) { - int code = TSDB_CODE_RPC_NETWORK_UNAVAIL; - SSqlCmd *pCmd = &pSql->cmd; - - if (allVgroupInfoRetrieved(pCmd, clauseIndex)) { +int tscGetSTableVgroupInfo(SSqlObj *pSql, SQueryInfo* pQueryInfo) { + int32_t code = TSDB_CODE_RPC_NETWORK_UNAVAIL; + if (allVgroupInfoRetrieved(pQueryInfo)) { return TSDB_CODE_SUCCESS; } @@ -2552,13 +2600,12 @@ int tscGetSTableVgroupInfo(SSqlObj *pSql, int32_t clauseIndex) { pNew->cmd.command = TSDB_SQL_STABLEVGROUP; // TODO TEST IT - SQueryInfo *pNewQueryInfo = tscGetQueryInfoS(&pNew->cmd, 0); + SQueryInfo *pNewQueryInfo = tscGetQueryInfoS(&pNew->cmd); if (pNewQueryInfo == NULL) { tscFreeSqlObj(pNew); return code; } - - SQueryInfo *pQueryInfo = tscGetQueryInfo(pCmd, clauseIndex); + for (int32_t i = 0; i < pQueryInfo->numOfTables; ++i) { STableMetaInfo *pMInfo = tscGetMetaInfo(pQueryInfo, i); STableMeta* pTableMeta = tscTableMetaDup(pMInfo->pTableMeta); @@ -2617,9 +2664,8 @@ void tscInitMsgsFp() { tscBuildMsg[TSDB_SQL_CONNECT] = tscBuildConnectMsg; tscBuildMsg[TSDB_SQL_USE_DB] = tscBuildUseDbMsg; - tscBuildMsg[TSDB_SQL_META] = tscBuildTableMetaMsg; +// tscBuildMsg[TSDB_SQL_META] = tscBuildTableMetaMsg; tscBuildMsg[TSDB_SQL_STABLEVGROUP] = tscBuildSTableVgroupMsg; - tscBuildMsg[TSDB_SQL_MULTI_META] = tscBuildMultiMeterMetaMsg; tscBuildMsg[TSDB_SQL_HB] = tscBuildHeartBeatMsg; tscBuildMsg[TSDB_SQL_SHOW] = tscBuildShowMsg; @@ -2637,7 +2683,7 @@ void tscInitMsgsFp() { tscProcessMsgRsp[TSDB_SQL_USE_DB] = tscProcessUseDbRsp; tscProcessMsgRsp[TSDB_SQL_META] = tscProcessTableMetaRsp; tscProcessMsgRsp[TSDB_SQL_STABLEVGROUP] = tscProcessSTableVgroupRsp; - tscProcessMsgRsp[TSDB_SQL_MULTI_META] = tscProcessMultiMeterMetaRsp; + tscProcessMsgRsp[TSDB_SQL_MULTI_META] = tscProcessMultiTableMetaRsp; tscProcessMsgRsp[TSDB_SQL_SHOW] = tscProcessShowRsp; tscProcessMsgRsp[TSDB_SQL_RETRIEVE] = tscProcessRetrieveRspFromNode; // rsp handled by same function. diff --git a/src/client/src/tscSql.c b/src/client/src/tscSql.c index e1a1ff7fd23a1da18cfb6982d602c49d9262a183..1ffa6416b221fdcaf7d8f1ba2a13d6cce14ca027 100644 --- a/src/client/src/tscSql.c +++ b/src/client/src/tscSql.c @@ -373,11 +373,15 @@ int taos_num_fields(TAOS_RES *res) { if (pSql == NULL || pSql->signature != pSql) return 0; int32_t num = 0; - SQueryInfo *pQueryInfo = tscGetQueryInfo(&pSql->cmd, 0); + SQueryInfo *pQueryInfo = tscGetQueryInfo(&pSql->cmd); if (pQueryInfo == NULL) { return num; } + while(pQueryInfo->pDownstream != NULL) { + pQueryInfo = pQueryInfo->pDownstream; + } + size_t numOfCols = tscNumOfFields(pQueryInfo); for(int32_t i = 0; i < numOfCols; ++i) { SInternalField* pInfo = taosArrayGet(pQueryInfo->fieldsInfo.internalField, i); @@ -408,7 +412,7 @@ TAOS_FIELD *taos_fetch_fields(TAOS_RES *res) { SSqlRes *pRes = &pSql->res; if (pSql == NULL || pSql->signature != pSql) return 0; - SQueryInfo *pQueryInfo = tscGetQueryInfo(&pSql->cmd, 0); + SQueryInfo *pQueryInfo = tscGetQueryInfo(&pSql->cmd); if (pQueryInfo == NULL) { return NULL; } @@ -560,7 +564,7 @@ static bool tscKillQueryInDnode(SSqlObj* pSql) { return true; } - SQueryInfo *pQueryInfo = tscGetQueryInfo(pCmd, 0); + SQueryInfo *pQueryInfo = tscGetQueryInfo(pCmd); if ((pQueryInfo == NULL) || tscIsTwoStageSTableQuery(pQueryInfo, 0)) { return true; @@ -614,7 +618,7 @@ int taos_errno(TAOS_RES *tres) { * why the sql is invalid */ static bool hasAdditionalErrorInfo(int32_t code, SSqlCmd *pCmd) { - if (code != TSDB_CODE_TSC_INVALID_SQL + if (code != TSDB_CODE_TSC_INVALID_OPERATION && code != TSDB_CODE_TSC_SQL_SYNTAX_ERROR) { return false; } @@ -673,7 +677,7 @@ char *taos_get_client_info() { return version; } static void tscKillSTableQuery(SSqlObj *pSql) { SSqlCmd* pCmd = &pSql->cmd; - SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd, pCmd->clauseIndex); + SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd); if (!tscIsTwoStageSTableQuery(pQueryInfo, 0)) { return; @@ -724,7 +728,7 @@ void taos_stop_query(TAOS_RES *res) { // set the error code for master pSqlObj firstly pSql->res.code = TSDB_CODE_TSC_QUERY_CANCELLED; - SQueryInfo *pQueryInfo = tscGetQueryInfo(pCmd, pCmd->clauseIndex); + SQueryInfo *pQueryInfo = tscGetQueryInfo(pCmd); if (tscIsTwoStageSTableQuery(pQueryInfo, 0)) { assert(pSql->rpcRid <= 0); @@ -754,7 +758,7 @@ bool taos_is_null(TAOS_RES *res, int32_t row, int32_t col) { return true; } - SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd, 0); + SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd); if (pQueryInfo == NULL) { return true; } @@ -829,9 +833,9 @@ int taos_print_row(char *str, TAOS_ROW row, TAOS_FIELD *fields, int num_fields) case TSDB_DATA_TYPE_NCHAR: { int32_t charLen = varDataLen((char*)row[i] - VARSTR_HEADER_SIZE); if (fields[i].type == TSDB_DATA_TYPE_BINARY) { - assert(charLen <= fields[i].bytes); + assert(charLen <= fields[i].bytes && charLen >= 0); } else { - assert(charLen <= fields[i].bytes * TSDB_NCHAR_SIZE); + assert(charLen <= fields[i].bytes * TSDB_NCHAR_SIZE && charLen >= 0); } memcpy(str + len, row[i], charLen); @@ -868,15 +872,11 @@ int taos_validate_sql(TAOS *taos, const char *sql) { SSqlObj* pSql = calloc(1, sizeof(SSqlObj)); - pSql->pTscObj = taos; + pSql->pTscObj = taos; pSql->signature = pSql; - - SSqlRes *pRes = &pSql->res; SSqlCmd *pCmd = &pSql->cmd; - pRes->numOfTotal = 0; - pRes->numOfClauseTotal = 0; - + pCmd->resColumnId = TSDB_RES_COL_ID; tscDebug("0x%"PRIx64" Valid SQL: %s pObj:%p", pSql->self, sql, pObj); @@ -896,10 +896,10 @@ int taos_validate_sql(TAOS *taos, const char *sql) { strtolower(pSql->sqlstr, sql); - pCmd->curSql = NULL; - if (NULL != pCmd->pTableBlockHashList) { - taosHashCleanup(pCmd->pTableBlockHashList); - pCmd->pTableBlockHashList = NULL; +// pCmd->curSql = NULL; + if (NULL != pCmd->insertParam.pTableBlockHashList) { + taosHashCleanup(pCmd->insertParam.pTableBlockHashList); + pCmd->insertParam.pTableBlockHashList = NULL; } pSql->fp = asyncCallback; @@ -921,90 +921,19 @@ int taos_validate_sql(TAOS *taos, const char *sql) { return code; } -static int tscParseTblNameList(SSqlObj *pSql, const char *tblNameList, int32_t tblListLen) { - // must before clean the sqlcmd object - tscResetSqlCmd(&pSql->cmd, false); - - SSqlCmd *pCmd = &pSql->cmd; - - pCmd->command = TSDB_SQL_MULTI_META; - pCmd->count = 0; - - int code = TSDB_CODE_TSC_INVALID_TABLE_ID_LENGTH; - char *str = (char *)tblNameList; - - SQueryInfo *pQueryInfo = tscGetQueryInfoS(pCmd, pCmd->clauseIndex); - if (pQueryInfo == NULL) { - pSql->res.code = terrno; - return terrno; - } - - STableMetaInfo *pTableMetaInfo = tscAddEmptyMetaInfo(pQueryInfo); - - if ((code = tscAllocPayload(pCmd, tblListLen + 16)) != TSDB_CODE_SUCCESS) { - return code; - } - - char *nextStr; - char tblName[TSDB_TABLE_FNAME_LEN]; - int payloadLen = 0; - char *pMsg = pCmd->payload; - while (1) { - nextStr = strchr(str, ','); - if (nextStr == NULL) { - break; - } - - memcpy(tblName, str, nextStr - str); - int32_t len = (int32_t)(nextStr - str); - tblName[len] = '\0'; - - str = nextStr + 1; - len = (int32_t)strtrim(tblName); - - SStrToken sToken = {.n = len, .type = TK_ID, .z = tblName}; - tGetToken(tblName, &sToken.type); - - // Check if the table name available or not - if (tscValidateName(&sToken) != TSDB_CODE_SUCCESS) { - code = TSDB_CODE_TSC_INVALID_TABLE_ID_LENGTH; - sprintf(pCmd->payload, "table name is invalid"); - return code; - } - - if ((code = tscSetTableFullName(pTableMetaInfo, &sToken, pSql)) != TSDB_CODE_SUCCESS) { - return code; - } - - if (++pCmd->count > TSDB_MULTI_TABLEMETA_MAX_NUM) { - code = TSDB_CODE_TSC_INVALID_TABLE_ID_LENGTH; - sprintf(pCmd->payload, "tables over the max number"); - return code; - } - - int32_t xlen = tNameLen(&pTableMetaInfo->name); - if (payloadLen + xlen + 128 >= pCmd->allocSize) { - char *pNewMem = realloc(pCmd->payload, pCmd->allocSize + tblListLen); - if (pNewMem == NULL) { - code = TSDB_CODE_TSC_OUT_OF_MEMORY; - sprintf(pCmd->payload, "failed to allocate memory"); - return code; - } - - pCmd->payload = pNewMem; - pCmd->allocSize = pCmd->allocSize + tblListLen; - pMsg = pCmd->payload; - } - - char n[TSDB_TABLE_FNAME_LEN] = {0}; - tNameExtractFullName(&pTableMetaInfo->name, n); - payloadLen += sprintf(pMsg + payloadLen, "%s,", n); +void loadMultiTableMetaCallback(void *param, TAOS_RES *res, int code) { + SSqlObj* pSql = (SSqlObj*)taosAcquireRef(tscObjRef, (int64_t)param); + if (pSql == NULL) { + return; } - *(pMsg + payloadLen) = '\0'; - pCmd->payloadLen = payloadLen + 1; + taosReleaseRef(tscObjRef, pSql->self); + pSql->res.code = code; + tsem_post(&pSql->rspSem); +} - return TSDB_CODE_SUCCESS; +static void freeElem(void* p) { + tfree(*(char**)p); } int taos_load_table_info(TAOS *taos, const char *tableNameList) { @@ -1020,38 +949,28 @@ int taos_load_table_info(TAOS *taos, const char *tableNameList) { pSql->pTscObj = taos; pSql->signature = pSql; - SSqlRes *pRes = &pSql->res; - - pRes->code = 0; - pRes->numOfTotal = 0; // the number of getting table meta from server - pRes->numOfClauseTotal = 0; - - assert(pSql->fp == NULL); - tscDebug("0x%"PRIx64" tableNameList: %s pObj:%p", pSql->self, tableNameList, pObj); + pSql->fp = NULL; // todo set the correct callback function pointer + pSql->cmd.pTableMetaMap = taosHashInit(4, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY), false, HASH_NO_LOCK); - int32_t tblListLen = (int32_t)strlen(tableNameList); - if (tblListLen > MAX_TABLE_NAME_LENGTH) { - tscError("0x%"PRIx64" tableNameList too long, length:%d, maximum allowed:%d", pSql->self, tblListLen, MAX_TABLE_NAME_LENGTH); + int32_t length = (int32_t)strlen(tableNameList); + if (length > MAX_TABLE_NAME_LENGTH) { + tscError("0x%"PRIx64" tableNameList too long, length:%d, maximum allowed:%d", pSql->self, length, MAX_TABLE_NAME_LENGTH); tscFreeSqlObj(pSql); - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } - char *str = calloc(1, tblListLen + 1); + char *str = calloc(1, length + 1); if (str == NULL) { - tscError("0x%"PRIx64" failed to malloc sql string buffer", pSql->self); + tscError("0x%"PRIx64" failed to allocate sql string buffer", pSql->self); tscFreeSqlObj(pSql); return TSDB_CODE_TSC_OUT_OF_MEMORY; } strtolower(str, tableNameList); - int32_t code = (uint8_t) tscParseTblNameList(pSql, str, tblListLen); - - /* - * set the qhandle to 0 before return in order to erase the qhandle value assigned in the previous successful query. - * If qhandle is NOT set 0, the function of taos_free_result() will send message to server by calling tscBuildAndSendRequest() - * to free connection, which may cause segment fault, when the parse phrase is not even successfully executed. - */ - pRes->qId = 0; + SArray* plist = taosArrayInit(4, POINTER_BYTES); + SArray* vgroupList = taosArrayInit(4, POINTER_BYTES); + + int32_t code = (uint8_t) tscTransferTableNameList(pSql, str, length, plist); free(str); if (code != TSDB_CODE_SUCCESS) { @@ -1059,12 +978,23 @@ int taos_load_table_info(TAOS *taos, const char *tableNameList) { return code; } - tscDoQuery(pSql); + registerSqlObj(pSql); + tscDebug("0x%"PRIx64" load multiple table meta, tableNameList: %s pObj:%p", pSql->self, tableNameList, pObj); - tscDebug("0x%"PRIx64" load multi-table meta result:%d %s pObj:%p", pSql->self, pRes->code, taos_errstr(pSql), pObj); - if ((code = pRes->code) != TSDB_CODE_SUCCESS) { - tscFreeSqlObj(pSql); + code = getMultiTableMetaFromMnode(pSql, plist, vgroupList, loadMultiTableMetaCallback); + if (code == TSDB_CODE_TSC_ACTION_IN_PROGRESS) { + code = TSDB_CODE_SUCCESS; } + taosArrayDestroyEx(plist, freeElem); + taosArrayDestroyEx(vgroupList, freeElem); + + if (code != TSDB_CODE_SUCCESS) { + tscFreeRegisteredSqlObj(pSql); + return code; + } + + tsem_wait(&pSql->rspSem); + tscFreeRegisteredSqlObj(pSql); return code; } diff --git a/src/client/src/tscStream.c b/src/client/src/tscStream.c index 17bf575b60eed9f71e72a608d0aef4944c887bef..8e11fd0cfb435e0eadc24ecdcd175163f81e797b 100644 --- a/src/client/src/tscStream.c +++ b/src/client/src/tscStream.c @@ -37,7 +37,7 @@ static int64_t getDelayValueAfterTimewindowClosed(SSqlStream* pStream, int64_t l static bool isProjectStream(SQueryInfo* pQueryInfo) { for (int32_t i = 0; i < pQueryInfo->fieldsInfo.numOfOutput; ++i) { - SExprInfo *pExpr = tscSqlExprGet(pQueryInfo, i); + SExprInfo *pExpr = tscExprGet(pQueryInfo, i); if (pExpr->base.functionId != TSDB_FUNC_PRJ) { return false; } @@ -89,12 +89,12 @@ static void doLaunchQuery(void* param, TAOS_RES* tres, int32_t code) { return; } - SQueryInfo *pQueryInfo = tscGetQueryInfo(&pSql->cmd, 0); + SQueryInfo *pQueryInfo = tscGetQueryInfo(&pSql->cmd); STableMetaInfo *pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0); code = tscGetTableMeta(pSql, pTableMetaInfo); if (code == 0 && UTIL_TABLE_IS_SUPER_TABLE(pTableMetaInfo)) { - code = tscGetSTableVgroupInfo(pSql, 0); + code = tscGetSTableVgroupInfo(pSql, pQueryInfo); } if (code == TSDB_CODE_TSC_ACTION_IN_PROGRESS) { @@ -138,7 +138,7 @@ static void tscProcessStreamTimer(void *handle, void *tmrId) { pStream->numOfRes = 0; // reset the numOfRes. SSqlObj *pSql = pStream->pSql; - SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd, 0); + SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd); tscDebug("0x%"PRIx64" timer launch query", pSql->self); if (pStream->isProject) { @@ -197,7 +197,7 @@ static void tscProcessStreamQueryCallback(void *param, TAOS_RES *tres, int numOf tscError("0x%"PRIx64" stream:%p, query data failed, code:0x%08x, retry in %" PRId64 "ms", pStream->pSql->self, pStream, numOfRows, retryDelay); - STableMetaInfo* pTableMetaInfo = tscGetTableMetaInfoFromCmd(&pStream->pSql->cmd, 0, 0); + STableMetaInfo* pTableMetaInfo = tscGetTableMetaInfoFromCmd(&pStream->pSql->cmd, 0); char name[TSDB_TABLE_FNAME_LEN] = {0}; tNameExtractFullName(&pTableMetaInfo->name, name); @@ -224,7 +224,7 @@ static void tscProcessStreamQueryCallback(void *param, TAOS_RES *tres, int numOf static void tscStreamFillTimeGap(SSqlStream* pStream, TSKEY ts) { #if 0 SSqlObj * pSql = pStream->pSql; - SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd, 0); + SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd); if (pQueryInfo->fillType != TSDB_FILL_SET_VALUE && pQueryInfo->fillType != TSDB_FILL_NULL) { return; @@ -273,7 +273,7 @@ static void tscProcessStreamRetrieveResult(void *param, TAOS_RES *res, int numOf return; } - SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd, 0); + SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd); STableMetaInfo *pTableMetaInfo = pQueryInfo->pTableMetaInfo[0]; if (numOfRows > 0) { // when reaching here the first execution of stream computing is successful. @@ -444,7 +444,7 @@ static int32_t tscSetSlidingWindowInfo(SSqlObj *pSql, SSqlStream *pStream) { int64_t minIntervalTime = (pStream->precision == TSDB_TIME_PRECISION_MICRO) ? tsMinIntervalTime * 1000L : tsMinIntervalTime; - SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd, 0); + SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd); if (!pStream->isProject && pQueryInfo->interval.interval == 0) { sprintf(pSql->cmd.payload, "the interval value is 0"); @@ -494,7 +494,7 @@ static int32_t tscSetSlidingWindowInfo(SSqlObj *pSql, SSqlStream *pStream) { } static int64_t tscGetStreamStartTimestamp(SSqlObj *pSql, SSqlStream *pStream, int64_t stime) { - SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd, 0); + SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd); if (pStream->isProject) { // no data in table, flush all data till now to destination meter, 10sec delay @@ -556,7 +556,7 @@ static void tscCreateStream(void *param, TAOS_RES *res, int code) { return; } - SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd, 0); + SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd); STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0); STableComInfo tinfo = tscGetTableInfo(pTableMetaInfo->pTableMeta); @@ -614,31 +614,32 @@ TAOS_STREAM *taos_open_stream(TAOS *taos, const char *sqlstr, void (*fp)(void *p return NULL; } - pStream->stime = stime; - pStream->fp = fp; + pStream->stime = stime; + pStream->fp = fp; pStream->callback = callback; - pStream->param = param; - pStream->pSql = pSql; - pSql->pStream = pStream; - pSql->param = pStream; - pSql->maxRetry = TSDB_MAX_REPLICA; + pStream->param = param; + pStream->pSql = pSql; - pSql->sqlstr = calloc(1, strlen(sqlstr) + 1); + pSql->pStream = pStream; + pSql->param = pStream; + pSql->maxRetry = TSDB_MAX_REPLICA; + pSql->sqlstr = calloc(1, strlen(sqlstr) + 1); if (pSql->sqlstr == NULL) { tscError("0x%"PRIx64" failed to malloc sql string buffer", pSql->self); tscFreeSqlObj(pSql); + free(pStream); return NULL; } strtolower(pSql->sqlstr, sqlstr); + pSql->fp = tscCreateStream; + pSql->fetchFp = tscCreateStream; + pSql->cmd.resColumnId = TSDB_RES_COL_ID; + tsem_init(&pSql->rspSem, 0, 0); registerSqlObj(pSql); tscDebugL("0x%"PRIx64" SQL: %s", pSql->self, pSql->sqlstr); - tsem_init(&pSql->rspSem, 0, 0); - - pSql->fp = tscCreateStream; - pSql->fetchFp = tscCreateStream; int32_t code = tsParseSql(pSql, true); if (code == TSDB_CODE_SUCCESS) { diff --git a/src/client/src/tscSub.c b/src/client/src/tscSub.c index 6928058f2301b05f7eda0b4a2b77f0d0edf0f45f..42330aa18e48eb60389ef3056c96a2aab386a6ca 100644 --- a/src/client/src/tscSub.c +++ b/src/client/src/tscSub.c @@ -151,6 +151,7 @@ static SSub* tscCreateSubscription(STscObj* pObj, const char* topic, const char* strtolower(pSql->sqlstr, pSql->sqlstr); pRes->qId = 0; pRes->numOfRows = 1; + pCmd->resColumnId = TSDB_RES_COL_ID; code = tscAllocPayload(pCmd, TSDB_DEFAULT_PAYLOAD_SIZE); if (code != TSDB_CODE_SUCCESS) { @@ -173,7 +174,7 @@ static SSub* tscCreateSubscription(STscObj* pObj, const char* topic, const char* if (pSql->cmd.command != TSDB_SQL_SELECT && pSql->cmd.command != TSDB_SQL_RETRIEVE_EMPTY_RESULT) { line = __LINE__; - code = TSDB_CODE_TSC_INVALID_SQL; + code = TSDB_CODE_TSC_INVALID_OPERATION; goto fail; } @@ -215,7 +216,7 @@ static void tscProcessSubscriptionTimer(void *handle, void *tmrId) { taosTmrReset(tscProcessSubscriptionTimer, pSub->interval, pSub, tscTmr, &pSub->pTimer); } - +//TODO refactor: extract table list name not simply from the sql static SArray* getTableList( SSqlObj* pSql ) { const char* p = strstr( pSql->sqlstr, " from " ); assert(p != NULL); // we are sure this is a 'select' statement @@ -224,11 +225,11 @@ static SArray* getTableList( SSqlObj* pSql ) { SSqlObj* pNew = taos_query(pSql->pTscObj, sql); if (pNew == NULL) { - tscError("0x%"PRIx64"failed to retrieve table id: cannot create new sql object.", pSql->self); + tscError("0x%"PRIx64" failed to retrieve table id: cannot create new sql object.", pSql->self); return NULL; } else if (taos_errno(pNew) != TSDB_CODE_SUCCESS) { - tscError("0x%"PRIx64"failed to retrieve table id,error: %s", pSql->self, tstrerror(taos_errno(pNew))); + tscError("0x%"PRIx64" failed to retrieve table id,error: %s", pSql->self, tstrerror(taos_errno(pNew))); return NULL; } @@ -266,7 +267,7 @@ static int tscUpdateSubscription(STscObj* pObj, SSub* pSub) { pSub->lastSyncTime = taosGetTimestampMs(); - STableMetaInfo *pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, pCmd->clauseIndex, 0); + STableMetaInfo *pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, 0); if (UTIL_TABLE_IS_NORMAL_TABLE(pTableMetaInfo)) { STableMeta * pTableMeta = pTableMetaInfo->pTableMeta; SSubscriptionProgress target = {.uid = pTableMeta->id.uid, .key = 0}; @@ -284,7 +285,7 @@ static int tscUpdateSubscription(STscObj* pObj, SSub* pSub) { } size_t numOfTables = taosArrayGetSize(tables); - SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd, 0); + SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd); SArray* progress = taosArrayInit(numOfTables, sizeof(SSubscriptionProgress)); for( size_t i = 0; i < numOfTables; i++ ) { STidTags* tt = taosArrayGet( tables, i ); @@ -304,7 +305,7 @@ static int tscUpdateSubscription(STscObj* pObj, SSub* pSub) { } taosArrayDestroy(tables); - TSDB_QUERY_SET_TYPE(tscGetQueryInfo(pCmd, 0)->type, TSDB_QUERY_TYPE_MULTITABLE_QUERY); + TSDB_QUERY_SET_TYPE(tscGetQueryInfo(pCmd)->type, TSDB_QUERY_TYPE_MULTITABLE_QUERY); return 1; } @@ -503,8 +504,8 @@ TAOS_RES *taos_consume(TAOS_SUB *tsub) { SSqlObj *pSql = pSub->pSql; SSqlRes *pRes = &pSql->res; SSqlCmd *pCmd = &pSql->cmd; - STableMetaInfo *pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, pCmd->clauseIndex, 0); - SQueryInfo *pQueryInfo = tscGetQueryInfo(pCmd, 0); + STableMetaInfo *pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, 0); + SQueryInfo *pQueryInfo = tscGetQueryInfo(pCmd); if (taosArrayGetSize(pSub->progress) > 0) { // fix crash in single table subscription size_t size = taosArrayGetSize(pSub->progress); diff --git a/src/client/src/tscSubquery.c b/src/client/src/tscSubquery.c index f5a6e857fbd45ae486ba7298a6cb2da542572ac0..325281dce59d765e3432cb8732d76ca8067e278b 100644 --- a/src/client/src/tscSubquery.c +++ b/src/client/src/tscSubquery.c @@ -65,15 +65,11 @@ static void skipRemainValue(STSBuf* pTSBuf, tVariant* tag1) { } static void subquerySetState(SSqlObj *pSql, SSubqueryState *subState, int idx, int8_t state) { - assert(idx < subState->numOfSub); - assert(subState->states); + assert(idx < subState->numOfSub && subState->states != NULL); + tscDebug("subquery:0x%"PRIx64",%d state set to %d", pSql->self, idx, state); pthread_mutex_lock(&subState->mutex); - - tscDebug("subquery:0x%"PRIx64",%d state set to %d", pSql->self, idx, state); - subState->states[idx] = state; - pthread_mutex_unlock(&subState->mutex); } @@ -86,8 +82,7 @@ static bool allSubqueryDone(SSqlObj *pParentSql) { for (int i = 0; i < subState->numOfSub; i++) { SSqlObj* pSub = pParentSql->pSubs[i]; if (0 == subState->states[i]) { - tscDebug("0x%"PRIx64" subquery:0x%"PRIx64", index: %d NOT finished, abort query completion check", pParentSql->self, - pSub->self, i); + tscDebug("0x%"PRIx64" subquery:0x%"PRIx64", index: %d NOT finished yet", pParentSql->self, pSub->self, i); done = false; break; } else { @@ -102,39 +97,31 @@ static bool allSubqueryDone(SSqlObj *pParentSql) { return done; } -static bool subAndCheckDone(SSqlObj *pSql, SSqlObj *pParentSql, int idx) { +bool subAndCheckDone(SSqlObj *pSql, SSqlObj *pParentSql, int idx) { SSubqueryState *subState = &pParentSql->subState; - assert(idx < subState->numOfSub); pthread_mutex_lock(&subState->mutex); - bool done = allSubqueryDone(pParentSql); - - if (done) { - tscDebug("0x%"PRIx64" subquery:0x%"PRIx64",%d all subs already done", pParentSql->self, - pSql->self, idx); - - pthread_mutex_unlock(&subState->mutex); - - return false; - } - - tscDebug("0x%"PRIx64" subquery:0x%"PRIx64",%d state set to 1", pParentSql->self, pSql->self, idx); +// bool done = allSubqueryDone(pParentSql); +// if (done) { +// tscDebug("0x%"PRIx64" subquery:0x%"PRIx64",%d all subs already done", pParentSql->self, pSql->self, idx); +// pthread_mutex_unlock(&subState->mutex); +// return false; +// } + tscDebug("0x%"PRIx64" subquery:0x%"PRIx64", index:%d state set to 1", pParentSql->self, pSql->self, idx); subState->states[idx] = 1; - done = allSubqueryDone(pParentSql); - + bool done = allSubqueryDone(pParentSql); pthread_mutex_unlock(&subState->mutex); - return done; } static int64_t doTSBlockIntersect(SSqlObj* pSql, STimeWindow * win) { - SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd, pSql->cmd.clauseIndex); + SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd); win->skey = INT64_MAX; win->ekey = INT64_MIN; @@ -159,7 +146,7 @@ static int64_t doTSBlockIntersect(SSqlObj* pSql, STimeWindow * win) { for (int32_t i = 0; i < joinNum; ++i) { STSBuf* output = tsBufCreate(true, pQueryInfo->order.order); - SQueryInfo* pSubQueryInfo = tscGetQueryInfo(&pSql->pSubs[i]->cmd, 0); + SQueryInfo* pSubQueryInfo = tscGetQueryInfo(&pSql->pSubs[i]->cmd); pSubQueryInfo->tsBuf = output; @@ -404,12 +391,12 @@ SJoinSupporter* tscCreateJoinSupporter(SSqlObj* pSql, int32_t index) { pSupporter->pObj = pSql; pSupporter->subqueryIndex = index; - SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd, pSql->cmd.clauseIndex); + SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd); memcpy(&pSupporter->interval, &pQueryInfo->interval, sizeof(pSupporter->interval)); pSupporter->limit = pQueryInfo->limit; - STableMetaInfo* pTableMetaInfo = tscGetTableMetaInfoFromCmd(&pSql->cmd, pSql->cmd.clauseIndex, index); + STableMetaInfo* pTableMetaInfo = tscGetTableMetaInfoFromCmd(&pSql->cmd, index); pSupporter->uid = pTableMetaInfo->pTableMeta->id.uid; assert (pSupporter->uid != 0); @@ -427,14 +414,15 @@ static void tscDestroyJoinSupporter(SJoinSupporter* pSupporter) { } if (pSupporter->exprList != NULL) { - tscSqlExprInfoDestroy(pSupporter->exprList); + tscExprDestroy(pSupporter->exprList); + pSupporter->exprList = NULL; } if (pSupporter->colList != NULL) { tscColumnListDestroy(pSupporter->colList); } - tscFieldInfoClear(&pSupporter->fieldsInfo); +// tscFieldInfoClear(&pSupporter->fieldsInfo); if (pSupporter->pTSBuf != NULL) { tsBufDestroy(pSupporter->pTSBuf); @@ -448,7 +436,6 @@ static void tscDestroyJoinSupporter(SJoinSupporter* pSupporter) { pSupporter->f = NULL; } - if (pSupporter->pVgroupTables != NULL) { taosArrayDestroy(pSupporter->pVgroupTables); pSupporter->pVgroupTables = NULL; @@ -559,12 +546,12 @@ static int32_t tscLaunchRealSubqueries(SSqlObj* pSql) { continue; } - SQueryInfo *pSubQueryInfo = tscGetQueryInfo(&pPrevSub->cmd, 0); + SQueryInfo *pSubQueryInfo = tscGetQueryInfo(&pPrevSub->cmd); STSBuf *pTsBuf = pSubQueryInfo->tsBuf; pSubQueryInfo->tsBuf = NULL; // free result for async object will also free sqlObj - assert(tscSqlExprNumOfExprs(pSubQueryInfo) == 1); // ts_comp query only requires one result columns + assert(tscNumOfExprs(pSubQueryInfo) == 1); // ts_comp query only requires one result columns taos_free_result(pPrevSub); SSqlObj *pNew = createSubqueryObj(pSql, (int16_t) i, tscJoinQueryCallback, pSupporter, TSDB_SQL_SELECT, NULL); @@ -577,7 +564,7 @@ static int32_t tscLaunchRealSubqueries(SSqlObj* pSql) { tscClearSubqueryInfo(&pNew->cmd); pSql->pSubs[i] = pNew; - SQueryInfo *pQueryInfo = tscGetQueryInfo(&pNew->cmd, 0); + SQueryInfo *pQueryInfo = tscGetQueryInfo(&pNew->cmd); pQueryInfo->tsBuf = pTsBuf; // transfer the ownership of timestamp comp-z data to the new created object // set the second stage sub query for join process @@ -592,7 +579,7 @@ static int32_t tscLaunchRealSubqueries(SSqlObj* pSql) { pQueryInfo->groupbyExpr = pSupporter->groupInfo; pQueryInfo->pUpstream = taosArrayInit(4, sizeof(POINTER_BYTES)); - assert(pNew->subState.numOfSub == 0 && pNew->cmd.numOfClause == 1 && pQueryInfo->numOfTables == 1); + assert(pNew->subState.numOfSub == 0 && pQueryInfo->numOfTables == 1); tscFieldInfoUpdateOffset(pQueryInfo); @@ -600,35 +587,34 @@ static int32_t tscLaunchRealSubqueries(SSqlObj* pSql) { pTableMetaInfo->pVgroupTables = pSupporter->pVgroupTables; pSupporter->exprList = NULL; - pSupporter->colList = NULL; + pSupporter->colList = NULL; pSupporter->pVgroupTables = NULL; memset(&pSupporter->fieldsInfo, 0, sizeof(SFieldInfo)); - memset(&pSupporter->groupInfo, 0, sizeof(SSqlGroupbyExpr)); + memset(&pSupporter->groupInfo, 0, sizeof(SGroupbyExpr)); /* * When handling the projection query, the offset value will be modified for table-table join, which is changed * during the timestamp intersection. */ pSupporter->limit = pQueryInfo->limit; -// pQueryInfo->limit = pSupporter->limit; - SColumnIndex index = {.tableIndex = 0, .columnIndex = PRIMARYKEY_TIMESTAMP_COL_INDEX}; SSchema* s = tscGetTableColumnSchema(pTableMetaInfo->pTableMeta, 0); - SExprInfo* pExpr = tscSqlExprGet(pQueryInfo, 0); + SExprInfo* pExpr = tscExprGet(pQueryInfo, 0); int16_t funcId = pExpr->base.functionId; // add the invisible timestamp column + printf("--------read:%p\n", pExpr); if ((pExpr->base.colInfo.colId != PRIMARYKEY_TIMESTAMP_COL_INDEX) || (funcId != TSDB_FUNC_TS && funcId != TSDB_FUNC_TS_DUMMY && funcId != TSDB_FUNC_PRJ)) { int16_t functionId = tscIsProjectionQuery(pQueryInfo)? TSDB_FUNC_PRJ : TSDB_FUNC_TS; - tscAddFuncInSelectClause(pQueryInfo, 0, functionId, &index, s, TSDB_COL_NORMAL); + tscAddFuncInSelectClause(pQueryInfo, 0, functionId, &index, s, TSDB_COL_NORMAL, getNewResColId(&pNew->cmd)); tscPrintSelNodeList(pNew, 0); tscFieldInfoUpdateOffset(pQueryInfo); - pExpr = tscSqlExprGet(pQueryInfo, 0); + pExpr = tscExprGet(pQueryInfo, 0); } // set the join condition tag column info, todo extract method @@ -658,11 +644,11 @@ static int32_t tscLaunchRealSubqueries(SSqlObj* pSql) { } } - subquerySetState(pPrevSub, &pSql->subState, i, 0); + subquerySetState(pNew, &pSql->subState, i, 0); size_t numOfCols = taosArrayGetSize(pQueryInfo->colList); - tscDebug("0x%"PRIx64" subquery:%p tableIndex:%d, vgroupIndex:%d, type:%d, exprInfo:%" PRIzu ", colList:%" PRIzu ", fieldsInfo:%d, name:%s", - pSql->self, pNew, 0, pTableMetaInfo->vgroupIndex, pQueryInfo->type, taosArrayGetSize(pQueryInfo->exprList), + tscDebug("0x%"PRIx64" subquery:0x%"PRIx64" tableIndex:%d, vgroupIndex:%d, type:%d, exprInfo:%" PRIzu ", colList:%" PRIzu ", fieldsInfo:%d, name:%s", + pSql->self, pNew->self, 0, pTableMetaInfo->vgroupIndex, pQueryInfo->type, taosArrayGetSize(pQueryInfo->exprList), numOfCols, pQueryInfo->fieldsInfo.numOfOutput, tNameGetTableName(&pTableMetaInfo->name)); } @@ -681,7 +667,7 @@ static int32_t tscLaunchRealSubqueries(SSqlObj* pSql) { continue; } - SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->pSubs[i]->cmd, 0); + SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->pSubs[i]->cmd); executeQuery(pSql->pSubs[i], pQueryInfo); } @@ -707,8 +693,6 @@ void freeJoinSubqueryObj(SSqlObj* pSql) { } tfree(pSql->subState.states); - - pSql->subState.numOfSub = 0; } @@ -727,8 +711,6 @@ static int32_t quitAllSubquery(SSqlObj* pSqlSub, SSqlObj* pSqlObj, SJoinSupporte static void updateQueryTimeRange(SQueryInfo* pQueryInfo, STimeWindow* win) { assert(pQueryInfo->window.skey <= win->skey && pQueryInfo->window.ekey >= win->ekey); pQueryInfo->window = *win; - - } int32_t tidTagsCompar(const void* p1, const void* p2) { @@ -820,12 +802,14 @@ static void issueTsCompQuery(SSqlObj* pSql, SJoinSupporter* pSupporter, SSqlObj* tscClearSubqueryInfo(pCmd); tscFreeSqlResult(pSql); - SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd, 0); + SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd); assert(pQueryInfo->numOfTables == 1); STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0); + STimeWindow window = pQueryInfo->window; tscInitQueryInfo(pQueryInfo); + pQueryInfo->window = window; TSDB_QUERY_CLEAR_TYPE(pQueryInfo->type, TSDB_QUERY_TYPE_TAG_FILTER_QUERY); TSDB_QUERY_SET_TYPE(pQueryInfo->type, TSDB_QUERY_TYPE_MULTITABLE_QUERY); @@ -835,11 +819,11 @@ static void issueTsCompQuery(SSqlObj* pSql, SJoinSupporter* pSupporter, SSqlObj* SSchema colSchema = {.type = TSDB_DATA_TYPE_BINARY, .bytes = 1}; SColumnIndex index = {0, PRIMARYKEY_TIMESTAMP_COL_INDEX}; - tscAddFuncInSelectClause(pQueryInfo, 0, TSDB_FUNC_TS_COMP, &index, &colSchema, TSDB_COL_NORMAL); + tscAddFuncInSelectClause(pQueryInfo, 0, TSDB_FUNC_TS_COMP, &index, &colSchema, TSDB_COL_NORMAL, getNewResColId(pCmd)); // set the tags value for ts_comp function if (UTIL_TABLE_IS_SUPER_TABLE(pTableMetaInfo)) { - SExprInfo *pExpr = tscSqlExprGet(pQueryInfo, 0); + SExprInfo *pExpr = tscExprGet(pQueryInfo, 0); int16_t tagColId = tscGetJoinTagColIdByUid(&pSupporter->tagCond, pTableMetaInfo->pTableMeta->id.uid); pExpr->base.param[0].i64 = tagColId; pExpr->base.param[0].nLen = sizeof(int64_t); @@ -867,7 +851,7 @@ static void issueTsCompQuery(SSqlObj* pSql, SJoinSupporter* pSupporter, SSqlObj* "0x%"PRIx64" subquery:0x%"PRIx64" tableIndex:%d, vgroupIndex:%d, numOfVgroups:%d, type:%d, ts_comp query to retrieve timestamps, " "numOfExpr:%" PRIzu ", colList:%" PRIzu ", numOfOutputFields:%d, name:%s", pParent->self, pSql->self, 0, pTableMetaInfo->vgroupIndex, pTableMetaInfo->vgroupList->numOfVgroups, pQueryInfo->type, - tscSqlExprNumOfExprs(pQueryInfo), numOfCols, pQueryInfo->fieldsInfo.numOfOutput, tNameGetTableName(&pTableMetaInfo->name)); + tscNumOfExprs(pQueryInfo), numOfCols, pQueryInfo->fieldsInfo.numOfOutput, tNameGetTableName(&pTableMetaInfo->name)); tscBuildAndSendRequest(pSql, NULL); } @@ -1109,7 +1093,7 @@ static void tidTagRetrieveCallback(void* param, TAOS_RES* tres, int32_t numOfRow SSqlCmd* pCmd = &pSql->cmd; SSqlRes* pRes = &pSql->res; - SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd, pCmd->clauseIndex); + SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd); // todo, the type may not include TSDB_QUERY_TYPE_TAG_FILTER_QUERY assert(TSDB_QUERY_HAS_TYPE(pQueryInfo->type, TSDB_QUERY_TYPE_TAG_FILTER_QUERY)); @@ -1231,7 +1215,7 @@ static void tidTagRetrieveCallback(void* param, TAOS_RES* tres, int32_t numOfRow SSqlCmd* pSubCmd = &pParentSql->pSubs[m]->cmd; SArray** s = taosArrayGet(resList, m); - SQueryInfo* pQueryInfo1 = tscGetQueryInfo(pSubCmd, 0); + SQueryInfo* pQueryInfo1 = tscGetQueryInfo(pSubCmd); STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo1, 0); tscBuildVgroupTableInfo(pParentSql, pTableMetaInfo, *s); @@ -1265,7 +1249,7 @@ static void tsCompRetrieveCallback(void* param, TAOS_RES* tres, int32_t numOfRow SSqlCmd* pCmd = &pSql->cmd; SSqlRes* pRes = &pSql->res; - SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd, pCmd->clauseIndex); + SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd); assert(!TSDB_QUERY_HAS_TYPE(pQueryInfo->type, TSDB_QUERY_TYPE_JOIN_SEC_STAGE)); if (pParentSql->res.code != TSDB_CODE_SUCCESS) { @@ -1300,7 +1284,6 @@ static void tsCompRetrieveCallback(void* param, TAOS_RES* tres, int32_t numOfRow if (pSupporter->f == NULL) { tscError("0x%"PRIx64" failed to create tmp file:%s, reason:%s", pSql->self, pSupporter->path, strerror(errno)); - pParentSql->res.code = TAOS_SYSTEM_ERROR(errno); if (quitAllSubquery(pSql, pParentSql, pSupporter)) { @@ -1397,7 +1380,7 @@ static void tsCompRetrieveCallback(void* param, TAOS_RES* tres, int32_t numOfRow } // launch the query the retrieve actual results from vnode along with the filtered timestamp - SQueryInfo* pPQueryInfo = tscGetQueryInfo(&pParentSql->cmd, pParentSql->cmd.clauseIndex); + SQueryInfo* pPQueryInfo = tscGetQueryInfo(&pParentSql->cmd); updateQueryTimeRange(pPQueryInfo, &win); //update the vgroup that involved in real data query @@ -1413,7 +1396,7 @@ static void joinRetrieveFinalResCallback(void* param, TAOS_RES* tres, int numOfR SSqlCmd* pCmd = &pSql->cmd; SSqlRes* pRes = &pSql->res; - SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd, pCmd->clauseIndex); + SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd); if (pParentSql->res.code != TSDB_CODE_SUCCESS) { tscError("0x%"PRIx64" abort query due to other subquery failure. code:%d, global code:%d", pSql->self, numOfRows, pParentSql->res.code); @@ -1422,7 +1405,6 @@ static void joinRetrieveFinalResCallback(void* param, TAOS_RES* tres, int numOfR } tscAsyncResultOnError(pParentSql); - return; } @@ -1522,7 +1504,7 @@ void tscFetchDatablockForSubquery(SSqlObj* pSql) { SSqlRes *pRes = &pSub->res; - SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSub->cmd, 0); + SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSub->cmd); if (!tscHasReachLimitation(pQueryInfo, pRes)) { if (pRes->row >= pRes->numOfRows) { // no data left in current result buffer @@ -1574,7 +1556,7 @@ void tscFetchDatablockForSubquery(SSqlObj* pSql) { continue; } - SQueryInfo* p = tscGetQueryInfo(&pSub->cmd, 0); + SQueryInfo* p = tscGetQueryInfo(&pSub->cmd); orderedPrjQuery = tscNonOrderedProjectionQueryOnSTable(p, 0); if (orderedPrjQuery) { break; @@ -1598,7 +1580,7 @@ void tscFetchDatablockForSubquery(SSqlObj* pSql) { continue; } - SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSub->cmd, 0); + SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSub->cmd); if (tscNonOrderedProjectionQueryOnSTable(pQueryInfo, 0) && pSub->res.row >= pSub->res.numOfRows && pSub->res.completed) { @@ -1655,7 +1637,6 @@ void tscFetchDatablockForSubquery(SSqlObj* pSql) { } SSqlRes* pRes1 = &pSql1->res; - if (pRes1->row >= pRes1->numOfRows) { subquerySetState(pSql1, &pSql->subState, i, 0); } @@ -1673,7 +1654,7 @@ void tscFetchDatablockForSubquery(SSqlObj* pSql) { pSupporter = (SJoinSupporter*)pSql1->param; // wait for all subqueries completed - SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd1, 0); + SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd1); assert(pRes1->numOfRows >= 0 && pQueryInfo->numOfTables == 1); STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0); @@ -1704,9 +1685,9 @@ void tscSetupOutputColumnIndex(SSqlObj* pSql) { return; } - SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd, pCmd->clauseIndex); + SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd); - int32_t numOfExprs = (int32_t)tscSqlExprNumOfExprs(pQueryInfo); + int32_t numOfExprs = (int32_t)tscNumOfExprs(pQueryInfo); pRes->pColumnIndex = calloc(1, sizeof(SColumnIndex) * numOfExprs); if (pRes->pColumnIndex == NULL) { pRes->code = TSDB_CODE_TSC_OUT_OF_MEMORY; @@ -1714,7 +1695,7 @@ void tscSetupOutputColumnIndex(SSqlObj* pSql) { } for (int32_t i = 0; i < numOfExprs; ++i) { - SExprInfo* pExpr = tscSqlExprGet(pQueryInfo, i); + SExprInfo* pExpr = tscExprGet(pQueryInfo, i); int32_t tableIndexOfSub = -1; for (int32_t j = 0; j < pQueryInfo->numOfTables; ++j) { @@ -1728,11 +1709,11 @@ void tscSetupOutputColumnIndex(SSqlObj* pSql) { assert(tableIndexOfSub >= 0 && tableIndexOfSub < pQueryInfo->numOfTables); SSqlCmd* pSubCmd = &pSql->pSubs[tableIndexOfSub]->cmd; - SQueryInfo* pSubQueryInfo = tscGetQueryInfo(pSubCmd, 0); + SQueryInfo* pSubQueryInfo = tscGetQueryInfo(pSubCmd); size_t numOfSubExpr = taosArrayGetSize(pSubQueryInfo->exprList); for (int32_t k = 0; k < numOfSubExpr; ++k) { - SExprInfo* pSubExpr = tscSqlExprGet(pSubQueryInfo, k); + SExprInfo* pSubExpr = tscExprGet(pSubQueryInfo, k); if (pExpr->base.functionId == pSubExpr->base.functionId && pExpr->base.colInfo.colId == pSubExpr->base.colInfo.colId) { pRes->pColumnIndex[i] = (SColumnIndex){.tableIndex = tableIndexOfSub, .columnIndex = k}; break; @@ -1741,8 +1722,8 @@ void tscSetupOutputColumnIndex(SSqlObj* pSql) { } // restore the offset value for super table query in case of final result. - tscRestoreFuncForSTableQuery(pQueryInfo); - tscFieldInfoUpdateOffset(pQueryInfo); +// tscRestoreFuncForSTableQuery(pQueryInfo); +// tscFieldInfoUpdateOffset(pQueryInfo); } void tscJoinQueryCallback(void* param, TAOS_RES* tres, int code) { @@ -1752,10 +1733,10 @@ void tscJoinQueryCallback(void* param, TAOS_RES* tres, int code) { SSqlObj* pParentSql = pSupporter->pObj; // There is only one subquery and table for each subquery. - SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd, 0); + SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd); STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0); - assert(pQueryInfo->numOfTables == 1 && pSql->cmd.numOfClause == 1); + assert(pQueryInfo->numOfTables == 1); // retrieve actual query results from vnode during the second stage join subquery if (pParentSql->res.code != TSDB_CODE_SUCCESS) { @@ -1836,7 +1817,7 @@ static SSqlObj *tscCreateSTableSubquery(SSqlObj *pSql, SRetrieveSupport *trsuppo int32_t tscCreateJoinSubquery(SSqlObj *pSql, int16_t tableIndex, SJoinSupporter *pSupporter) { SSqlCmd * pCmd = &pSql->cmd; - SQueryInfo *pQueryInfo = tscGetQueryInfo(pCmd, pCmd->clauseIndex); + SQueryInfo *pQueryInfo = tscGetQueryInfo(pCmd); pSql->res.qId = 0x1; assert(pSql->res.numOfRows == 0); @@ -1859,15 +1840,9 @@ int32_t tscCreateJoinSubquery(SSqlObj *pSql, int16_t tableIndex, SJoinSupporter addGroupInfoForSubquery(pSql, pNew, 0, tableIndex); // refactor as one method - SQueryInfo *pNewQueryInfo = tscGetQueryInfo(&pNew->cmd, 0); + SQueryInfo *pNewQueryInfo = tscGetQueryInfo(&pNew->cmd); assert(pNewQueryInfo != NULL); - // update the table index -// size_t num = taosArrayGetSize(pNewQueryInfo->colList); -// for (int32_t i = 0; i < num; ++i) { -// SColumn* pCol = taosArrayGetP(pNewQueryInfo->colList, i); -// } - pSupporter->colList = pNewQueryInfo->colList; pNewQueryInfo->colList = NULL; @@ -1883,7 +1858,7 @@ int32_t tscCreateJoinSubquery(SSqlObj *pSql, int16_t tableIndex, SJoinSupporter } pSupporter->groupInfo = pNewQueryInfo->groupbyExpr; - memset(&pNewQueryInfo->groupbyExpr, 0, sizeof(SSqlGroupbyExpr)); + memset(&pNewQueryInfo->groupbyExpr, 0, sizeof(SGroupbyExpr)); pNew->cmd.numOfCols = 0; pNewQueryInfo->interval.interval = 0; @@ -1896,9 +1871,12 @@ int32_t tscCreateJoinSubquery(SSqlObj *pSql, int16_t tableIndex, SJoinSupporter pNewQueryInfo->order.orderColId = INT32_MIN; // backup the data and clear it in the sqlcmd object - memset(&pNewQueryInfo->groupbyExpr, 0, sizeof(SSqlGroupbyExpr)); - + memset(&pNewQueryInfo->groupbyExpr, 0, sizeof(SGroupbyExpr)); + + STimeWindow range = pNewQueryInfo->window; tscInitQueryInfo(pNewQueryInfo); + + pNewQueryInfo->window = range; STableMetaInfo *pTableMetaInfo = tscGetMetaInfo(pNewQueryInfo, 0); if (UTIL_TABLE_IS_SUPER_TABLE(pTableMetaInfo)) { // return the tableId & tag @@ -1924,21 +1902,21 @@ int32_t tscCreateJoinSubquery(SSqlObj *pSql, int16_t tableIndex, SJoinSupporter // set get tags query type TSDB_QUERY_SET_TYPE(pNewQueryInfo->type, TSDB_QUERY_TYPE_TAG_FILTER_QUERY); - tscAddFuncInSelectClause(pNewQueryInfo, 0, TSDB_FUNC_TID_TAG, &colIndex, &s1, TSDB_COL_TAG); + tscAddFuncInSelectClause(pNewQueryInfo, 0, TSDB_FUNC_TID_TAG, &colIndex, &s1, TSDB_COL_TAG, getNewResColId(pCmd)); size_t numOfCols = taosArrayGetSize(pNewQueryInfo->colList); tscDebug( "%p subquery:%p tableIndex:%d, vgroupIndex:%d, type:%d, transfer to tid_tag query to retrieve (tableId, tags), " "exprInfo:%" PRIzu ", colList:%" PRIzu ", fieldsInfo:%d, tagIndex:%d, name:%s", - pSql, pNew, tableIndex, pTableMetaInfo->vgroupIndex, pNewQueryInfo->type, tscSqlExprNumOfExprs(pNewQueryInfo), + pSql, pNew, tableIndex, pTableMetaInfo->vgroupIndex, pNewQueryInfo->type, tscNumOfExprs(pNewQueryInfo), numOfCols, pNewQueryInfo->fieldsInfo.numOfOutput, colIndex.columnIndex, tNameGetTableName(&pNewQueryInfo->pTableMetaInfo[0]->name)); } else { SSchema colSchema = {.type = TSDB_DATA_TYPE_BINARY, .bytes = 1}; SColumnIndex colIndex = {0, PRIMARYKEY_TIMESTAMP_COL_INDEX}; - tscAddFuncInSelectClause(pNewQueryInfo, 0, TSDB_FUNC_TS_COMP, &colIndex, &colSchema, TSDB_COL_NORMAL); + tscAddFuncInSelectClause(pNewQueryInfo, 0, TSDB_FUNC_TS_COMP, &colIndex, &colSchema, TSDB_COL_NORMAL, getNewResColId(pCmd)); // set the tags value for ts_comp function - SExprInfo *pExpr = tscSqlExprGet(pNewQueryInfo, 0); + SExprInfo *pExpr = tscExprGet(pNewQueryInfo, 0); if (UTIL_TABLE_IS_SUPER_TABLE(pTableMetaInfo)) { int16_t tagColId = tscGetJoinTagColIdByUid(&pSupporter->tagCond, pTableMetaInfo->pTableMeta->id.uid); @@ -1965,12 +1943,12 @@ int32_t tscCreateJoinSubquery(SSqlObj *pSql, int16_t tableIndex, SJoinSupporter tscDebug( "%p subquery:%p tableIndex:%d, vgroupIndex:%d, type:%u, transfer to ts_comp query to retrieve timestamps, " "exprInfo:%" PRIzu ", colList:%" PRIzu ", fieldsInfo:%d, name:%s", - pSql, pNew, tableIndex, pTableMetaInfo->vgroupIndex, pNewQueryInfo->type, tscSqlExprNumOfExprs(pNewQueryInfo), + pSql, pNew, tableIndex, pTableMetaInfo->vgroupIndex, pNewQueryInfo->type, tscNumOfExprs(pNewQueryInfo), numOfCols, pNewQueryInfo->fieldsInfo.numOfOutput, tNameGetTableName(&pNewQueryInfo->pTableMetaInfo[0]->name)); } } else { assert(0); - SQueryInfo *pNewQueryInfo = tscGetQueryInfo(&pNew->cmd, 0); + SQueryInfo *pNewQueryInfo = tscGetQueryInfo(&pNew->cmd); pNewQueryInfo->type |= TSDB_QUERY_TYPE_SUBQUERY; } @@ -1981,7 +1959,7 @@ void tscHandleMasterJoinQuery(SSqlObj* pSql) { SSqlCmd* pCmd = &pSql->cmd; SSqlRes* pRes = &pSql->res; - SQueryInfo *pQueryInfo = tscGetQueryInfo(pCmd, pCmd->clauseIndex); + SQueryInfo *pQueryInfo = tscGetQueryInfo(pCmd); assert((pQueryInfo->type & TSDB_QUERY_TYPE_SUBQUERY) == 0); int32_t code = TSDB_CODE_SUCCESS; @@ -2016,7 +1994,7 @@ void tscHandleMasterJoinQuery(SSqlObj* pSql) { } SSqlObj* pSub = pSql->pSubs[i]; - STableMetaInfo* pTableMetaInfo = tscGetTableMetaInfoFromCmd(&pSub->cmd, 0, 0); + STableMetaInfo* pTableMetaInfo = tscGetTableMetaInfoFromCmd(&pSub->cmd, 0); if (UTIL_TABLE_IS_SUPER_TABLE(pTableMetaInfo) && (pTableMetaInfo->vgroupList->numOfVgroups == 0)) { pSql->cmd.command = TSDB_SQL_RETRIEVE_EMPTY_RESULT; break; @@ -2104,7 +2082,7 @@ typedef struct SFirstRoundQuerySup { void doAppendData(SInterResult* pInterResult, TAOS_ROW row, int32_t numOfCols, SQueryInfo* pQueryInfo) { TSKEY key = INT64_MIN; for(int32_t i = 0; i < numOfCols; ++i) { - SExprInfo* pExpr = tscSqlExprGet(pQueryInfo, i); + SExprInfo* pExpr = tscExprGet(pQueryInfo, i); if (TSDB_COL_IS_TAG(pExpr->base.colInfo.flag) || pExpr->base.functionId == TSDB_FUNC_PRJ) { continue; } @@ -2165,7 +2143,7 @@ void tscFirstRoundRetrieveCallback(void* param, TAOS_RES* tres, int numOfRows) { SFirstRoundQuerySup* pSup = param; SSqlObj* pParent = pSup->pParent; - SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd, 0); + SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd); int32_t code = taos_errno(pSql); if (code != TSDB_CODE_SUCCESS) { @@ -2197,7 +2175,7 @@ void tscFirstRoundRetrieveCallback(void* param, TAOS_RES* tres, int numOfRows) { int32_t offset = 0; for (int32_t i = 0; i < numOfCols && offset < pSup->tagLen; ++i) { - SExprInfo* pExpr = tscSqlExprGet(pQueryInfo, i); + SExprInfo* pExpr = tscExprGet(pQueryInfo, i); // tag or group by column if (TSDB_COL_IS_TAG(pExpr->base.colInfo.flag) || pExpr->base.functionId == TSDB_FUNC_PRJ) { @@ -2248,7 +2226,7 @@ void tscFirstRoundRetrieveCallback(void* param, TAOS_RES* tres, int numOfRows) { // set the parameters for the second round query process SSqlCmd *pPCmd = &pParent->cmd; - SQueryInfo *pQueryInfo1 = tscGetQueryInfo(pPCmd, 0); + SQueryInfo *pQueryInfo1 = tscGetQueryInfo(pPCmd); int32_t resRows = pSup->numOfRows; if (pSup->numOfRows > 0) { @@ -2298,8 +2276,8 @@ void tscFirstRoundCallback(void* param, TAOS_RES* tres, int code) { } int32_t tscHandleFirstRoundStableQuery(SSqlObj *pSql) { - SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd, 0); - STableMetaInfo* pTableMetaInfo1 = tscGetTableMetaInfoFromCmd(&pSql->cmd, 0, 0); + SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd); + STableMetaInfo* pTableMetaInfo1 = tscGetTableMetaInfoFromCmd(&pSql->cmd, 0); SFirstRoundQuerySup *pSup = calloc(1, sizeof(SFirstRoundQuerySup)); @@ -2311,15 +2289,20 @@ int32_t tscHandleFirstRoundStableQuery(SSqlObj *pSql) { SSqlObj *pNew = createSubqueryObj(pSql, 0, tscFirstRoundCallback, pSup, TSDB_SQL_SELECT, NULL); SSqlCmd *pCmd = &pNew->cmd; + SQueryInfo* pNewQueryInfo = tscGetQueryInfo(pCmd); + assert(pQueryInfo->numOfTables == 1); + + SArray* pColList = pNewQueryInfo->colList; + pNewQueryInfo->colList = NULL; + tscClearSubqueryInfo(pCmd); tscFreeSqlResult(pSql); - SQueryInfo* pNewQueryInfo = tscGetQueryInfo(pCmd, 0); - assert(pQueryInfo->numOfTables == 1); - STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pNewQueryInfo, 0); tscInitQueryInfo(pNewQueryInfo); + + // add the group cond pNewQueryInfo->groupbyExpr = pQueryInfo->groupbyExpr; if (pQueryInfo->groupbyExpr.columnInfo != NULL) { pNewQueryInfo->groupbyExpr.columnInfo = taosArrayDup(pQueryInfo->groupbyExpr.columnInfo); @@ -2329,28 +2312,31 @@ int32_t tscHandleFirstRoundStableQuery(SSqlObj *pSql) { } } + // add the tag filter cond if (tscTagCondCopy(&pNewQueryInfo->tagCond, &pQueryInfo->tagCond) != 0) { terrno = TSDB_CODE_TSC_OUT_OF_MEMORY; goto _error; } + pNewQueryInfo->window = pQueryInfo->window; pNewQueryInfo->interval = pQueryInfo->interval; + pNewQueryInfo->sessionWindow = pQueryInfo->sessionWindow; pCmd->command = TSDB_SQL_SELECT; pNew->fp = tscFirstRoundCallback; - int32_t numOfExprs = (int32_t) tscSqlExprNumOfExprs(pQueryInfo); + int32_t numOfExprs = (int32_t) tscNumOfExprs(pQueryInfo); int32_t index = 0; for(int32_t i = 0; i < numOfExprs; ++i) { - SExprInfo* pExpr = tscSqlExprGet(pQueryInfo, i); + SExprInfo* pExpr = tscExprGet(pQueryInfo, i); if (pExpr->base.functionId == TSDB_FUNC_TS && pQueryInfo->interval.interval > 0) { taosArrayPush(pSup->pColsInfo, &pExpr->base.resColId); SColumnIndex colIndex = {.tableIndex = 0, .columnIndex = PRIMARYKEY_TIMESTAMP_COL_INDEX}; SSchema* schema = tscGetColumnSchemaById(pTableMetaInfo1->pTableMeta, pExpr->base.colInfo.colId); - SExprInfo* p = tscAddFuncInSelectClause(pNewQueryInfo, index++, TSDB_FUNC_TS, &colIndex, schema, TSDB_COL_NORMAL); + SExprInfo* p = tscAddFuncInSelectClause(pNewQueryInfo, index++, TSDB_FUNC_TS, &colIndex, schema, TSDB_COL_NORMAL, getNewResColId(pCmd)); p->base.resColId = pExpr->base.resColId; // update the result column id } else if (pExpr->base.functionId == TSDB_FUNC_STDDEV_DST) { taosArrayPush(pSup->pColsInfo, &pExpr->base.resColId); @@ -2359,7 +2345,7 @@ int32_t tscHandleFirstRoundStableQuery(SSqlObj *pSql) { SSchema schema = {.type = TSDB_DATA_TYPE_DOUBLE, .bytes = sizeof(double)}; tstrncpy(schema.name, pExpr->base.aliasName, tListLen(schema.name)); - SExprInfo* p = tscAddFuncInSelectClause(pNewQueryInfo, index++, TSDB_FUNC_AVG, &colIndex, &schema, TSDB_COL_NORMAL); + SExprInfo* p = tscAddFuncInSelectClause(pNewQueryInfo, index++, TSDB_FUNC_AVG, &colIndex, &schema, TSDB_COL_NORMAL, getNewResColId(pCmd)); p->base.resColId = pExpr->base.resColId; // update the result column id } else if (pExpr->base.functionId == TSDB_FUNC_TAG) { pSup->tagLen += pExpr->base.resBytes; @@ -2372,7 +2358,7 @@ int32_t tscHandleFirstRoundStableQuery(SSqlObj *pSql) { schema = tGetTbnameColumnSchema(); } - SExprInfo* p = tscAddFuncInSelectClause(pNewQueryInfo, index++, TSDB_FUNC_TAG, &colIndex, schema, TSDB_COL_TAG); + SExprInfo* p = tscAddFuncInSelectClause(pNewQueryInfo, index++, TSDB_FUNC_TAG, &colIndex, schema, TSDB_COL_TAG, getNewResColId(pCmd)); p->base.resColId = pExpr->base.resColId; } else if (pExpr->base.functionId == TSDB_FUNC_PRJ) { int32_t num = (int32_t) taosArrayGetSize(pNewQueryInfo->groupbyExpr.columnInfo); @@ -2386,7 +2372,7 @@ int32_t tscHandleFirstRoundStableQuery(SSqlObj *pSql) { SSchema* schema = tscGetColumnSchemaById(pTableMetaInfo1->pTableMeta, pExpr->base.colInfo.colId); //doLimitOutputNormalColOfGroupby - SExprInfo* p = tscAddFuncInSelectClause(pNewQueryInfo, index++, TSDB_FUNC_PRJ, &colIndex, schema, TSDB_COL_NORMAL); + SExprInfo* p = tscAddFuncInSelectClause(pNewQueryInfo, index++, TSDB_FUNC_PRJ, &colIndex, schema, TSDB_COL_NORMAL, getNewResColId(pCmd)); p->base.numOfParams = 1; p->base.param[0].i64 = 1; p->base.param[0].nType = TSDB_DATA_TYPE_INT; @@ -2396,6 +2382,21 @@ int32_t tscHandleFirstRoundStableQuery(SSqlObj *pSql) { } } + // add the normal column filter cond + if (pColList != NULL) { + size_t s = taosArrayGetSize(pColList); + for (int32_t i = 0; i < s; ++i) { + SColumn *pCol = taosArrayGetP(pColList, i); + + if (pCol->info.flist.numOfFilters > 0) { // copy to the pNew->cmd.colList if it is filtered. + SColumn *p = tscColumnClone(pCol); + taosArrayPush(pNewQueryInfo->colList, &p); + } + } + + tscColumnListDestroy(pColList); + } + tscInsertPrimaryTsSourceColumn(pNewQueryInfo, pTableMetaInfo->pTableMeta->id.uid); tscTansformFuncForSTableQuery(pNewQueryInfo); @@ -2403,7 +2404,7 @@ int32_t tscHandleFirstRoundStableQuery(SSqlObj *pSql) { "0x%"PRIx64" first round subquery:0x%"PRIx64" tableIndex:%d, vgroupIndex:%d, numOfVgroups:%d, type:%d, query to retrieve timestamps, " "numOfExpr:%" PRIzu ", colList:%d, numOfOutputFields:%d, name:%s", pSql->self, pNew->self, 0, pTableMetaInfo->vgroupIndex, pTableMetaInfo->vgroupList->numOfVgroups, pNewQueryInfo->type, - tscSqlExprNumOfExprs(pNewQueryInfo), index+1, pNewQueryInfo->fieldsInfo.numOfOutput, tNameGetTableName(&pTableMetaInfo->name)); + tscNumOfExprs(pNewQueryInfo), index+1, pNewQueryInfo->fieldsInfo.numOfOutput, tNameGetTableName(&pTableMetaInfo->name)); tscHandleMasterSTableQuery(pNew); return TSDB_CODE_SUCCESS; @@ -2428,14 +2429,12 @@ int32_t tscHandleMasterSTableQuery(SSqlObj *pSql) { tExtMemBuffer **pMemoryBuf = NULL; tOrderDescriptor *pDesc = NULL; - SColumnModel *pModel = NULL; - SColumnModel *pFinalModel = NULL; pRes->qId = 0x1; // hack the qhandle check const uint32_t nBufferSize = (1u << 16u); // 64KB - SQueryInfo *pQueryInfo = tscGetActiveQueryInfo(pCmd); + SQueryInfo *pQueryInfo = tscGetQueryInfo(pCmd); STableMetaInfo *pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0); SSubqueryState *pState = &pSql->subState; @@ -2448,9 +2447,9 @@ int32_t tscHandleMasterSTableQuery(SSqlObj *pSql) { assert(pState->numOfSub > 0); - int32_t ret = tscLocalReducerEnvCreate(pSql, &pMemoryBuf, &pDesc, &pModel, &pFinalModel, nBufferSize); + int32_t ret = tscLocalReducerEnvCreate(pQueryInfo, &pMemoryBuf, pSql->subState.numOfSub, &pDesc, nBufferSize, pSql->self); if (ret != 0) { - pRes->code = TSDB_CODE_TSC_OUT_OF_MEMORY; + pRes->code = ret; tscAsyncResultOnError(pSql); tfree(pMemoryBuf); return ret; @@ -2461,7 +2460,7 @@ int32_t tscHandleMasterSTableQuery(SSqlObj *pSql) { if (pSql->pSubs == NULL) { tfree(pSql->pSubs); pRes->code = TSDB_CODE_TSC_OUT_OF_MEMORY; - tscLocalReducerEnvDestroy(pMemoryBuf, pDesc, pModel, pFinalModel,pState->numOfSub); + tscLocalReducerEnvDestroy(pMemoryBuf, pDesc,pState->numOfSub); tscAsyncResultOnError(pSql); return ret; @@ -2504,8 +2503,6 @@ int32_t tscHandleMasterSTableQuery(SSqlObj *pSql) { trs->subqueryIndex = i; trs->pParentSql = pSql; - trs->pFinalColModel = pModel; - trs->pFFColModel = pFinalModel; SSqlObj *pNew = tscCreateSTableSubquery(pSql, trs, NULL); if (pNew == NULL) { @@ -2517,7 +2514,7 @@ int32_t tscHandleMasterSTableQuery(SSqlObj *pSql) { // todo handle multi-vnode situation if (pQueryInfo->tsBuf) { - SQueryInfo *pNewQueryInfo = tscGetQueryInfo(&pNew->cmd, 0); + SQueryInfo *pNewQueryInfo = tscGetQueryInfo(&pNew->cmd); pNewQueryInfo->tsBuf = tsBufClone(pQueryInfo->tsBuf); assert(pNewQueryInfo->tsBuf != NULL); } @@ -2530,13 +2527,13 @@ int32_t tscHandleMasterSTableQuery(SSqlObj *pSql) { tscError("0x%"PRIx64" failed to prepare subquery structure and launch subqueries", pSql->self); pRes->code = TSDB_CODE_TSC_OUT_OF_MEMORY; - tscLocalReducerEnvDestroy(pMemoryBuf, pDesc, pModel, pFinalModel, pState->numOfSub); + tscLocalReducerEnvDestroy(pMemoryBuf, pDesc, pState->numOfSub); doCleanupSubqueries(pSql, i); return pRes->code; // free all allocated resource } if (pRes->code == TSDB_CODE_TSC_QUERY_CANCELLED) { - tscLocalReducerEnvDestroy(pMemoryBuf, pDesc, pModel, pFinalModel, pState->numOfSub); + tscLocalReducerEnvDestroy(pMemoryBuf, pDesc, pState->numOfSub); doCleanupSubqueries(pSql, i); return pRes->code; } @@ -2604,7 +2601,7 @@ static int32_t tscReissueSubquery(SRetrieveSupport *oriTrs, SSqlObj *pSql, int32 SSqlObj *pParentSql = trsupport->pParentSql; int32_t subqueryIndex = trsupport->subqueryIndex; - STableMetaInfo* pTableMetaInfo = tscGetTableMetaInfoFromCmd(&pSql->cmd, 0, 0); + STableMetaInfo* pTableMetaInfo = tscGetTableMetaInfoFromCmd(&pSql->cmd, 0); SVgroupInfo* pVgroup = &pTableMetaInfo->vgroupList->vgroups[0]; tExtMemBufferClear(trsupport->pExtMemBuffer[subqueryIndex]); @@ -2701,13 +2698,13 @@ void tscHandleSubqueryError(SRetrieveSupport *trsupport, SSqlObj *pSql, int numO tstrerror(pParentSql->res.code)); // release allocated resource - tscLocalReducerEnvDestroy(trsupport->pExtMemBuffer, trsupport->pOrderDescriptor, trsupport->pFinalColModel, trsupport->pFFColModel, + tscLocalReducerEnvDestroy(trsupport->pExtMemBuffer, trsupport->pOrderDescriptor, pState->numOfSub); tscFreeRetrieveSup(pSql); // in case of second stage join subquery, invoke its callback function instead of regular QueueAsyncRes - SQueryInfo *pQueryInfo = tscGetQueryInfo(&pParentSql->cmd, 0); + SQueryInfo *pQueryInfo = tscGetQueryInfo(&pParentSql->cmd); if (!TSDB_QUERY_HAS_TYPE(pQueryInfo->type, TSDB_QUERY_TYPE_JOIN_SEC_STAGE)) { (*pParentSql->fp)(pParentSql->param, pParentSql, pParentSql->res.code); @@ -2724,7 +2721,7 @@ static void tscAllDataRetrievedFromDnode(SRetrieveSupport *trsupport, SSqlObj* p tOrderDescriptor *pDesc = trsupport->pOrderDescriptor; SSubqueryState* pState = &pParentSql->subState; - SQueryInfo *pQueryInfo = tscGetQueryInfo(&pSql->cmd, 0); + SQueryInfo *pQueryInfo = tscGetQueryInfo(&pSql->cmd); STableMetaInfo* pTableMetaInfo = pQueryInfo->pTableMetaInfo[0]; @@ -2773,22 +2770,28 @@ static void tscAllDataRetrievedFromDnode(SRetrieveSupport *trsupport, SSqlObj* p tscDebug("0x%"PRIx64" retrieve from %d vnodes completed.final NumOfRows:%" PRId64 ",start to build loser tree", pParentSql->self, pState->numOfSub, pState->numOfRetrievedRows); - SQueryInfo *pPQueryInfo = tscGetQueryInfo(&pParentSql->cmd, 0); + SQueryInfo *pPQueryInfo = tscGetQueryInfo(&pParentSql->cmd); tscClearInterpInfo(pPQueryInfo); - tscCreateLocalMerger(trsupport->pExtMemBuffer, pState->numOfSub, pDesc, trsupport->pFinalColModel, trsupport->pFFColModel, pParentSql); + code = tscCreateLocalMerger(trsupport->pExtMemBuffer, pState->numOfSub, pDesc, pPQueryInfo, &pParentSql->res.pLocalMerger, pParentSql->self); + pParentSql->res.code = code; + + if (code == TSDB_CODE_SUCCESS && trsupport->pExtMemBuffer == NULL) { + pParentSql->cmd.command = TSDB_SQL_RETRIEVE_EMPTY_RESULT; // no result, set the result empty + } else { + pParentSql->cmd.command = TSDB_SQL_RETRIEVE_LOCALMERGE; + } + + tscCreateResPointerInfo(&pParentSql->res, pPQueryInfo); + tscDebug("0x%"PRIx64" build loser tree completed", pParentSql->self); pParentSql->res.precision = pSql->res.precision; pParentSql->res.numOfRows = 0; pParentSql->res.row = 0; - - tscFreeRetrieveSup(pSql); + pParentSql->res.numOfGroups = 0; - // set the command flag must be after the semaphore been correctly set. - if (pParentSql->cmd.command != TSDB_SQL_RETRIEVE_EMPTY_RESULT) { - pParentSql->cmd.command = TSDB_SQL_RETRIEVE_LOCALMERGE; - } + tscFreeRetrieveSup(pSql); if (pParentSql->res.code == TSDB_CODE_SUCCESS) { (*pParentSql->fp)(pParentSql->param, pParentSql, 0); @@ -2814,7 +2817,7 @@ static void tscRetrieveFromDnodeCallBack(void *param, TAOS_RES *tres, int numOfR SSubqueryState* pState = &pParentSql->subState; - STableMetaInfo *pTableMetaInfo = tscGetTableMetaInfoFromCmd(&pSql->cmd, 0, 0); + STableMetaInfo *pTableMetaInfo = tscGetTableMetaInfoFromCmd(&pSql->cmd, 0); SVgroupInfo *pVgroup = &pTableMetaInfo->vgroupList->vgroups[0]; if (pParentSql->res.code != TSDB_CODE_SUCCESS) { @@ -2852,7 +2855,7 @@ static void tscRetrieveFromDnodeCallBack(void *param, TAOS_RES *tres, int numOfR } SSqlRes * pRes = &pSql->res; - SQueryInfo *pQueryInfo = tscGetQueryInfo(&pSql->cmd, 0); + SQueryInfo *pQueryInfo = tscGetQueryInfo(&pSql->cmd); if (numOfRows > 0) { assert(pRes->numOfRows == numOfRows); @@ -2861,7 +2864,7 @@ static void tscRetrieveFromDnodeCallBack(void *param, TAOS_RES *tres, int numOfR tscDebug("0x%"PRIx64" sub:%p retrieve numOfRows:%d totalNumOfRows:%" PRIu64 " from ep:%s, orderOfSub:%d", pParentSql->self, pSql, pRes->numOfRows, pState->numOfRetrievedRows, pSql->epSet.fqdn[pSql->epSet.inUse], idx); - if (num > tsMaxNumOfOrderedResults && tscIsProjectionQueryOnSTable(pQueryInfo, 0) && !(tscGetQueryInfo(&pParentSql->cmd, 0)->distinctTag)) { + if (num > tsMaxNumOfOrderedResults && tscIsProjectionQueryOnSTable(pQueryInfo, 0) && !(tscGetQueryInfo(&pParentSql->cmd)->distinctTag)) { tscError("0x%"PRIx64" sub:0x%"PRIx64" num of OrderedRes is too many, max allowed:%" PRId32 " , current:%" PRId64, pParentSql->self, pSql->self, tsMaxNumOfOrderedResults, num); tscAbortFurtherRetryRetrieval(trsupport, tres, TSDB_CODE_TSC_SORTED_RES_TOO_MANY); @@ -2904,7 +2907,7 @@ static SSqlObj *tscCreateSTableSubquery(SSqlObj *pSql, SRetrieveSupport *trsuppo SSqlObj *pNew = createSubqueryObj(pSql, table_index, tscRetrieveDataRes, trsupport, TSDB_SQL_SELECT, prevSqlObj); if (pNew != NULL) { // the sub query of two-stage super table query - SQueryInfo *pQueryInfo = tscGetQueryInfo(&pNew->cmd, 0); + SQueryInfo *pQueryInfo = tscGetQueryInfo(&pNew->cmd); pNew->cmd.active = pQueryInfo; pQueryInfo->type |= TSDB_QUERY_TYPE_STABLE_SUBQUERY; @@ -2913,7 +2916,7 @@ static SSqlObj *tscCreateSTableSubquery(SSqlObj *pSql, SRetrieveSupport *trsuppo pQueryInfo->limit.limit = -1; pQueryInfo->limit.offset = 0; - assert(pQueryInfo->numOfTables == 1 && pNew->cmd.numOfClause == 1 && trsupport->subqueryIndex < pSql->subState.numOfSub); + assert(trsupport->subqueryIndex < pSql->subState.numOfSub); // launch subquery for each vnode, so the subquery index equals to the vgroupIndex. STableMetaInfo *pTableMetaInfo = tscGetMetaInfo(pQueryInfo, table_index); @@ -2939,10 +2942,10 @@ void tscRetrieveDataRes(void *param, TAOS_RES *tres, int code) { SSqlObj* pParentSql = trsupport->pParentSql; SSqlObj* pSql = (SSqlObj *) tres; - SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd, 0); - assert(pSql->cmd.numOfClause == 1 && pQueryInfo->numOfTables == 1); + SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd); + assert(pQueryInfo->numOfTables == 1); - STableMetaInfo *pTableMetaInfo = tscGetTableMetaInfoFromCmd(&pSql->cmd, 0, 0); + STableMetaInfo *pTableMetaInfo = tscGetTableMetaInfoFromCmd(&pSql->cmd, 0); SVgroupInfo* pVgroup = &pTableMetaInfo->vgroupList->vgroups[trsupport->subqueryIndex]; // stable query killed or other subquery failed, all query stopped @@ -2969,7 +2972,6 @@ void tscRetrieveDataRes(void *param, TAOS_RES *tres, int code) { tscError("0x%"PRIx64" sub:0x%"PRIx64" failed code:%s, retry:%d", pParentSql->self, pSql->self, tstrerror(code), trsupport->numOfRetry); int32_t sent = 0; - tscReissueSubquery(trsupport, pSql, code, &sent); if (sent) { return; @@ -3041,8 +3043,8 @@ static void multiVnodeInsertFinalize(void* param, TAOS_RES* tres, int numOfRows) pParentObj->res.code = pSql->res.code; // set the flag in the parent sqlObj - if (pSql->cmd.submitSchema) { - pParentObj->cmd.submitSchema = 1; + if (pSql->cmd.insertParam.schemaAttached) { + pParentObj->cmd.insertParam.schemaAttached = 1; } } @@ -3077,8 +3079,8 @@ static void multiVnodeInsertFinalize(void* param, TAOS_RES* tres, int numOfRows) // clean up tableMeta in cache tscFreeQueryInfo(&pSql->cmd, false); - SQueryInfo* pQueryInfo = tscGetQueryInfoS(&pSql->cmd, 0); - STableMetaInfo* pMasterTableMetaInfo = tscGetTableMetaInfoFromCmd(&pParentObj->cmd, pSql->cmd.clauseIndex, 0); + SQueryInfo* pQueryInfo = tscGetQueryInfoS(&pSql->cmd); + STableMetaInfo* pMasterTableMetaInfo = tscGetTableMetaInfoFromCmd(&pParentObj->cmd, 0); tscAddTableMetaInfo(pQueryInfo, &pMasterTableMetaInfo->name, NULL, NULL, NULL, NULL); subquerySetState(pSql, &pParentObj->subState, i, 0); @@ -3090,15 +3092,15 @@ static void multiVnodeInsertFinalize(void* param, TAOS_RES* tres, int numOfRows) tscError("0x%"PRIx64" Async insertion completed, total inserted:%d rows, numOfFailed:%d, numOfTotal:%d", pParentObj->self, pParentObj->res.numOfRows, numOfFailed, numOfSub); - tscDebug("0x%"PRIx64" cleanup %d tableMeta in hashTable before reparse sql", pParentObj->self, pParentObj->cmd.numOfTables); - for(int32_t i = 0; i < pParentObj->cmd.numOfTables; ++i) { + tscDebug("0x%"PRIx64" cleanup %d tableMeta in hashTable before reparse sql", pParentObj->self, pParentObj->cmd.insertParam.numOfTables); + for(int32_t i = 0; i < pParentObj->cmd.insertParam.numOfTables; ++i) { char name[TSDB_TABLE_FNAME_LEN] = {0}; - tNameExtractFullName(pParentObj->cmd.pTableNameList[i], name); + tNameExtractFullName(pParentObj->cmd.insertParam.pTableNameList[i], name); taosHashRemove(tscTableMetaInfo, name, strnlen(name, TSDB_TABLE_FNAME_LEN)); } pParentObj->res.code = TSDB_CODE_SUCCESS; - pParentObj->cmd.parseFinished = false; +// pParentObj->cmd.parseFinished = false; tscResetSqlCmd(&pParentObj->cmd, false); @@ -3133,7 +3135,7 @@ int32_t tscHandleInsertRetry(SSqlObj* pParent, SSqlObj* pSql) { SInsertSupporter* pSupporter = (SInsertSupporter*) pSql->param; assert(pSupporter->index < pSupporter->pSql->subState.numOfSub); - STableDataBlocks* pTableDataBlock = taosArrayGetP(pParent->cmd.pDataBlocks, pSupporter->index); + STableDataBlocks* pTableDataBlock = taosArrayGetP(pParent->cmd.insertParam.pDataBlocks, pSupporter->index); int32_t code = tscCopyDataBlockToPayload(pSql, pTableDataBlock); if ((pRes->code = code)!= TSDB_CODE_SUCCESS) { @@ -3166,7 +3168,7 @@ int32_t tscHandleMultivnodeInsert(SSqlObj *pSql) { return TSDB_CODE_SUCCESS; } - pSql->subState.numOfSub = (uint16_t)taosArrayGetSize(pCmd->pDataBlocks); + pSql->subState.numOfSub = (uint16_t)taosArrayGetSize(pCmd->insertParam.pDataBlocks); assert(pSql->subState.numOfSub > 0); pRes->code = TSDB_CODE_SUCCESS; @@ -3216,7 +3218,7 @@ int32_t tscHandleMultivnodeInsert(SSqlObj *pSql) { pNew->fetchFp = pNew->fp; pSql->pSubs[numOfSub] = pNew; - STableDataBlocks* pTableDataBlock = taosArrayGetP(pCmd->pDataBlocks, numOfSub); + STableDataBlocks* pTableDataBlock = taosArrayGetP(pCmd->insertParam.pDataBlocks, numOfSub); pRes->code = tscCopyDataBlockToPayload(pNew, pTableDataBlock); if (pRes->code == TSDB_CODE_SUCCESS) { tscDebug("0x%"PRIx64" sub:%p create subObj success. orderOfSub:%d", pSql->self, pNew, numOfSub); @@ -3234,7 +3236,7 @@ int32_t tscHandleMultivnodeInsert(SSqlObj *pSql) { goto _error; } - pCmd->pDataBlocks = tscDestroyBlockArrayList(pCmd->pDataBlocks); + pCmd->insertParam.pDataBlocks = tscDestroyBlockArrayList(pCmd->insertParam.pDataBlocks); // use the local variable for (int32_t j = 0; j < numOfSub; ++j) { @@ -3250,7 +3252,7 @@ int32_t tscHandleMultivnodeInsert(SSqlObj *pSql) { } static char* getResultBlockPosition(SSqlCmd* pCmd, SSqlRes* pRes, int32_t columnIndex, int16_t* bytes) { - SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd, pCmd->clauseIndex); + SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd); SInternalField* pInfo = (SInternalField*) TARRAY_GET_ELEM(pQueryInfo->fieldsInfo.internalField, columnIndex); assert(pInfo->pExpr->pExpr == NULL); @@ -3266,7 +3268,7 @@ static char* getResultBlockPosition(SSqlCmd* pCmd, SSqlRes* pRes, int32_t column static void doBuildResFromSubqueries(SSqlObj* pSql) { SSqlRes* pRes = &pSql->res; - SQueryInfo *pQueryInfo = tscGetQueryInfo(&pSql->cmd, pSql->cmd.clauseIndex); + SQueryInfo *pQueryInfo = tscGetQueryInfo(&pSql->cmd); int32_t numOfRes = INT32_MAX; for (int32_t i = 0; i < pSql->subState.numOfSub; ++i) { @@ -3284,7 +3286,7 @@ static void doBuildResFromSubqueries(SSqlObj* pSql) { return; } - tscRestoreFuncForSTableQuery(pQueryInfo); +// tscRestoreFuncForSTableQuery(pQueryInfo); int32_t rowSize = tscGetResRowLength(pQueryInfo->exprList); assert(numOfRes * rowSize > 0); @@ -3312,12 +3314,12 @@ static void doBuildResFromSubqueries(SSqlObj* pSql) { continue; } - SQueryInfo* pSubQueryInfo = pSub->cmd.pQueryInfo[0]; + SQueryInfo* pSubQueryInfo = pSub->cmd.pQueryInfo; tscRestoreFuncForSTableQuery(pSubQueryInfo); tscFieldInfoUpdateOffset(pSubQueryInfo); } - size_t numOfExprs = tscSqlExprNumOfExprs(pQueryInfo); + size_t numOfExprs = tscNumOfExprs(pQueryInfo); for(int32_t i = 0; i < numOfExprs; ++i) { SColumnIndex* pIndex = &pRes->pColumnIndex[i]; SSqlRes* pRes1 = &pSql->pSubs[pIndex->tableIndex]->res; @@ -3363,8 +3365,8 @@ void tscBuildResFromSubqueries(SSqlObj *pSql) { } if (pRes->tsrow == NULL) { - SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd, pSql->cmd.clauseIndex); - pRes->numOfCols = (int16_t) tscSqlExprNumOfExprs(pQueryInfo); + SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSql->cmd); + pRes->numOfCols = (int16_t) tscNumOfExprs(pQueryInfo); pRes->tsrow = calloc(pRes->numOfCols, POINTER_BYTES); pRes->urow = calloc(pRes->numOfCols, POINTER_BYTES); @@ -3376,8 +3378,6 @@ void tscBuildResFromSubqueries(SSqlObj *pSql) { tscAsyncResultOnError(pSql); return; } - - tscRestoreFuncForSTableQuery(pQueryInfo); } assert (pRes->row >= pRes->numOfRows); @@ -3417,7 +3417,7 @@ TAOS_ROW doSetResultRowData(SSqlObj *pSql) { return pRes->tsrow; } - SQueryInfo *pQueryInfo = tscGetQueryInfo(pCmd, pCmd->clauseIndex); + SQueryInfo *pQueryInfo = tscGetQueryInfo(pCmd); size_t size = tscNumOfFields(pQueryInfo); @@ -3450,7 +3450,7 @@ static UNUSED_FUNC bool tscHasRemainDataInSubqueryResultSet(SSqlObj *pSql) { bool hasData = true; SSqlCmd *pCmd = &pSql->cmd; - SQueryInfo *pQueryInfo = tscGetQueryInfo(pCmd, pCmd->clauseIndex); + SQueryInfo *pQueryInfo = tscGetQueryInfo(pCmd); if (tscNonOrderedProjectionQueryOnSTable(pQueryInfo, 0)) { bool allSubqueryExhausted = true; @@ -3462,7 +3462,7 @@ static UNUSED_FUNC bool tscHasRemainDataInSubqueryResultSet(SSqlObj *pSql) { SSqlRes *pRes1 = &pSql->pSubs[i]->res; SSqlCmd *pCmd1 = &pSql->pSubs[i]->cmd; - SQueryInfo *pQueryInfo1 = tscGetQueryInfo(pCmd1, pCmd1->clauseIndex); + SQueryInfo *pQueryInfo1 = tscGetQueryInfo(pCmd1); assert(pQueryInfo1->numOfTables == 1); STableMetaInfo *pTableMetaInfo = tscGetMetaInfo(pQueryInfo1, 0); @@ -3486,7 +3486,7 @@ static UNUSED_FUNC bool tscHasRemainDataInSubqueryResultSet(SSqlObj *pSql) { } SSqlRes * pRes1 = &pSql->pSubs[i]->res; - SQueryInfo *pQueryInfo1 = tscGetQueryInfo(&pSql->pSubs[i]->cmd, 0); + SQueryInfo *pQueryInfo1 = tscGetQueryInfo(&pSql->pSubs[i]->cmd); if ((pRes1->row >= pRes1->numOfRows && tscHasReachLimitation(pQueryInfo1, pRes1) && tscIsProjectionQuery(pQueryInfo1)) || @@ -3500,11 +3500,10 @@ static UNUSED_FUNC bool tscHasRemainDataInSubqueryResultSet(SSqlObj *pSql) { return hasData; } -void* createQueryInfoFromQueryNode(SQueryInfo* pQueryInfo, SExprInfo* pExprs, STableGroupInfo* pTableGroupInfo, - SOperatorInfo* pSourceOperator, char* sql, void* merger, int32_t stage) { +// todo remove pExprs +void* createQInfoFromQueryNode(SQueryInfo* pQueryInfo, SExprInfo* pExprs, STableGroupInfo* pTableGroupInfo, + SOperatorInfo* pSourceOperator, char* sql, void* merger, int32_t stage) { assert(pQueryInfo != NULL); - int16_t numOfOutput = pQueryInfo->fieldsInfo.numOfOutput; - SQInfo *pQInfo = (SQInfo *)calloc(1, sizeof(SQInfo)); if (pQInfo == NULL) { goto _cleanup; @@ -3522,13 +3521,25 @@ void* createQueryInfoFromQueryNode(SQueryInfo* pQueryInfo, SExprInfo* pExprs, ST pQueryAttr->tableGroupInfo = *pTableGroupInfo; // calculate the result row size - for (int16_t col = 0; col < numOfOutput; ++col) { - assert(pExprs[col].base.resBytes > 0); - pQueryAttr->resultRowSize += pExprs[col].base.resBytes; + SExprInfo* pEx = NULL; + int32_t num = 0; + if (pQueryAttr->pExpr3 != NULL) { + pEx = pQueryAttr->pExpr3; + num = pQueryAttr->numOfExpr3; + } else if (pQueryAttr->pExpr2 != NULL) { + pEx = pQueryAttr->pExpr2; + num = pQueryAttr->numOfExpr2; + } else { + pEx = pQueryAttr->pExpr1; + num = pQueryAttr->numOfOutput; + } + + for (int16_t col = 0; col < num; ++col) { + pQueryAttr->resultRowSize += pEx[col].base.resBytes; // keep the tag length - if (TSDB_COL_IS_TAG(pExprs[col].base.colInfo.flag)) { - pQueryAttr->tagLen += pExprs[col].base.resBytes; + if (TSDB_COL_IS_TAG(pEx[col].base.colInfo.flag)) { + pQueryAttr->tagLen += pEx[col].base.resBytes; } } @@ -3586,15 +3597,9 @@ void* createQueryInfoFromQueryNode(SQueryInfo* pQueryInfo, SExprInfo* pExprs, ST } } - for (int32_t i = 0; i < numOfOutput; ++i) { - SExprInfo* pExprInfo = &pExprs[i]; - if (pExprInfo->pExpr != NULL) { - tExprTreeDestroy(pExprInfo->pExpr, NULL); - pExprInfo->pExpr = NULL; - } - } - - tfree(pExprs); + // todo refactor: filter should not be applied here. + createFilterInfo(pQueryAttr, 0); + pQueryAttr->numOfFilterCols = 0; SArray* pa = NULL; if (stage == MASTER_SCAN) { diff --git a/src/client/src/tscUtil.c b/src/client/src/tscUtil.c index 0ecfbe229a4bdb72f2fb26fdb99cc8ea4b3befb2..4596846d4c79f8d09eff0124c8b159efd0c7ae2c 100644 --- a/src/client/src/tscUtil.c +++ b/src/client/src/tscUtil.c @@ -14,10 +14,11 @@ */ #include "tscUtil.h" +#include "tsched.h" #include "hash.h" #include "os.h" -#include "texpr.h" #include "taosmsg.h" +#include "texpr.h" #include "tkey.h" #include "tmd5.h" #include "tscLocalMerge.h" @@ -30,7 +31,6 @@ #include "ttokendef.h" static void freeQueryInfoImpl(SQueryInfo* pQueryInfo); -static void clearAllTableMetaInfo(SQueryInfo* pQueryInfo, bool removeMeta); int32_t converToStr(char *str, int type, void *buf, int32_t bufSize, int32_t *len) { int32_t n = 0; @@ -139,10 +139,10 @@ void tsSetSTableQueryCond(STagCond* pTagCond, uint64_t uid, SBufferWriter* bw) { } bool tscQueryTags(SQueryInfo* pQueryInfo) { - int32_t numOfCols = (int32_t) tscSqlExprNumOfExprs(pQueryInfo); + int32_t numOfCols = (int32_t) tscNumOfExprs(pQueryInfo); for (int32_t i = 0; i < numOfCols; ++i) { - SExprInfo* pExpr = tscSqlExprGet(pQueryInfo, i); + SExprInfo* pExpr = tscExprGet(pQueryInfo, i); int32_t functId = pExpr->base.functionId; // "select count(tbname)" query @@ -159,10 +159,10 @@ bool tscQueryTags(SQueryInfo* pQueryInfo) { } bool tscQueryBlockInfo(SQueryInfo* pQueryInfo) { - int32_t numOfCols = (int32_t) tscSqlExprNumOfExprs(pQueryInfo); + int32_t numOfCols = (int32_t) tscNumOfExprs(pQueryInfo); for (int32_t i = 0; i < numOfCols; ++i) { - SExprInfo* pExpr = tscSqlExprGet(pQueryInfo, i); + SExprInfo* pExpr = tscExprGet(pQueryInfo, i); int32_t functId = pExpr->base.functionId; if (functId == TSDB_FUNC_BLKINFO) { @@ -207,14 +207,14 @@ bool tscIsProjectionQueryOnSTable(SQueryInfo* pQueryInfo, int32_t tableIndex) { * 1. failed to get tableMeta from server; 2. not a super table; 3. limitation is 0; * 4. show queries, instead of a select query */ - size_t numOfExprs = tscSqlExprNumOfExprs(pQueryInfo); + size_t numOfExprs = tscNumOfExprs(pQueryInfo); if (pTableMetaInfo == NULL || !UTIL_TABLE_IS_SUPER_TABLE(pTableMetaInfo) || pQueryInfo->command == TSDB_SQL_RETRIEVE_EMPTY_RESULT || numOfExprs == 0) { return false; } for (int32_t i = 0; i < numOfExprs; ++i) { - int32_t functionId = tscSqlExprGet(pQueryInfo, i)->base.functionId; + int32_t functionId = tscExprGet(pQueryInfo, i)->base.functionId; if (functionId != TSDB_FUNC_PRJ && functionId != TSDB_FUNC_TAGPRJ && @@ -250,10 +250,10 @@ bool tscOrderedProjectionQueryOnSTable(SQueryInfo* pQueryInfo, int32_t tableInde } bool tscIsProjectionQuery(SQueryInfo* pQueryInfo) { - size_t size = tscSqlExprNumOfExprs(pQueryInfo); + size_t size = tscNumOfExprs(pQueryInfo); for (int32_t i = 0; i < size; ++i) { - int32_t functionId = tscSqlExprGet(pQueryInfo, i)->base.functionId; + int32_t functionId = tscExprGet(pQueryInfo, i)->base.functionId; if (functionId != TSDB_FUNC_PRJ && functionId != TSDB_FUNC_TAGPRJ && functionId != TSDB_FUNC_TAG && functionId != TSDB_FUNC_TS && functionId != TSDB_FUNC_ARITHM) { @@ -264,10 +264,27 @@ bool tscIsProjectionQuery(SQueryInfo* pQueryInfo) { return true; } +bool tscHasColumnFilter(SQueryInfo* pQueryInfo) { + // filter on primary timestamp column + if (pQueryInfo->window.skey != INT64_MIN || pQueryInfo->window.ekey != INT64_MAX) { + return true; + } + + size_t size = taosArrayGetSize(pQueryInfo->colList); + for (int32_t i = 0; i < size; ++i) { + SColumn* pCol = taosArrayGet(pQueryInfo->colList, i); + if (pCol->info.flist.numOfFilters > 0) { + return true; + } + } + + return false; +} + bool tscIsPointInterpQuery(SQueryInfo* pQueryInfo) { - size_t size = tscSqlExprNumOfExprs(pQueryInfo); + size_t size = tscNumOfExprs(pQueryInfo); for (int32_t i = 0; i < size; ++i) { - SExprInfo* pExpr = tscSqlExprGet(pQueryInfo, i); + SExprInfo* pExpr = tscExprGet(pQueryInfo, i); assert(pExpr != NULL); int32_t functionId = pExpr->base.functionId; @@ -283,7 +300,7 @@ bool tscIsPointInterpQuery(SQueryInfo* pQueryInfo) { return true; } -bool tscIsSecondStageQuery(SQueryInfo* pQueryInfo) { +bool tsIsArithmeticQueryOnAggResult(SQueryInfo* pQueryInfo) { if (tscIsProjectionQuery(pQueryInfo)) { return false; } @@ -303,7 +320,7 @@ bool tscGroupbyColumn(SQueryInfo* pQueryInfo) { STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0); int32_t numOfCols = tscGetNumOfColumns(pTableMetaInfo->pTableMeta); - SSqlGroupbyExpr* pGroupbyExpr = &pQueryInfo->groupbyExpr; + SGroupbyExpr* pGroupbyExpr = &pQueryInfo->groupbyExpr; for (int32_t k = 0; k < pGroupbyExpr->numOfGroupCols; ++k) { SColIndex* pIndex = taosArrayGet(pGroupbyExpr->columnInfo, k); if (!TSDB_COL_IS_TAG(pIndex->flag) && pIndex->colIndex < numOfCols) { // group by normal columns @@ -315,10 +332,10 @@ bool tscGroupbyColumn(SQueryInfo* pQueryInfo) { } bool tscIsTopBotQuery(SQueryInfo* pQueryInfo) { - size_t numOfExprs = tscSqlExprNumOfExprs(pQueryInfo); + size_t numOfExprs = tscNumOfExprs(pQueryInfo); for (int32_t i = 0; i < numOfExprs; ++i) { - SExprInfo* pExpr = tscSqlExprGet(pQueryInfo, i); + SExprInfo* pExpr = tscExprGet(pQueryInfo, i); if (pExpr == NULL) { continue; } @@ -336,8 +353,8 @@ bool tscIsTopBotQuery(SQueryInfo* pQueryInfo) { } bool isTsCompQuery(SQueryInfo* pQueryInfo) { - size_t numOfExprs = tscSqlExprNumOfExprs(pQueryInfo); - SExprInfo* pExpr1 = tscSqlExprGet(pQueryInfo, 0); + size_t numOfExprs = tscNumOfExprs(pQueryInfo); + SExprInfo* pExpr1 = tscExprGet(pQueryInfo, 0); if (numOfExprs != 1) { return false; } @@ -346,15 +363,15 @@ bool isTsCompQuery(SQueryInfo* pQueryInfo) { } bool hasTagValOutput(SQueryInfo* pQueryInfo) { - size_t numOfExprs = tscSqlExprNumOfExprs(pQueryInfo); - SExprInfo* pExpr1 = tscSqlExprGet(pQueryInfo, 0); + size_t numOfExprs = tscNumOfExprs(pQueryInfo); + SExprInfo* pExpr1 = tscExprGet(pQueryInfo, 0); if (numOfExprs == 1 && pExpr1->base.functionId == TSDB_FUNC_TS_COMP) { return true; } for (int32_t i = 0; i < numOfExprs; ++i) { - SExprInfo* pExpr = tscSqlExprGet(pQueryInfo, i); + SExprInfo* pExpr = tscExprGet(pQueryInfo, i); if (pExpr == NULL) { continue; } @@ -369,9 +386,9 @@ bool hasTagValOutput(SQueryInfo* pQueryInfo) { } bool timeWindowInterpoRequired(SQueryInfo *pQueryInfo) { - size_t numOfExprs = tscSqlExprNumOfExprs(pQueryInfo); + size_t numOfExprs = tscNumOfExprs(pQueryInfo); for (int32_t i = 0; i < numOfExprs; ++i) { - SExprInfo* pExpr = tscSqlExprGet(pQueryInfo, i); + SExprInfo* pExpr = tscExprGet(pQueryInfo, i); if (pExpr == NULL) { continue; } @@ -386,9 +403,9 @@ bool timeWindowInterpoRequired(SQueryInfo *pQueryInfo) { } bool isStabledev(SQueryInfo* pQueryInfo) { - size_t numOfExprs = tscSqlExprNumOfExprs(pQueryInfo); + size_t numOfExprs = tscNumOfExprs(pQueryInfo); for (int32_t i = 0; i < numOfExprs; ++i) { - SExprInfo* pExpr = tscSqlExprGet(pQueryInfo, i); + SExprInfo* pExpr = tscExprGet(pQueryInfo, i); if (pExpr == NULL) { continue; } @@ -403,9 +420,9 @@ bool isStabledev(SQueryInfo* pQueryInfo) { } bool tscIsTWAQuery(SQueryInfo* pQueryInfo) { - size_t numOfExprs = tscSqlExprNumOfExprs(pQueryInfo); + size_t numOfExprs = tscNumOfExprs(pQueryInfo); for (int32_t i = 0; i < numOfExprs; ++i) { - SExprInfo* pExpr = tscSqlExprGet(pQueryInfo, i); + SExprInfo* pExpr = tscExprGet(pQueryInfo, i); if (pExpr == NULL) { continue; } @@ -419,9 +436,9 @@ bool tscIsTWAQuery(SQueryInfo* pQueryInfo) { } bool tscNeedReverseScan(SQueryInfo* pQueryInfo) { - size_t numOfExprs = tscSqlExprNumOfExprs(pQueryInfo); + size_t numOfExprs = tscNumOfExprs(pQueryInfo); for (int32_t i = 0; i < numOfExprs; ++i) { - SExprInfo* pExpr = tscSqlExprGet(pQueryInfo, i); + SExprInfo* pExpr = tscExprGet(pQueryInfo, i); if (pExpr == NULL) { continue; } @@ -457,9 +474,9 @@ bool isSimpleAggregate(SQueryInfo* pQueryInfo) { return true; } - size_t numOfExprs = tscSqlExprNumOfExprs(pQueryInfo); + size_t numOfExprs = tscNumOfExprs(pQueryInfo); for (int32_t i = 0; i < numOfExprs; ++i) { - SExprInfo* pExpr = tscSqlExprGet(pQueryInfo, i); + SExprInfo* pExpr = tscExprGet(pQueryInfo, i); if (pExpr == NULL) { continue; } @@ -477,9 +494,39 @@ bool isSimpleAggregate(SQueryInfo* pQueryInfo) { return false; } +bool isSimpleAggregateRv(SQueryInfo* pQueryInfo) { + if (pQueryInfo->interval.interval > 0 || pQueryInfo->sessionWindow.gap > 0) { + return false; + } + + if (tscGroupbyColumn(pQueryInfo) || isTsCompQuery(pQueryInfo)) { + return false; + } + + size_t numOfExprs = tscNumOfExprs(pQueryInfo); + for (int32_t i = 0; i < numOfExprs; ++i) { + SExprInfo* pExpr = tscExprGet(pQueryInfo, i); + if (pExpr == NULL) { + continue; + } + + int32_t functionId = pExpr->base.functionId; + if (functionId == TSDB_FUNC_TS || functionId == TSDB_FUNC_TS_DUMMY) { + continue; + } + + if (!IS_MULTIOUTPUT(aAggs[functionId].status)) { + return true; + } + } + + return false; + +} + bool isBlockDistQuery(SQueryInfo* pQueryInfo) { - size_t numOfExprs = tscSqlExprNumOfExprs(pQueryInfo); - SExprInfo* pExpr = tscSqlExprGet(pQueryInfo, 0); + size_t numOfExprs = tscNumOfExprs(pQueryInfo); + SExprInfo* pExpr = tscExprGet(pQueryInfo, 0); return (numOfExprs == 1 && pExpr->base.colInfo.colId == TSDB_BLOCK_DIST_COLUMN_INDEX); } @@ -493,9 +540,9 @@ void tscClearInterpInfo(SQueryInfo* pQueryInfo) { } int32_t tscCreateResPointerInfo(SSqlRes* pRes, SQueryInfo* pQueryInfo) { - if (pRes->tsrow == NULL) { - pRes->numOfCols = pQueryInfo->fieldsInfo.numOfOutput; + pRes->numOfCols = pQueryInfo->fieldsInfo.numOfOutput; + if (pRes->tsrow == NULL) { pRes->tsrow = calloc(pRes->numOfCols, POINTER_BYTES); pRes->urow = calloc(pRes->numOfCols, POINTER_BYTES); pRes->length = calloc(pRes->numOfCols, sizeof(int32_t)); @@ -648,58 +695,237 @@ void tscSetResRawPtrRv(SSqlRes* pRes, SQueryInfo* pQueryInfo, SSDataBlock* pBloc } } -static SColumnInfo* extractColumnInfoFromResult(STableMeta* pTableMeta, SArray* pTableCols) { +static SColumnInfo* extractColumnInfoFromResult(SArray* pTableCols) { int32_t numOfCols = (int32_t) taosArrayGetSize(pTableCols); SColumnInfo* pColInfo = calloc(numOfCols, sizeof(SColumnInfo)); - - SSchema *pSchema = pTableMeta->schema; for(int32_t i = 0; i < numOfCols; ++i) { SColumn* pCol = taosArrayGetP(pTableCols, i); - int32_t index = pCol->columnIndex; - - pColInfo[i].type = pSchema[index].type; - pColInfo[i].bytes = pSchema[index].bytes; - pColInfo[i].colId = pSchema[index].colId; + pColInfo[i] = pCol->info;//[index].type; } return pColInfo; } typedef struct SDummyInputInfo { - SSDataBlock *block; - SSqlRes *pRes; // refactor: remove it + SSDataBlock *block; + SSqlObj *pSql; // refactor: remove it + int32_t numOfFilterCols; + SSingleColumnFilterInfo *pFilterInfo; } SDummyInputInfo; -SSDataBlock* doGetDataBlock(void* param, bool* newgroup) { - SOperatorInfo *pOperator = (SOperatorInfo*) param; - - SDummyInputInfo *pInput = pOperator->info; - char* pData = pInput->pRes->data; +typedef struct SJoinStatus { + SSDataBlock* pBlock; // point to the upstream block + int32_t index; + bool completed;// current upstream is completed or not +} SJoinStatus; - SSDataBlock* pBlock = pInput->block; - pBlock->info.rows = pInput->pRes->numOfRows; - if (pBlock->info.rows == 0) { - return NULL; - } +typedef struct SJoinOperatorInfo { + SSDataBlock *pRes; + SJoinStatus *status; + int32_t numOfUpstream; + SRspResultInfo resultInfo; // todo refactor, add this info for each operator +} SJoinOperatorInfo; - //TODO refactor +static void doSetupSDataBlock(SSqlRes* pRes, SSDataBlock* pBlock) { int32_t offset = 0; + char* pData = pRes->data; for(int32_t i = 0; i < pBlock->info.numOfCols; ++i) { SColumnInfoData* pColData = taosArrayGet(pBlock->pDataBlock, i); if (pData != NULL) { pColData->pData = pData + offset * pBlock->info.rows; } else { - pColData->pData = pInput->pRes->urow[i]; + pColData->pData = pRes->urow[i]; } offset += pColData->info.bytes; } - pInput->pRes->numOfRows = 0; + pRes->numOfRows = 0; +} + +// NOTE: there is already exists data blocks before this function calls. +SSDataBlock* doGetDataBlock(void* param, bool* newgroup) { + SOperatorInfo *pOperator = (SOperatorInfo*) param; + if (pOperator->status == OP_EXEC_DONE) { + return NULL; + } + + SDummyInputInfo *pInput = pOperator->info; + SSqlObj* pSql = pInput->pSql; + SSqlRes* pRes = &pSql->res; + + SSDataBlock* pBlock = pInput->block; + + pBlock->info.rows = pRes->numOfRows; + if (pRes->numOfRows != 0) { + doSetupSDataBlock(pRes, pBlock); + + if (pInput->numOfFilterCols > 0) { + doSetFilterColumnInfo(pInput->pFilterInfo, pInput->numOfFilterCols, pBlock); + int8_t* p = calloc(pBlock->info.rows, sizeof(int8_t)); + bool all = doFilterDataBlock(pInput->pFilterInfo, pInput->numOfFilterCols, pBlock->info.rows, p); + if (!all) { + doCompactSDataBlock(pBlock, pBlock->info.rows, p); + } + + tfree(p); + } + + *newgroup = false; + return pBlock; + } + + // No data block exists. So retrieve and transfer it into to SSDataBlock + TAOS_ROW pRow = NULL; + taos_fetch_block(pSql, &pRow); + + if (pRes->numOfRows == 0) { + pOperator->status = OP_EXEC_DONE; + return NULL; + } + + pBlock->info.rows = pRes->numOfRows; + doSetupSDataBlock(pRes, pBlock); *newgroup = false; return pBlock; } +SSDataBlock* doDataBlockJoin(void* param, bool* newgroup) { + SOperatorInfo *pOperator = (SOperatorInfo*) param; + if (pOperator->status == OP_EXEC_DONE) { + return NULL; + } + + assert(pOperator->numOfUpstream > 1); + + SJoinOperatorInfo* pJoinInfo = pOperator->info; + pJoinInfo->pRes->info.rows = 0; + + while(1) { + for (int32_t i = 0; i < pOperator->numOfUpstream; ++i) { + SJoinStatus* pStatus = &pJoinInfo->status[i]; + if (pStatus->pBlock == NULL || pStatus->index >= pStatus->pBlock->info.rows) { + pStatus->pBlock = pOperator->upstream[i]->exec(pOperator->upstream[i], newgroup); + pStatus->index = 0; + + if (pStatus->pBlock == NULL) { + pOperator->status = OP_EXEC_DONE; + + pJoinInfo->resultInfo.total += pJoinInfo->pRes->info.rows; + return pJoinInfo->pRes; + } + } + } + + SJoinStatus* st0 = &pJoinInfo->status[0]; + SColumnInfoData* p0 = taosArrayGet(st0->pBlock->pDataBlock, 0); + int64_t* ts0 = (int64_t*) p0->pData; + + bool prefixEqual = true; + + while(1) { + prefixEqual = true; + for (int32_t i = 1; i < pJoinInfo->numOfUpstream; ++i) { + SJoinStatus* st = &pJoinInfo->status[i]; + + SColumnInfoData* p = taosArrayGet(st->pBlock->pDataBlock, 0); + int64_t* ts = (int64_t*)p->pData; + + if (ts[st->index] < ts0[st0->index]) { // less than the first + prefixEqual = false; + if ((++(st->index)) >= st->pBlock->info.rows) { + break; + } + } else if (ts[st->index] > ts0[st0->index]) { // greater than the first; + if (prefixEqual == true) { + prefixEqual = false; + for (int32_t j = 0; j < i; ++j) { + SJoinStatus* stx = &pJoinInfo->status[j]; + if ((++(stx->index)) >= stx->pBlock->info.rows) { + break; + } + } + } else { + if ((++(st0->index)) >= st0->pBlock->info.rows) { + break; + } + } + } + } + + if (prefixEqual) { + int32_t offset = 0; + bool completed = false; + for (int32_t i = 0; i < pOperator->numOfUpstream; ++i) { + SJoinStatus* st1 = &pJoinInfo->status[i]; + int32_t rows = pJoinInfo->pRes->info.rows; + + for (int32_t j = 0; j < st1->pBlock->info.numOfCols; ++j) { + SColumnInfoData* pCol1 = taosArrayGet(pJoinInfo->pRes->pDataBlock, j + offset); + SColumnInfoData* pSrc = taosArrayGet(st1->pBlock->pDataBlock, j); + + int32_t bytes = pSrc->info.bytes; + memcpy(pCol1->pData + rows * bytes, pSrc->pData + st1->index * bytes, bytes); + } + + offset += st1->pBlock->info.numOfCols; + if ((++(st1->index)) == st1->pBlock->info.rows) { + completed = true; + } + } + + if ((++pJoinInfo->pRes->info.rows) >= pJoinInfo->resultInfo.capacity) { + pJoinInfo->resultInfo.total += pJoinInfo->pRes->info.rows; + return pJoinInfo->pRes; + } + + if (completed == true) { + break; + } + } + } +/* + while (st0->index < st0->pBlock->info.rows && st1->index < st1->pBlock->info.rows) { + SColumnInfoData* p0 = taosArrayGet(st0->pBlock->pDataBlock, 0); + SColumnInfoData* p1 = taosArrayGet(st1->pBlock->pDataBlock, 0); + + int64_t* ts0 = (int64_t*)p0->pData; + int64_t* ts1 = (int64_t*)p1->pData; + if (ts0[st0->index] == ts1[st1->index]) { // add to the final result buffer + // check if current output buffer is over the threshold to pause current loop + int32_t rows = pJoinInfo->pRes->info.rows; + for (int32_t j = 0; j < st0->pBlock->info.numOfCols; ++j) { + SColumnInfoData* pCol1 = taosArrayGet(pJoinInfo->pRes->pDataBlock, j); + SColumnInfoData* pSrc = taosArrayGet(st0->pBlock->pDataBlock, j); + + int32_t bytes = pSrc->info.bytes; + memcpy(pCol1->pData + rows * bytes, pSrc->pData + st0->index * bytes, bytes); + } + + for (int32_t j = 0; j < st1->pBlock->info.numOfCols; ++j) { + SColumnInfoData* pCol1 = taosArrayGet(pJoinInfo->pRes->pDataBlock, j + st0->pBlock->info.numOfCols); + SColumnInfoData* pSrc = taosArrayGet(st1->pBlock->pDataBlock, j); + + int32_t bytes = pSrc->info.bytes; + memcpy(pCol1->pData + rows * bytes, pSrc->pData + st1->index * bytes, bytes); + } + + st0->index++; + st1->index++; + + if ((++pJoinInfo->pRes->info.rows) >= pJoinInfo->resultInfo.capacity) { + pJoinInfo->resultInfo.total += pJoinInfo->pRes->info.rows; + return pJoinInfo->pRes; + } + } else if (ts0[st0->index] < ts1[st1->index]) { + st0->index++; + } else { + st1->index++; + } + }*/ + } +} + static void destroyDummyInputOperator(void* param, int32_t numOfOutput) { SDummyInputInfo* pInfo = (SDummyInputInfo*) param; @@ -710,15 +936,17 @@ static void destroyDummyInputOperator(void* param, int32_t numOfOutput) { } pInfo->block = destroyOutputBuf(pInfo->block); - pInfo->pRes = NULL; + pInfo->pSql = NULL; } // todo this operator servers as the adapter for Operator tree and SqlRes result, remove it later -SOperatorInfo* createDummyInputOperator(char* pResult, SSchema* pSchema, int32_t numOfCols) { +SOperatorInfo* createDummyInputOperator(SSqlObj* pSql, SSchema* pSchema, int32_t numOfCols, SSingleColumnFilterInfo* pFilterInfo, int32_t numOfFilterCols) { assert(numOfCols > 0); SDummyInputInfo* pInfo = calloc(1, sizeof(SDummyInputInfo)); - pInfo->pRes = (SSqlRes*) pResult; + pInfo->pSql = pSql; + pInfo->pFilterInfo = pFilterInfo; + pInfo->numOfFilterCols = numOfFilterCols; pInfo->block = calloc(numOfCols, sizeof(SSDataBlock)); pInfo->block->info.numOfCols = numOfCols; @@ -744,6 +972,52 @@ SOperatorInfo* createDummyInputOperator(char* pResult, SSchema* pSchema, int32_t return pOptr; } +static void destroyJoinOperator(void* param, int32_t numOfOutput) { + SJoinOperatorInfo* pInfo = (SJoinOperatorInfo*) param; + tfree(pInfo->status); + + pInfo->pRes = destroyOutputBuf(pInfo->pRes); +} + +SOperatorInfo* createJoinOperatorInfo(SOperatorInfo** pUpstream, int32_t numOfUpstream, SSchema* pSchema, int32_t numOfOutput) { + SJoinOperatorInfo* pInfo = calloc(1, sizeof(SJoinOperatorInfo)); + + pInfo->numOfUpstream = numOfUpstream; + pInfo->status = calloc(numOfUpstream, sizeof(SJoinStatus)); + + SRspResultInfo* pResInfo = &pInfo->resultInfo; + pResInfo->capacity = 4096; + pResInfo->threshold = (int32_t) (4096 * 0.8); + + pInfo->pRes = calloc(1, sizeof(SSDataBlock)); + pInfo->pRes->info.numOfCols = numOfOutput; + pInfo->pRes->pDataBlock = taosArrayInit(numOfOutput, sizeof(SColumnInfoData)); + for(int32_t i = 0; i < numOfOutput; ++i) { + SColumnInfoData colData = {{0}}; + colData.info.bytes = pSchema[i].bytes; + colData.info.type = pSchema[i].type; + colData.info.colId = pSchema[i].colId; + colData.pData = calloc(1, colData.info.bytes * pResInfo->capacity); + + taosArrayPush(pInfo->pRes->pDataBlock, &colData); + } + + SOperatorInfo* pOperator = calloc(1, sizeof(SOperatorInfo)); + pOperator->name = "JoinOperator"; + pOperator->operatorType = OP_Join; + pOperator->numOfOutput = numOfOutput; + pOperator->blockingOptr = false; + pOperator->info = pInfo; + pOperator->exec = doDataBlockJoin; + pOperator->cleanup = destroyJoinOperator; + + for(int32_t i = 0; i < numOfUpstream; ++i) { + appendUpstream(pOperator, pUpstream[i]); + } + + return pOperator; +} + void convertQueryResult(SSqlRes* pRes, SQueryInfo* pQueryInfo) { // set the correct result SSDataBlock* p = pQueryInfo->pQInfo->runtimeEnv.outputBuf; @@ -758,18 +1032,26 @@ void convertQueryResult(SSqlRes* pRes, SQueryInfo* pQueryInfo) { pRes->completed = (pRes->numOfRows == 0); } -void handleDownstreamOperator(SSqlRes* pRes, SQueryInfo* pQueryInfo) { - if (pQueryInfo->pDownstream != NULL) { - // handle the following query process - SQueryInfo *px = pQueryInfo->pDownstream; - SColumnInfo* pColumnInfo = extractColumnInfoFromResult(px->pTableMetaInfo[0]->pTableMeta, px->colList); - int32_t numOfOutput = (int32_t) tscSqlExprNumOfExprs(px); +void handleDownstreamOperator(SSqlObj** pSqlObjList, int32_t numOfUpstream, SQueryInfo* px, SSqlRes* pOutput) { + // handle the following query process + if (px->pQInfo == NULL) { + SColumnInfo* pColumnInfo = extractColumnInfoFromResult(px->colList); + + int32_t numOfOutput = (int32_t)tscNumOfExprs(px); + int32_t numOfCols = (int32_t)taosArrayGetSize(px->colList); + + SQueriedTableInfo info = { + .colList = pColumnInfo, + .numOfCols = numOfCols, + }; - int32_t numOfCols = (int32_t) taosArrayGetSize(px->colList); - SQueriedTableInfo info = {.colList = pColumnInfo, .numOfCols = numOfCols,}; SSchema* pSchema = tscGetTableSchema(px->pTableMetaInfo[0]->pTableMeta); - STableGroupInfo tableGroupInfo = {.numOfTables = 1, .pGroupList = taosArrayInit(1, POINTER_BYTES),}; + STableGroupInfo tableGroupInfo = { + .numOfTables = 1, + .pGroupList = taosArrayInit(1, POINTER_BYTES), + }; + tableGroupInfo.map = taosHashInit(1, taosGetDefaultHashFunction(TSDB_DATA_TYPE_INT), true, HASH_NO_LOCK); STableKeyInfo tableKeyInfo = {.pTable = NULL, .lastKey = INT64_MIN}; @@ -779,18 +1061,93 @@ void handleDownstreamOperator(SSqlRes* pRes, SQueryInfo* pQueryInfo) { taosArrayPush(tableGroupInfo.pGroupList, &group); - SOperatorInfo* pSourceOptr = createDummyInputOperator((char*)pRes, pSchema, numOfCols); + // if it is a join query, create join operator here + int32_t numOfCol1 = px->pTableMetaInfo[0]->pTableMeta->tableInfo.numOfColumns; + + int32_t numOfFilterCols = 0; + SColumnInfo* tableCols = calloc(numOfCol1, sizeof(SColumnInfo)); + for(int32_t i = 0; i < numOfCol1; ++i) { + SColumn* pCol = taosArrayGetP(px->colList, i); + if (pCol->info.flist.numOfFilters > 0) { + numOfFilterCols += 1; + } + + tableCols[i] = pCol->info; + } + + SSingleColumnFilterInfo* pFilterInfo = NULL; + if (numOfFilterCols > 0) { + doCreateFilterInfo(tableCols, numOfCol1, numOfFilterCols, &pFilterInfo, 0); + } + + SOperatorInfo* pSourceOperator = createDummyInputOperator(pSqlObjList[0], pSchema, numOfCol1, pFilterInfo, numOfFilterCols); + + SSchema* schema = NULL; + if (px->numOfTables > 1) { + SOperatorInfo** p = calloc(px->numOfTables, POINTER_BYTES); + p[0] = pSourceOperator; + + int32_t num = (int32_t) taosArrayGetSize(px->colList); + schema = calloc(num, sizeof(SSchema)); + memcpy(schema, pSchema, numOfCol1*sizeof(SSchema)); - SExprInfo *exprInfo = NULL; - /*int32_t code = */createQueryFunc(&info, numOfOutput, &exprInfo, px->exprList->pData, NULL, px->type, NULL); - px->pQInfo = createQueryInfoFromQueryNode(px, exprInfo, &tableGroupInfo, pSourceOptr, NULL, NULL, MASTER_SCAN); + int32_t offset = pSourceOperator->numOfOutput; - uint64_t qId = 0; - qTableQuery(px->pQInfo, &qId); - convertQueryResult(pRes, px); + for(int32_t i = 1; i < px->numOfTables; ++i) { + SSchema* pSchema1 = tscGetTableSchema(px->pTableMetaInfo[i]->pTableMeta); + int32_t n = px->pTableMetaInfo[i]->pTableMeta->tableInfo.numOfColumns; + int32_t numOfFilterCols1 = 0; + SColumnInfo* tableCols1 = calloc(numOfCol1, sizeof(SColumnInfo)); + for(int32_t j = 0; j < numOfCol1; ++j) { + SColumn* pCol = taosArrayGetP(px->colList, j); + if (pCol->info.flist.numOfFilters > 0) { + numOfFilterCols += 1; + } + + tableCols1[j] = pCol->info; + } + + SSingleColumnFilterInfo* pFilterInfo1 = NULL; + if (numOfFilterCols1 > 0) { + doCreateFilterInfo(tableCols1, numOfCol1, numOfFilterCols1, &pFilterInfo1, 0); + } + + p[i] = createDummyInputOperator(pSqlObjList[i], pSchema1, n, pFilterInfo1, numOfFilterCols1); + memcpy(&schema[offset], pSchema1, n * sizeof(SSchema)); + offset += n; + } + + pSourceOperator = createJoinOperatorInfo(p, px->numOfTables, schema, num); + tfree(p); + } else { + size_t num = taosArrayGetSize(px->colList); + schema = calloc(num, sizeof(SSchema)); + memcpy(schema, pSchema, numOfCol1*sizeof(SSchema)); + } + + SExprInfo* exprInfo = NULL; + /*int32_t code = */ createQueryFunc(&info, numOfOutput, &exprInfo, px->exprList->pData, NULL, px->type, NULL); + for(int32_t i = 0; i < numOfOutput; ++i) { + SExprInfo* pex = taosArrayGetP(px->exprList, i); + int32_t colId = pex->base.colInfo.colId; + for(int32_t j = 0; j < pSourceOperator->numOfOutput; ++j) { + if (colId == schema[j].colId) { + pex->base.colInfo.colIndex = j; + break; + } + } + } + + px->pQInfo = createQInfoFromQueryNode(px, exprInfo, &tableGroupInfo, pSourceOperator, NULL, NULL, MASTER_SCAN); tfree(pColumnInfo); + tfree(schema); + tfree(exprInfo); } + + uint64_t qId = 0; + qTableQuery(px->pQInfo, &qId); + convertQueryResult(pOutput, px); } static void tscDestroyResPointerInfo(SSqlRes* pRes) { @@ -823,26 +1180,29 @@ static void tscDestroyResPointerInfo(SSqlRes* pRes) { } void tscFreeQueryInfo(SSqlCmd* pCmd, bool removeMeta) { - if (pCmd == NULL || pCmd->numOfClause == 0) { + if (pCmd == NULL) { return; } - - for (int32_t i = 0; i < pCmd->numOfClause; ++i) { - SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd, i); - - // recursive call it - if (taosArrayGetSize(pQueryInfo->pUpstream) > 0) { - SQueryInfo* pUp = taosArrayGetP(pQueryInfo->pUpstream, 0); - freeQueryInfoImpl(pUp); - clearAllTableMetaInfo(pUp, removeMeta); - if (pUp->pQInfo != NULL) { - qDestroyQueryInfo(pUp->pQInfo); - pUp->pQInfo = NULL; + + SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd); + + while(pQueryInfo != NULL) { + SQueryInfo* p = pQueryInfo->sibling; + + size_t numOfUpstream = taosArrayGetSize(pQueryInfo->pUpstream); + for(int32_t i = 0; i < numOfUpstream; ++i) { + SQueryInfo* pUpQueryInfo = taosArrayGetP(pQueryInfo->pUpstream, i); + freeQueryInfoImpl(pUpQueryInfo); + + clearAllTableMetaInfo(pUpQueryInfo, removeMeta); + if (pUpQueryInfo->pQInfo != NULL) { + qDestroyQueryInfo(pUpQueryInfo->pQInfo); + pUpQueryInfo->pQInfo = NULL; } - tfree(pUp); + tfree(pUpQueryInfo); } - + freeQueryInfoImpl(pQueryInfo); clearAllTableMetaInfo(pQueryInfo, removeMeta); @@ -852,48 +1212,60 @@ void tscFreeQueryInfo(SSqlCmd* pCmd, bool removeMeta) { } tfree(pQueryInfo); + pQueryInfo = p; } - - pCmd->numOfClause = 0; - tfree(pCmd->pQueryInfo); + + pCmd->pQueryInfo = NULL; + pCmd->active = NULL; } void destroyTableNameList(SSqlCmd* pCmd) { - if (pCmd->numOfTables == 0) { - assert(pCmd->pTableNameList == NULL); + if (pCmd->insertParam.numOfTables == 0) { + assert(pCmd->insertParam.pTableNameList == NULL); return; } - for(int32_t i = 0; i < pCmd->numOfTables; ++i) { - tfree(pCmd->pTableNameList[i]); + for(int32_t i = 0; i < pCmd->insertParam.numOfTables; ++i) { + tfree(pCmd->insertParam.pTableNameList[i]); } - pCmd->numOfTables = 0; - tfree(pCmd->pTableNameList); + pCmd->insertParam.numOfTables = 0; + tfree(pCmd->insertParam.pTableNameList); } -void tscResetSqlCmd(SSqlCmd* pCmd, bool removeMeta) { +void tscResetSqlCmd(SSqlCmd* pCmd, bool clearCachedMeta) { pCmd->command = 0; pCmd->numOfCols = 0; pCmd->count = 0; pCmd->curSql = NULL; pCmd->msgType = 0; - pCmd->parseFinished = 0; - pCmd->autoCreated = 0; destroyTableNameList(pCmd); - pCmd->pTableBlockHashList = tscDestroyBlockHashTable(pCmd->pTableBlockHashList, removeMeta); - pCmd->pDataBlocks = tscDestroyBlockArrayList(pCmd->pDataBlocks); - tscFreeQueryInfo(pCmd, removeMeta); + pCmd->insertParam.pTableBlockHashList = tscDestroyBlockHashTable(pCmd->insertParam.pTableBlockHashList, clearCachedMeta); + pCmd->insertParam.pDataBlocks = tscDestroyBlockArrayList(pCmd->insertParam.pDataBlocks); + tscFreeQueryInfo(pCmd, clearCachedMeta); + + if (pCmd->pTableMetaMap != NULL) { + STableMetaVgroupInfo* p = taosHashIterate(pCmd->pTableMetaMap, NULL); + while (p) { + tfree(p->pVgroupInfo); + tfree(p->pTableMeta); + p = taosHashIterate(pCmd->pTableMetaMap, p); + } + + taosHashCleanup(pCmd->pTableMetaMap); + pCmd->pTableMetaMap = NULL; + } } void tscFreeSqlResult(SSqlObj* pSql) { - tscDestroyLocalMerger(pSql); - SSqlRes* pRes = &pSql->res; - tscDestroyResPointerInfo(pRes); + tscDestroyLocalMerger(pRes->pLocalMerger); + pRes->pLocalMerger = NULL; + + tscDestroyResPointerInfo(pRes); memset(&pSql->res, 0, sizeof(SSqlRes)); } @@ -915,7 +1287,6 @@ void tscFreeSubobj(SSqlObj* pSql) { } tfree(pSql->subState.states); - pSql->subState.numOfSub = 0; } @@ -939,7 +1310,6 @@ void tscFreeRegisteredSqlObj(void *pSql) { tscDebug("0x%"PRIx64" free SqlObj, total in tscObj:%d, total:%d", p->self, num, total); tscFreeSqlObj(p); taosReleaseRef(tscRefId, pTscObj->rid); - } void tscFreeMetaSqlObj(int64_t *rid){ @@ -1083,14 +1453,11 @@ void* tscDestroyBlockHashTable(SHashObj* pBlockHashTable, bool removeMeta) { int32_t tscCopyDataBlockToPayload(SSqlObj* pSql, STableDataBlocks* pDataBlock) { SSqlCmd* pCmd = &pSql->cmd; - assert(pDataBlock->pTableMeta != NULL); + assert(pDataBlock->pTableMeta != NULL && pDataBlock->size <= pDataBlock->nAllocSize && pDataBlock->size > sizeof(SMsgDesc)); - pCmd->numOfTablesInSubmit = pDataBlock->numOfTables; + STableMetaInfo* pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, 0); - assert(pCmd->numOfClause == 1); - STableMetaInfo* pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, pCmd->clauseIndex, 0); - - // todo refactor + // todo remove it later // set the correct table meta object, the table meta has been locked in pDataBlocks, so it must be in the cache if (pTableMetaInfo->pTableMeta != pDataBlock->pTableMeta) { tNameAssign(&pTableMetaInfo->name, &pDataBlock->tableName); @@ -1100,33 +1467,42 @@ int32_t tscCopyDataBlockToPayload(SSqlObj* pSql, STableDataBlocks* pDataBlock) { } pTableMetaInfo->pTableMeta = tscTableMetaDup(pDataBlock->pTableMeta); - pTableMetaInfo->tableMetaSize = tscGetTableMetaSize(pDataBlock->pTableMeta); + pTableMetaInfo->tableMetaSize = tscGetTableMetaSize(pDataBlock->pTableMeta); } /* - * the submit message consists of : [RPC header|message body|digest] - * the dataBlock only includes the RPC Header buffer and actual submit message body, space for digest needs - * additional space. + * the format of submit message is as follows [RPC header|message body|digest] + * the dataBlock only includes the RPC Header buffer and actual submit message body, + * space for digest needs additional space. */ - int ret = tscAllocPayload(pCmd, pDataBlock->size + 100); + int ret = tscAllocPayload(pCmd, pDataBlock->size); if (TSDB_CODE_SUCCESS != ret) { return ret; } - assert(pDataBlock->size <= pDataBlock->nAllocSize); memcpy(pCmd->payload, pDataBlock->pData, pDataBlock->size); - /* - * the payloadLen should be actual message body size - * the old value of payloadLen is the allocated payload size - */ + //the payloadLen should be actual message body size, the old value of payloadLen is the allocated payload size pCmd->payloadLen = pDataBlock->size; + assert(pCmd->allocSize >= (uint32_t)(pCmd->payloadLen)); - assert(pCmd->allocSize >= (uint32_t)(pCmd->payloadLen + 100) && pCmd->payloadLen > 0); + // NOTE: shell message size should not include SMsgDesc + int32_t size = pCmd->payloadLen - sizeof(SMsgDesc); + + SMsgDesc* pMsgDesc = (SMsgDesc*) pCmd->payload; + pMsgDesc->numOfVnodes = htonl(1); // always for one vnode + + SSubmitMsg *pShellMsg = (SSubmitMsg *)(pCmd->payload + sizeof(SMsgDesc)); + pShellMsg->header.vgId = htonl(pDataBlock->pTableMeta->vgId); // data in current block all routes to the same vgroup + pShellMsg->header.contLen = htonl(size); // the length not includes the size of SMsgDesc + pShellMsg->length = pShellMsg->header.contLen; + pShellMsg->numOfBlocks = htonl(pDataBlock->numOfTables); // the number of tables to be inserted + + tscDebug("0x%"PRIx64" submit msg built, vgId:%d numOfTables:%d", pSql->self, pDataBlock->pTableMeta->vgId, pDataBlock->numOfTables); return TSDB_CODE_SUCCESS; } -SQueryInfo* tscGetActiveQueryInfo(SSqlCmd* pCmd) { +SQueryInfo* tscGetQueryInfo(SSqlCmd* pCmd) { return pCmd->active; } @@ -1284,25 +1660,25 @@ static int32_t getRowExpandSize(STableMeta* pTableMeta) { } static void extractTableNameList(SSqlCmd* pCmd, bool freeBlockMap) { - pCmd->numOfTables = (int32_t) taosHashGetSize(pCmd->pTableBlockHashList); - if (pCmd->pTableNameList == NULL) { - pCmd->pTableNameList = calloc(pCmd->numOfTables, POINTER_BYTES); + pCmd->insertParam.numOfTables = (int32_t) taosHashGetSize(pCmd->insertParam.pTableBlockHashList); + if (pCmd->insertParam.pTableNameList == NULL) { + pCmd->insertParam.pTableNameList = calloc(pCmd->insertParam.numOfTables, POINTER_BYTES); } else { - memset(pCmd->pTableNameList, 0, pCmd->numOfTables * POINTER_BYTES); + memset(pCmd->insertParam.pTableNameList, 0, pCmd->insertParam.numOfTables * POINTER_BYTES); } - STableDataBlocks **p1 = taosHashIterate(pCmd->pTableBlockHashList, NULL); + STableDataBlocks **p1 = taosHashIterate(pCmd->insertParam.pTableBlockHashList, NULL); int32_t i = 0; while(p1) { STableDataBlocks* pBlocks = *p1; - tfree(pCmd->pTableNameList[i]); + tfree(pCmd->insertParam.pTableNameList[i]); - pCmd->pTableNameList[i++] = tNameDup(&pBlocks->tableName); - p1 = taosHashIterate(pCmd->pTableBlockHashList, p1); + pCmd->insertParam.pTableNameList[i++] = tNameDup(&pBlocks->tableName); + p1 = taosHashIterate(pCmd->insertParam.pTableBlockHashList, p1); } if (freeBlockMap) { - pCmd->pTableBlockHashList = tscDestroyBlockHashTable(pCmd->pTableBlockHashList, false); + pCmd->insertParam.pTableBlockHashList = tscDestroyBlockHashTable(pCmd->insertParam.pTableBlockHashList, false); } } @@ -1313,7 +1689,7 @@ int32_t tscMergeTableDataBlocks(SSqlObj* pSql, bool freeBlockMap) { void* pVnodeDataBlockHashList = taosHashInit(128, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BIGINT), true, false); SArray* pVnodeDataBlockList = taosArrayInit(8, POINTER_BYTES); - STableDataBlocks** p = taosHashIterate(pCmd->pTableBlockHashList, NULL); + STableDataBlocks** p = taosHashIterate(pCmd->insertParam.pTableBlockHashList, NULL); STableDataBlocks* pOneTableBlock = *p; while(pOneTableBlock) { @@ -1369,7 +1745,7 @@ int32_t tscMergeTableDataBlocks(SSqlObj* pSql, bool freeBlockMap) { pBlocks->schemaLen = 0; // erase the empty space reserved for binary data - int32_t finalLen = trimDataBlock(dataBuf->pData + dataBuf->size, pOneTableBlock, pCmd->submitSchema); + int32_t finalLen = trimDataBlock(dataBuf->pData + dataBuf->size, pOneTableBlock, pCmd->insertParam.schemaAttached); assert(finalLen <= len); dataBuf->size += (finalLen + sizeof(SSubmitBlk)); @@ -1384,7 +1760,7 @@ int32_t tscMergeTableDataBlocks(SSqlObj* pSql, bool freeBlockMap) { tscDebug("0x%"PRIx64" table %s data block is empty", pSql->self, pOneTableBlock->tableName.tname); } - p = taosHashIterate(pCmd->pTableBlockHashList, p); + p = taosHashIterate(pCmd->insertParam.pTableBlockHashList, p); if (p == NULL) { break; } @@ -1395,7 +1771,7 @@ int32_t tscMergeTableDataBlocks(SSqlObj* pSql, bool freeBlockMap) { extractTableNameList(pCmd, freeBlockMap); // free the table data blocks; - pCmd->pDataBlocks = pVnodeDataBlockList; + pCmd->insertParam.pDataBlocks = pVnodeDataBlockList; taosHashCleanup(pVnodeDataBlockHashList); return TSDB_CODE_SUCCESS; @@ -1427,18 +1803,22 @@ bool tscIsInsertData(char* sqlstr) { } int tscAllocPayload(SSqlCmd* pCmd, int size) { - assert(size > 0); - if (pCmd->payload == NULL) { assert(pCmd->allocSize == 0); pCmd->payload = (char*)calloc(1, size); - if (pCmd->payload == NULL) return TSDB_CODE_TSC_OUT_OF_MEMORY; + if (pCmd->payload == NULL) { + return TSDB_CODE_TSC_OUT_OF_MEMORY; + } + pCmd->allocSize = size; } else { if (pCmd->allocSize < (uint32_t)size) { char* b = realloc(pCmd->payload, size); - if (b == NULL) return TSDB_CODE_TSC_OUT_OF_MEMORY; + if (b == NULL) { + return TSDB_CODE_TSC_OUT_OF_MEMORY; + } + pCmd->payload = b; pCmd->allocSize = size; } @@ -1446,7 +1826,7 @@ int tscAllocPayload(SSqlCmd* pCmd, int size) { memset(pCmd->payload, 0, pCmd->allocSize); } - assert(pCmd->allocSize >= (uint32_t)size); + assert(pCmd->allocSize >= (uint32_t)size && size > 0); return TSDB_CODE_SUCCESS; } @@ -1475,16 +1855,14 @@ SInternalField* tscFieldInfoInsert(SFieldInfo* pFieldInfo, int32_t index, TAOS_F } void tscFieldInfoUpdateOffset(SQueryInfo* pQueryInfo) { - size_t numOfExprs = tscSqlExprNumOfExprs(pQueryInfo); - - SExprInfo* pExpr = taosArrayGetP(pQueryInfo->exprList, 0); - pExpr->base.offset = 0; + int32_t offset = 0; + size_t numOfExprs = tscNumOfExprs(pQueryInfo); - for (int32_t i = 1; i < numOfExprs; ++i) { - SExprInfo* prev = taosArrayGetP(pQueryInfo->exprList, i - 1); + for (int32_t i = 0; i < numOfExprs; ++i) { SExprInfo* p = taosArrayGetP(pQueryInfo->exprList, i); - p->base.offset = prev->base.offset + prev->base.resBytes; + p->base.offset = offset; + offset += p->base.resBytes; } } @@ -1525,6 +1903,7 @@ int32_t tscFieldInfoCompare(const SFieldInfo* pFieldInfo1, const SFieldInfo* pFi *diffSize = 1; if (pField2->bytes > pField1->bytes) { + assert(IS_VAR_DATA_TYPE(pField1->type)); pField1->bytes = pField2->bytes; } } @@ -1616,8 +1995,43 @@ void tscFieldInfoClear(SFieldInfo* pFieldInfo) { memset(pFieldInfo, 0, sizeof(SFieldInfo)); } -static SExprInfo* doCreateSqlExpr(SQueryInfo* pQueryInfo, int16_t functionId, SColumnIndex* pColIndex, int16_t type, - int16_t size, int16_t resColId, int16_t interSize, int32_t colType) { +void tscFieldInfoCopy(SFieldInfo* pFieldInfo, const SFieldInfo* pSrc, const SArray* pExprList) { + assert(pFieldInfo != NULL && pSrc != NULL && pExprList != NULL); + pFieldInfo->numOfOutput = pSrc->numOfOutput; + + if (pSrc->final != NULL) { + pFieldInfo->final = calloc(pSrc->numOfOutput, sizeof(TAOS_FIELD)); + memcpy(pFieldInfo->final, pSrc->final, sizeof(TAOS_FIELD) * pSrc->numOfOutput); + } + + if (pSrc->internalField != NULL) { + size_t num = taosArrayGetSize(pSrc->internalField); + size_t numOfExpr = taosArrayGetSize(pExprList); + + for (int32_t i = 0; i < num; ++i) { + SInternalField* pfield = taosArrayGet(pSrc->internalField, i); + + SInternalField p = {.visible = pfield->visible, .field = pfield->field}; + + int32_t resColId = pfield->pExpr->base.resColId; + for(int32_t j = 0; j < numOfExpr; ++j) { + SExprInfo* pExpr = taosArrayGetP(pExprList, j); + if (pExpr->base.resColId == resColId) { + p.pExpr = pExpr; + break; + } + } +// p.pExpr = calloc(1, sizeof(SExprInfo)); + +// tscExprAssign(p.pExpr, pfield->pExpr); + taosArrayPush(pFieldInfo->internalField, &p); + } + } +} + + +SExprInfo* tscExprCreate(SQueryInfo* pQueryInfo, int16_t functionId, SColumnIndex* pColIndex, int16_t type, + int16_t size, int16_t resColId, int16_t interSize, int32_t colType) { STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, pColIndex->tableIndex); SExprInfo* pExpr = calloc(1, sizeof(SExprInfo)); @@ -1676,29 +2090,29 @@ static SExprInfo* doCreateSqlExpr(SQueryInfo* pQueryInfo, int16_t functionId, SC return pExpr; } -SExprInfo* tscSqlExprInsert(SQueryInfo* pQueryInfo, int32_t index, int16_t functionId, SColumnIndex* pColIndex, int16_t type, +SExprInfo* tscExprInsert(SQueryInfo* pQueryInfo, int32_t index, int16_t functionId, SColumnIndex* pColIndex, int16_t type, int16_t size, int16_t resColId, int16_t interSize, bool isTagCol) { int32_t num = (int32_t)taosArrayGetSize(pQueryInfo->exprList); if (index == num) { - return tscSqlExprAppend(pQueryInfo, functionId, pColIndex, type, size, resColId, interSize, isTagCol); + return tscExprAppend(pQueryInfo, functionId, pColIndex, type, size, resColId, interSize, isTagCol); } - SExprInfo* pExpr = doCreateSqlExpr(pQueryInfo, functionId, pColIndex, type, size, resColId, interSize, isTagCol); + SExprInfo* pExpr = tscExprCreate(pQueryInfo, functionId, pColIndex, type, size, resColId, interSize, isTagCol); taosArrayInsert(pQueryInfo->exprList, index, &pExpr); return pExpr; } -SExprInfo* tscSqlExprAppend(SQueryInfo* pQueryInfo, int16_t functionId, SColumnIndex* pColIndex, int16_t type, +SExprInfo* tscExprAppend(SQueryInfo* pQueryInfo, int16_t functionId, SColumnIndex* pColIndex, int16_t type, int16_t size, int16_t resColId, int16_t interSize, bool isTagCol) { - SExprInfo* pExpr = doCreateSqlExpr(pQueryInfo, functionId, pColIndex, type, size, resColId, interSize, isTagCol); + SExprInfo* pExpr = tscExprCreate(pQueryInfo, functionId, pColIndex, type, size, resColId, interSize, isTagCol); taosArrayPush(pQueryInfo->exprList, &pExpr); return pExpr; } -SExprInfo* tscSqlExprUpdate(SQueryInfo* pQueryInfo, int32_t index, int16_t functionId, int16_t srcColumnIndex, +SExprInfo* tscExprUpdate(SQueryInfo* pQueryInfo, int32_t index, int16_t functionId, int16_t srcColumnIndex, int16_t type, int16_t size) { STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0); - SExprInfo* pExpr = tscSqlExprGet(pQueryInfo, index); + SExprInfo* pExpr = tscExprGet(pQueryInfo, index); if (pExpr == NULL) { return NULL; } @@ -1720,9 +2134,9 @@ bool tscMultiRoundQuery(SQueryInfo* pQueryInfo, int32_t index) { return false; } - int32_t numOfExprs = (int32_t) tscSqlExprNumOfExprs(pQueryInfo); + int32_t numOfExprs = (int32_t) tscNumOfExprs(pQueryInfo); for(int32_t i = 0; i < numOfExprs; ++i) { - SExprInfo* pExpr = tscSqlExprGet(pQueryInfo, i); + SExprInfo* pExpr = tscExprGet(pQueryInfo, i); if (pExpr->base.functionId == TSDB_FUNC_STDDEV_DST) { return true; } @@ -1731,11 +2145,11 @@ bool tscMultiRoundQuery(SQueryInfo* pQueryInfo, int32_t index) { return false; } -size_t tscSqlExprNumOfExprs(SQueryInfo* pQueryInfo) { +size_t tscNumOfExprs(SQueryInfo* pQueryInfo) { return taosArrayGetSize(pQueryInfo->exprList); } -void addExprParams(SSqlExpr* pExpr, char* argument, int32_t type, int32_t bytes) { +void tscExprAddParams(SSqlExpr* pExpr, char* argument, int32_t type, int32_t bytes) { assert (pExpr != NULL || argument != NULL || bytes != 0); // set parameter value @@ -1746,14 +2160,14 @@ void addExprParams(SSqlExpr* pExpr, char* argument, int32_t type, int32_t bytes) assert(pExpr->numOfParams <= 3); } -SExprInfo* tscSqlExprGet(SQueryInfo* pQueryInfo, int32_t index) { +SExprInfo* tscExprGet(SQueryInfo* pQueryInfo, int32_t index) { return taosArrayGetP(pQueryInfo->exprList, index); } /* * NOTE: Does not release SExprInfo here. */ -void tscSqlExprInfoDestroy(SArray* pExprInfo) { +void tscExprDestroy(SArray* pExprInfo) { size_t size = taosArrayGetSize(pExprInfo); for(int32_t i = 0; i < size; ++i) { @@ -1764,23 +2178,42 @@ void tscSqlExprInfoDestroy(SArray* pExprInfo) { taosArrayDestroy(pExprInfo); } -int32_t tscSqlExprCopy(SArray* dst, const SArray* src, uint64_t uid, bool deepcopy) { +int32_t tscExprCopy(SArray* dst, const SArray* src, uint64_t uid, bool deepcopy) { assert(src != NULL && dst != NULL); size_t size = taosArrayGetSize(src); for (int32_t i = 0; i < size; ++i) { SExprInfo* pExpr = taosArrayGetP(src, i); - + if (pExpr->base.uid == uid) { if (deepcopy) { SExprInfo* p1 = calloc(1, sizeof(SExprInfo)); - tscSqlExprAssign(p1, pExpr); + tscExprAssign(p1, pExpr); taosArrayPush(dst, &p1); } else { taosArrayPush(dst, &pExpr); } + } + } + + return 0; +} + +int32_t tscExprCopyAll(SArray* dst, const SArray* src, bool deepcopy) { + assert(src != NULL && dst != NULL); + + size_t size = taosArrayGetSize(src); + for (int32_t i = 0; i < size; ++i) { + SExprInfo* pExpr = taosArrayGetP(src, i); + + if (deepcopy) { + SExprInfo* p1 = calloc(1, sizeof(SExprInfo)); + tscExprAssign(p1, pExpr); + taosArrayPush(dst, &p1); + } else { + taosArrayPush(dst, &pExpr); } } @@ -1813,7 +2246,7 @@ bool tscColumnExists(SArray* pColumnList, int32_t columnIndex, uint64_t uid) { return true; } -void tscSqlExprAssign(SExprInfo* dst, const SExprInfo* src) { +void tscExprAssign(SExprInfo* dst, const SExprInfo* src) { assert(dst != NULL && src != NULL); *dst = *src; @@ -1925,6 +2358,18 @@ void tscColumnListCopy(SArray* dst, const SArray* src, uint64_t tableUid) { } } +void tscColumnListCopyAll(SArray* dst, const SArray* src) { + assert(src != NULL && dst != NULL); + + size_t num = taosArrayGetSize(src); + for (int32_t i = 0; i < num; ++i) { + SColumn* pCol = taosArrayGetP(src, i); + SColumn* p = tscColumnClone(pCol); + taosArrayPush(dst, &p); + } +} + + void tscColumnListDestroy(SArray* pColumnList) { if (pColumnList == NULL) { return; @@ -1963,7 +2408,7 @@ static int32_t validateQuoteToken(SStrToken* pToken) { } if (k != pToken->n || pToken->type != TK_ID) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } return TSDB_CODE_SUCCESS; } @@ -2013,7 +2458,7 @@ void tscDequoteAndTrimToken(SStrToken* pToken) { int32_t tscValidateName(SStrToken* pToken) { if (pToken->type != TK_STRING && pToken->type != TK_ID) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } char* sep = strnchr(pToken->z, TS_PATH_DELIMITER[0], pToken->n, true); @@ -2032,14 +2477,14 @@ int32_t tscValidateName(SStrToken* pToken) { } else { sep = strnchr(pToken->z, TS_PATH_DELIMITER[0], pToken->n, true); if (sep == NULL) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } return tscValidateName(pToken); } } else { if (isNumber(pToken)) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } } } else { // two part @@ -2052,15 +2497,15 @@ int32_t tscValidateName(SStrToken* pToken) { pToken->n = tGetToken(pToken->z, &pToken->type); if (pToken->z[pToken->n] != TS_PATH_DELIMITER[0]) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } if (pToken->type != TK_STRING && pToken->type != TK_ID) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } if (pToken->type == TK_STRING && validateQuoteToken(pToken) != TSDB_CODE_SUCCESS) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } int32_t firstPartLen = pToken->n; @@ -2069,11 +2514,11 @@ int32_t tscValidateName(SStrToken* pToken) { pToken->n = (uint32_t)(oldLen - (sep - pStr) - 1); int32_t len = tGetToken(pToken->z, &pToken->type); if (len != pToken->n || (pToken->type != TK_STRING && pToken->type != TK_ID)) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } if (pToken->type == TK_STRING && validateQuoteToken(pToken) != TSDB_CODE_SUCCESS) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } // re-build the whole name string @@ -2227,9 +2672,9 @@ void tscGetSrcColumnInfo(SSrcColumnInfo* pColInfo, SQueryInfo* pQueryInfo) { STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0); SSchema* pSchema = tscGetTableSchema(pTableMetaInfo->pTableMeta); - size_t numOfExprs = tscSqlExprNumOfExprs(pQueryInfo); + size_t numOfExprs = tscNumOfExprs(pQueryInfo); for (int32_t i = 0; i < numOfExprs; ++i) { - SExprInfo* pExpr = tscSqlExprGet(pQueryInfo, i); + SExprInfo* pExpr = tscExprGet(pQueryInfo, i); pColInfo[i].functionId = pExpr->base.functionId; if (TSDB_COL_IS_TAG(pExpr->base.colInfo.flag)) { @@ -2281,14 +2726,9 @@ bool tscShouldBeFreed(SSqlObj* pSql) { * @param tableIndex denote the table index for join query, where more than one table exists * @return */ -STableMetaInfo* tscGetTableMetaInfoFromCmd(SSqlCmd* pCmd, int32_t clauseIndex, int32_t tableIndex) { - if (pCmd == NULL || pCmd->numOfClause == 0) { - return NULL; - } - - assert(clauseIndex >= 0 && clauseIndex < pCmd->numOfClause); - - SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd, clauseIndex); +STableMetaInfo* tscGetTableMetaInfoFromCmd(SSqlCmd* pCmd, int32_t tableIndex) { + assert(pCmd != NULL); + SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd); return tscGetMetaInfo(pQueryInfo, tableIndex); } @@ -2305,8 +2745,8 @@ STableMetaInfo* tscGetMetaInfo(SQueryInfo* pQueryInfo, int32_t tableIndex) { return pQueryInfo->pTableMetaInfo[tableIndex]; } -SQueryInfo* tscGetQueryInfoS(SSqlCmd* pCmd, int32_t subClauseIndex) { - SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd, subClauseIndex); +SQueryInfo* tscGetQueryInfoS(SSqlCmd* pCmd) { + SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd); int32_t ret = TSDB_CODE_SUCCESS; while ((pQueryInfo) == NULL) { @@ -2315,7 +2755,7 @@ SQueryInfo* tscGetQueryInfoS(SSqlCmd* pCmd, int32_t subClauseIndex) { return NULL; } - pQueryInfo = tscGetQueryInfo(pCmd, subClauseIndex); + pQueryInfo = tscGetQueryInfo(pCmd); } return pQueryInfo; @@ -2339,6 +2779,7 @@ STableMetaInfo* tscGetTableMetaInfoByUid(SQueryInfo* pQueryInfo, uint64_t uid, i return tscGetMetaInfo(pQueryInfo, k); } +// todo refactor void tscInitQueryInfo(SQueryInfo* pQueryInfo) { assert(pQueryInfo->fieldsInfo.internalField == NULL); pQueryInfo->fieldsInfo.internalField = taosArrayInit(4, sizeof(SInternalField)); @@ -2348,27 +2789,17 @@ void tscInitQueryInfo(SQueryInfo* pQueryInfo) { pQueryInfo->exprList = taosArrayInit(4, POINTER_BYTES); pQueryInfo->colList = taosArrayInit(4, POINTER_BYTES); pQueryInfo->udColumnId = TSDB_UD_COLUMN_INDEX; - pQueryInfo->resColumnId = TSDB_RES_COL_ID; pQueryInfo->limit.limit = -1; pQueryInfo->limit.offset = 0; pQueryInfo->slimit.limit = -1; pQueryInfo->slimit.offset = 0; pQueryInfo->pUpstream = taosArrayInit(4, POINTER_BYTES); + pQueryInfo->window = TSWINDOW_INITIALIZER; } int32_t tscAddQueryInfo(SSqlCmd* pCmd) { assert(pCmd != NULL); - - // todo refactor: remove this structure - size_t s = pCmd->numOfClause + 1; - char* tmp = realloc(pCmd->pQueryInfo, s * POINTER_BYTES); - if (tmp == NULL) { - return TSDB_CODE_TSC_OUT_OF_MEMORY; - } - - pCmd->pQueryInfo = (SQueryInfo**)tmp; - SQueryInfo* pQueryInfo = calloc(1, sizeof(SQueryInfo)); if (pQueryInfo == NULL) { return TSDB_CODE_TSC_OUT_OF_MEMORY; @@ -2376,10 +2807,20 @@ int32_t tscAddQueryInfo(SSqlCmd* pCmd) { tscInitQueryInfo(pQueryInfo); - pQueryInfo->window = TSWINDOW_INITIALIZER; pQueryInfo->msg = pCmd->payload; // pointer to the parent error message buffer - pCmd->pQueryInfo[pCmd->numOfClause++] = pQueryInfo; + if (pCmd->pQueryInfo == NULL) { + pCmd->pQueryInfo = pQueryInfo; + } else { + SQueryInfo* p = pCmd->pQueryInfo; + while(p->sibling != NULL) { + p = p->sibling; + } + + p->sibling = pQueryInfo; + } + + pCmd->active = pQueryInfo; return TSDB_CODE_SUCCESS; } @@ -2387,9 +2828,14 @@ static void freeQueryInfoImpl(SQueryInfo* pQueryInfo) { tscTagCondRelease(&pQueryInfo->tagCond); tscFieldInfoClear(&pQueryInfo->fieldsInfo); - tscSqlExprInfoDestroy(pQueryInfo->exprList); + tscExprDestroy(pQueryInfo->exprList); pQueryInfo->exprList = NULL; + if (pQueryInfo->exprList1 != NULL) { + tscExprDestroy(pQueryInfo->exprList1); + pQueryInfo->exprList1 = NULL; + } + tscColumnListDestroy(pQueryInfo->colList); pQueryInfo->colList = NULL; @@ -2409,10 +2855,92 @@ static void freeQueryInfoImpl(SQueryInfo* pQueryInfo) { } void tscClearSubqueryInfo(SSqlCmd* pCmd) { - for (int32_t i = 0; i < pCmd->numOfClause; ++i) { - SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd, i); + SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd); + while (pQueryInfo != NULL) { + SQueryInfo* p = pQueryInfo->sibling; freeQueryInfoImpl(pQueryInfo); + pQueryInfo = p; + } +} + +int32_t tscQueryInfoCopy(SQueryInfo* pQueryInfo, const SQueryInfo* pSrc) { + assert(pQueryInfo != NULL && pSrc != NULL); + int32_t code = TSDB_CODE_SUCCESS; + + memcpy(&pQueryInfo->interval, &pSrc->interval, sizeof(pQueryInfo->interval)); + + pQueryInfo->command = pSrc->command; + pQueryInfo->type = pSrc->type; + pQueryInfo->window = pSrc->window; + pQueryInfo->limit = pSrc->limit; + pQueryInfo->slimit = pSrc->slimit; + pQueryInfo->order = pSrc->order; + pQueryInfo->vgroupLimit = pSrc->vgroupLimit; + pQueryInfo->tsBuf = NULL; + pQueryInfo->fillType = pSrc->fillType; + pQueryInfo->fillVal = NULL; + pQueryInfo->clauseLimit = pSrc->clauseLimit; + pQueryInfo->numOfTables = 0; + pQueryInfo->window = pSrc->window; + pQueryInfo->sessionWindow = pSrc->sessionWindow; + pQueryInfo->pTableMetaInfo = NULL; + + pQueryInfo->bufLen = pSrc->bufLen; + pQueryInfo->buf = malloc(pSrc->bufLen); + if (pQueryInfo->buf == NULL) { + code = TSDB_CODE_TSC_OUT_OF_MEMORY; + goto _error; + } + + if (pSrc->bufLen > 0) { + memcpy(pQueryInfo->buf, pSrc->buf, pSrc->bufLen); + } + + pQueryInfo->groupbyExpr = pSrc->groupbyExpr; + if (pSrc->groupbyExpr.columnInfo != NULL) { + pQueryInfo->groupbyExpr.columnInfo = taosArrayDup(pSrc->groupbyExpr.columnInfo); + if (pQueryInfo->groupbyExpr.columnInfo == NULL) { + code = TSDB_CODE_TSC_OUT_OF_MEMORY; + goto _error; + } + } + + if (tscTagCondCopy(&pQueryInfo->tagCond, &pSrc->tagCond) != 0) { + code = TSDB_CODE_TSC_OUT_OF_MEMORY; + goto _error; + } + + if (pSrc->fillType != TSDB_FILL_NONE) { + pQueryInfo->fillVal = malloc(pSrc->fieldsInfo.numOfOutput * sizeof(int64_t)); + if (pQueryInfo->fillVal == NULL) { + code = TSDB_CODE_TSC_OUT_OF_MEMORY; + goto _error; + } + + memcpy(pQueryInfo->fillVal, pSrc->fillVal, pSrc->fieldsInfo.numOfOutput * sizeof(int64_t)); + } + + if (tscExprCopyAll(pQueryInfo->exprList, pSrc->exprList, true) != 0) { + code = TSDB_CODE_TSC_OUT_OF_MEMORY; + goto _error; } + + tscColumnListCopyAll(pQueryInfo->colList, pSrc->colList); + tscFieldInfoCopy(&pQueryInfo->fieldsInfo, &pSrc->fieldsInfo, pQueryInfo->exprList); + + for(int32_t i = 0; i < pSrc->numOfTables; ++i) { + STableMetaInfo* p1 = tscGetMetaInfo((SQueryInfo*) pSrc, i); + + STableMeta* pMeta = tscTableMetaDup(p1->pTableMeta); + if (pMeta == NULL) { + // todo handle the error + } + + tscAddTableMetaInfo(pQueryInfo, &p1->name, pMeta, p1->vgroupList, p1->tagColList, p1->pVgroupTables); + } + + _error: + return code; } void tscFreeVgroupTableInfo(SArray* pVgroupTables) { @@ -2486,28 +3014,29 @@ void clearAllTableMetaInfo(SQueryInfo* pQueryInfo, bool removeMeta) { if (removeMeta) { char name[TSDB_TABLE_FNAME_LEN] = {0}; tNameExtractFullName(&pTableMetaInfo->name, name); - taosHashRemove(tscTableMetaInfo, name, strnlen(name, TSDB_TABLE_FNAME_LEN)); } tscFreeVgroupTableInfo(pTableMetaInfo->pVgroupTables); tscClearTableMetaInfo(pTableMetaInfo); + free(pTableMetaInfo); } - + tfree(pQueryInfo->pTableMetaInfo); } STableMetaInfo* tscAddTableMetaInfo(SQueryInfo* pQueryInfo, SName* name, STableMeta* pTableMeta, SVgroupsInfo* vgroupList, SArray* pTagCols, SArray* pVgroupTables) { - void* pAlloc = realloc(pQueryInfo->pTableMetaInfo, (pQueryInfo->numOfTables + 1) * POINTER_BYTES); - if (pAlloc == NULL) { + void* tmp = realloc(pQueryInfo->pTableMetaInfo, (pQueryInfo->numOfTables + 1) * POINTER_BYTES); + if (tmp == NULL) { terrno = TSDB_CODE_TSC_OUT_OF_MEMORY; return NULL; } - pQueryInfo->pTableMetaInfo = pAlloc; + pQueryInfo->pTableMetaInfo = tmp; STableMetaInfo* pTableMetaInfo = calloc(1, sizeof(STableMetaInfo)); + if (pTableMetaInfo == NULL) { terrno = TSDB_CODE_TSC_OUT_OF_MEMORY; return NULL; @@ -2592,8 +3121,6 @@ SSqlObj* createSimpleSubObj(SSqlObj* pSql, __async_cb_func_t fp, void* param, in SSqlCmd* pCmd = &pNew->cmd; pCmd->command = cmd; - pCmd->parseFinished = 1; - pCmd->autoCreated = pSql->cmd.autoCreated; int32_t code = copyTagData(&pNew->cmd.tagData, &pSql->cmd.tagData); if (code != TSDB_CODE_SUCCESS) { @@ -2617,10 +3144,8 @@ SSqlObj* createSimpleSubObj(SSqlObj* pSql, __async_cb_func_t fp, void* param, in pNew->sqlstr = NULL; pNew->maxRetry = TSDB_MAX_REPLICA; - SQueryInfo* pQueryInfo = tscGetQueryInfoS(pCmd, 0); - - assert(pSql->cmd.clauseIndex == 0); - STableMetaInfo* pMasterTableMetaInfo = tscGetTableMetaInfoFromCmd(&pSql->cmd, pSql->cmd.clauseIndex, 0); + SQueryInfo* pQueryInfo = tscGetQueryInfoS(pCmd); + STableMetaInfo* pMasterTableMetaInfo = tscGetTableMetaInfoFromCmd(&pSql->cmd, 0); tscAddTableMetaInfo(pQueryInfo, &pMasterTableMetaInfo->name, NULL, NULL, NULL, NULL); registerSqlObj(pNew); @@ -2629,14 +3154,14 @@ SSqlObj* createSimpleSubObj(SSqlObj* pSql, __async_cb_func_t fp, void* param, in } static void doSetSqlExprAndResultFieldInfo(SQueryInfo* pNewQueryInfo, int64_t uid) { - int32_t numOfOutput = (int32_t)tscSqlExprNumOfExprs(pNewQueryInfo); + int32_t numOfOutput = (int32_t)tscNumOfExprs(pNewQueryInfo); if (numOfOutput == 0) { return; } // set the field info in pNewQueryInfo object according to sqlExpr information for (int32_t i = 0; i < numOfOutput; ++i) { - SExprInfo* pExpr = tscSqlExprGet(pNewQueryInfo, i); + SExprInfo* pExpr = tscExprGet(pNewQueryInfo, i); TAOS_FIELD f = tscCreateField((int8_t) pExpr->base.resType, pExpr->base.aliasName, pExpr->base.resBytes); SInternalField* pInfo1 = tscFieldInfoAppend(&pNewQueryInfo->fieldsInfo, &f); @@ -2650,7 +3175,7 @@ static void doSetSqlExprAndResultFieldInfo(SQueryInfo* pNewQueryInfo, int64_t ui bool matched = false; for (int32_t k1 = 0; k1 < numOfOutput; ++k1) { - SExprInfo* pExpr1 = tscSqlExprGet(pNewQueryInfo, k1); + SExprInfo* pExpr1 = tscExprGet(pNewQueryInfo, k1); if (strcmp(field->name, pExpr1->base.aliasName) == 0) { // establish link according to the result field name SInternalField* pInfo = tscFieldInfoGetInternalField(&pNewQueryInfo->fieldsInfo, f); @@ -2678,7 +3203,7 @@ SSqlObj* createSubqueryObj(SSqlObj* pSql, int16_t tableIndex, __async_cb_func_t return NULL; } - SQueryInfo* pQueryInfo = tscGetActiveQueryInfo(pCmd); + SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd); STableMetaInfo* pTableMetaInfo = pQueryInfo->pTableMetaInfo[tableIndex]; pNew->pTscObj = pSql->pTscObj; @@ -2687,20 +3212,18 @@ SSqlObj* createSubqueryObj(SSqlObj* pSql, int16_t tableIndex, __async_cb_func_t SSqlCmd* pnCmd = &pNew->cmd; memcpy(pnCmd, pCmd, sizeof(SSqlCmd)); - + pnCmd->command = cmd; pnCmd->payload = NULL; pnCmd->allocSize = 0; + pnCmd->pTableMetaMap = NULL; pnCmd->pQueryInfo = NULL; - pnCmd->numOfClause = 0; - pnCmd->clauseIndex = 0; - pnCmd->pDataBlocks = NULL; - - pnCmd->numOfTables = 0; - pnCmd->parseFinished = 1; - pnCmd->pTableNameList = NULL; - pnCmd->pTableBlockHashList = NULL; + pnCmd->insertParam.pDataBlocks = NULL; + + pnCmd->insertParam.numOfTables = 0; + pnCmd->insertParam.pTableNameList = NULL; + pnCmd->insertParam.pTableBlockHashList = NULL; pnCmd->tagData.data = NULL; pnCmd->tagData.dataLen = 0; @@ -2709,7 +3232,7 @@ SSqlObj* createSubqueryObj(SSqlObj* pSql, int16_t tableIndex, __async_cb_func_t goto _error; } - SQueryInfo* pNewQueryInfo = tscGetQueryInfo(pnCmd, 0); + SQueryInfo* pNewQueryInfo = tscGetQueryInfo(pnCmd); pNewQueryInfo->command = pQueryInfo->command; pnCmd->active = pNewQueryInfo; @@ -2774,13 +3297,13 @@ SSqlObj* createSubqueryObj(SSqlObj* pSql, int16_t tableIndex, __async_cb_func_t // set the correct query type if (pPrevSql != NULL) { - SQueryInfo* pPrevQueryInfo = tscGetQueryInfo(&pPrevSql->cmd, pPrevSql->cmd.clauseIndex); + SQueryInfo* pPrevQueryInfo = tscGetQueryInfo(&pPrevSql->cmd); pNewQueryInfo->type = pPrevQueryInfo->type; } else { TSDB_QUERY_SET_TYPE(pNewQueryInfo->type, TSDB_QUERY_TYPE_SUBQUERY);// it must be the subquery } - if (tscSqlExprCopy(pNewQueryInfo->exprList, pQueryInfo->exprList, uid, true) != 0) { + if (tscExprCopy(pNewQueryInfo->exprList, pQueryInfo->exprList, uid, true) != 0) { terrno = TSDB_CODE_TSC_OUT_OF_MEMORY; goto _error; } @@ -2802,7 +3325,7 @@ SSqlObj* createSubqueryObj(SSqlObj* pSql, int16_t tableIndex, __async_cb_func_t pTableMetaInfo->tagColList, pTableMetaInfo->pVgroupTables); } else { // transfer the ownership of pTableMeta to the newly create sql object. - STableMetaInfo* pPrevInfo = tscGetTableMetaInfoFromCmd(&pPrevSql->cmd, pPrevSql->cmd.clauseIndex, 0); + STableMetaInfo* pPrevInfo = tscGetTableMetaInfoFromCmd(&pPrevSql->cmd, 0); if (pPrevInfo->pTableMeta && pPrevInfo->pTableMeta->tableType < 0) { terrno = TSDB_CODE_TSC_APP_ERROR; goto _error; @@ -2841,7 +3364,7 @@ SSqlObj* createSubqueryObj(SSqlObj* pSql, int16_t tableIndex, __async_cb_func_t tscDebug("0x%"PRIx64" new subquery:0x%"PRIx64", tableIndex:%d, vgroupIndex:%d, type:%d, exprInfo:%" PRIzu ", colList:%" PRIzu "," "fieldInfo:%d, name:%s, qrang:%" PRId64 " - %" PRId64 " order:%d, limit:%" PRId64, - pSql->self, pNew->self, tableIndex, pTableMetaInfo->vgroupIndex, pNewQueryInfo->type, tscSqlExprNumOfExprs(pNewQueryInfo), + pSql->self, pNew->self, tableIndex, pTableMetaInfo->vgroupIndex, pNewQueryInfo->type, tscNumOfExprs(pNewQueryInfo), size, pNewQueryInfo->fieldsInfo.numOfOutput, tNameGetTableName(&pFinalInfo->name), pNewQueryInfo->window.skey, pNewQueryInfo->window.ekey, pNewQueryInfo->order.order, pNewQueryInfo->limit.limit); @@ -2876,6 +3399,58 @@ void doExecuteQuery(SSqlObj* pSql, SQueryInfo* pQueryInfo) { } } +void doRetrieveSubqueryData(SSchedMsg *pMsg) { + SSqlObj* pSql = (SSqlObj*) pMsg->ahandle; + if (pSql == NULL || pSql->signature != pSql) { + tscDebug("%p SqlObj is freed, not add into queue async res", pMsg->ahandle); + return; + } + + SQueryInfo *pQueryInfo = tscGetQueryInfo(&pSql->cmd); + handleDownstreamOperator(pSql->pSubs, pSql->subState.numOfSub, pQueryInfo, &pSql->res); + + pSql->res.qId = -1; + if (pSql->res.code == TSDB_CODE_SUCCESS) { + (*pSql->fp)(pSql->param, pSql, pSql->res.numOfRows); + } else { + tscAsyncResultOnError(pSql); + } +} + +// NOTE: the blocking query can not be executed in the rpc message handler thread +static void tscSubqueryRetrieveCallback(void* param, TAOS_RES* tres, int code) { + // handle the pDownStream process + SRetrieveSupport* ps = param; + SSqlObj* pParentSql = ps->pParentSql; + SSqlObj* pSql = tres; + + int32_t index = ps->subqueryIndex; + bool ret = subAndCheckDone(pSql, pParentSql, index); + + // TODO refactor + tfree(ps); + pSql->param = NULL; + + if (!ret) { + tscDebug("0x%"PRIx64" sub:0x%"PRIx64" orderOfSub:%d completed, not all subquery finished", pParentSql->self, pSql->self, index); + return; + } + + pParentSql->cmd.active = pParentSql->cmd.pQueryInfo; + + SSchedMsg schedMsg = {0}; + schedMsg.fp = doRetrieveSubqueryData; + schedMsg.ahandle = (void *)pParentSql; + schedMsg.thandle = (void *)1; + schedMsg.msg = 0; + taosScheduleTask(tscQhandle, &schedMsg); +} + +// todo handle the failure +static void tscSubqueryCompleteCallback(void* param, TAOS_RES* tres, int code) { + taos_fetch_rows_a(tres, tscSubqueryRetrieveCallback, param); +} + // do execute the query according to the query execution plan void executeQuery(SSqlObj* pSql, SQueryInfo* pQueryInfo) { if (pSql->cmd.command == TSDB_SQL_RETRIEVE_EMPTY_RESULT) { @@ -2888,84 +3463,55 @@ void executeQuery(SSqlObj* pSql, SQueryInfo* pQueryInfo) { } if (taosArrayGetSize(pQueryInfo->pUpstream) > 0) { // nest query. do execute it firstly - SQueryInfo* pq = taosArrayGetP(pQueryInfo->pUpstream, 0); + pSql->subState.numOfSub = (int32_t) taosArrayGetSize(pQueryInfo->pUpstream); - pSql->cmd.active = pq; - pSql->cmd.command = TSDB_SQL_SELECT; + pSql->pSubs = calloc(pSql->subState.numOfSub, POINTER_BYTES); + pSql->subState.states = calloc(pSql->subState.numOfSub, sizeof(int8_t)); - executeQuery(pSql, pq); + for(int32_t i = 0; i < pSql->subState.numOfSub; ++i) { + SQueryInfo* pSub = taosArrayGetP(pQueryInfo->pUpstream, i); - // merge nest query result and generate final results - return; - } + pSql->cmd.active = pSub; + pSql->cmd.command = TSDB_SQL_SELECT; - pSql->cmd.active = pQueryInfo; - doExecuteQuery(pSql, pQueryInfo); -} + // TODO handle memory failure + SSqlObj* pNew = (SSqlObj*)calloc(1, sizeof(SSqlObj)); + if (pNew == NULL) { + terrno = TSDB_CODE_TSC_OUT_OF_MEMORY; + // return NULL; + } -/** - * todo remove it - * To decide if current is a two-stage super table query, join query, or insert. And invoke different - * procedure accordingly - * @param pSql - */ -void tscDoQuery(SSqlObj* pSql) { - SSqlCmd* pCmd = &pSql->cmd; - SSqlRes* pRes = &pSql->res; - - pRes->code = TSDB_CODE_SUCCESS; - - if (pCmd->command > TSDB_SQL_LOCAL) { - tscProcessLocalCmd(pSql); - return; - } - - if (pCmd->dataSourceType == DATA_FROM_DATA_FILE) { - tscImportDataFromFile(pSql); - } else { - SQueryInfo *pQueryInfo = tscGetQueryInfo(pCmd, pCmd->clauseIndex); - uint16_t type = pQueryInfo->type; + pNew->pTscObj = pSql->pTscObj; + pNew->signature = pNew; + pNew->sqlstr = strdup(pSql->sqlstr); // todo refactor + pNew->fp = tscSubqueryCompleteCallback; - if ((pCmd->command == TSDB_SQL_SELECT) && (!TSDB_QUERY_HAS_TYPE(type, TSDB_QUERY_TYPE_SUBQUERY)) && (!TSDB_QUERY_HAS_TYPE(type, TSDB_QUERY_TYPE_STABLE_SUBQUERY))) { - tscAddIntoSqlList(pSql); - } - - if (TSDB_QUERY_HAS_TYPE(type, TSDB_QUERY_TYPE_INSERT)) { // multi-vnodes insertion - tscHandleMultivnodeInsert(pSql); - return; - } - - if (QUERY_IS_JOIN_QUERY(type)) { - if (!TSDB_QUERY_HAS_TYPE(type, TSDB_QUERY_TYPE_SUBQUERY)) { - tscHandleMasterJoinQuery(pSql); - } else { // for first stage sub query, iterate all vnodes to get all timestamp - if (!TSDB_QUERY_HAS_TYPE(type, TSDB_QUERY_TYPE_JOIN_SEC_STAGE)) { - tscBuildAndSendRequest(pSql, NULL); - } else { // secondary stage join query. - if (tscIsTwoStageSTableQuery(pQueryInfo, 0)) { // super table query - tscLockByThread(&pSql->squeryLock); - tscHandleMasterSTableQuery(pSql); - tscUnlockByThread(&pSql->squeryLock); - } else { - tscBuildAndSendRequest(pSql, NULL); - } - } + SRetrieveSupport* ps = calloc(1, sizeof(SRetrieveSupport)); // todo use object id + ps->pParentSql = pSql; + ps->subqueryIndex = i; + + pNew->param = ps; + pSql->pSubs[i] = pNew; + registerSqlObj(pNew); + + SSqlCmd* pCmd = &pNew->cmd; + pCmd->command = TSDB_SQL_SELECT; + if (tscAddQueryInfo(pCmd) != TSDB_CODE_SUCCESS) { } - return; - } else if (tscMultiRoundQuery(pQueryInfo, 0) && pQueryInfo->round == 0) { - tscHandleFirstRoundStableQuery(pSql); // todo lock? - return; - } else if (tscIsTwoStageSTableQuery(pQueryInfo, 0)) { // super table query - tscLockByThread(&pSql->squeryLock); - tscHandleMasterSTableQuery(pSql); - tscUnlockByThread(&pSql->squeryLock); - return; + SQueryInfo* pNewQueryInfo = tscGetQueryInfo(pCmd); + tscQueryInfoCopy(pNewQueryInfo, pSub); + + // create sub query to handle the sub query. + executeQuery(pNew, pNewQueryInfo); } - pCmd->active = pQueryInfo; - tscBuildAndSendRequest(pSql, NULL); + // merge sub query result and generate final results + return; } + + pSql->cmd.active = pQueryInfo; + doExecuteQuery(pSql, pQueryInfo); } int16_t tscGetJoinTagColIdByUid(STagCond* pTagCond, uint64_t uid) { @@ -3023,17 +3569,15 @@ bool tscIsQueryWithLimit(SSqlObj* pSql) { } SSqlCmd* pCmd = &pSql->cmd; - for (int32_t i = 0; i < pCmd->numOfClause; ++i) { - SQueryInfo* pqi = tscGetQueryInfoS(pCmd, i); - if (pqi == NULL) { - continue; - } - + SQueryInfo* pqi = tscGetQueryInfo(pCmd); + while(pqi != NULL) { if (pqi->limit.limit > 0) { return true; } + + pqi = pqi->sibling; } - + return false; } @@ -3075,7 +3619,7 @@ int32_t tscInvalidSQLErrMsg(char* msg, const char* additionalInfo, const char* s if (sql == NULL) { assert(additionalInfo != NULL); sprintf(msg, msgFormat1, additionalInfo); - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } char buf[64] = {0}; // only extract part of sql string @@ -3087,7 +3631,7 @@ int32_t tscInvalidSQLErrMsg(char* msg, const char* additionalInfo, const char* s sprintf(msg, msgFormat3, buf); // no additional information for invalid sql error } - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } bool tscHasReachLimitation(SQueryInfo* pQueryInfo, SSqlRes* pRes) { @@ -3109,7 +3653,7 @@ bool hasMoreVnodesToTry(SSqlObj* pSql) { } assert(pRes->completed); - SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd, pCmd->clauseIndex); + SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd); STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0); // for normal table, no need to try any more if results are all retrieved from one vnode @@ -3127,14 +3671,15 @@ bool hasMoreVnodesToTry(SSqlObj* pSql) { } bool hasMoreClauseToTry(SSqlObj* pSql) { - return pSql->cmd.clauseIndex < pSql->cmd.numOfClause - 1; + SSqlCmd* pCmd = &pSql->cmd; + return pCmd->active->sibling != NULL; } void tscTryQueryNextVnode(SSqlObj* pSql, __async_cb_func_t fp) { SSqlCmd* pCmd = &pSql->cmd; SSqlRes* pRes = &pSql->res; - SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd, pCmd->clauseIndex); + SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd); /* * no result returned from the current virtual node anymore, try the next vnode if exists @@ -3190,11 +3735,7 @@ void tscTryQueryNextClause(SSqlObj* pSql, __async_cb_func_t fp) { SSqlCmd* pCmd = &pSql->cmd; SSqlRes* pRes = &pSql->res; - // current subclause is completed, try the next subclause - assert(pCmd->clauseIndex < pCmd->numOfClause - 1); - - pCmd->clauseIndex++; - SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd, pCmd->clauseIndex); + SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd); pSql->cmd.command = pQueryInfo->command; @@ -3214,7 +3755,7 @@ void tscTryQueryNextClause(SSqlObj* pSql, __async_cb_func_t fp) { pSql->subState.numOfSub = 0; pSql->fp = fp; - tscDebug("0x%"PRIx64" try data in the next subclause:%d, total subclause:%d", pSql->self, pCmd->clauseIndex, pCmd->numOfClause); + tscDebug("0x%"PRIx64" try data in the next subclause", pSql->self); if (pCmd->command > TSDB_SQL_LOCAL) { tscProcessLocalCmd(pSql); } else { @@ -3470,75 +4011,94 @@ uint32_t tscGetTableMetaMaxSize() { STableMeta* tscTableMetaDup(STableMeta* pTableMeta) { assert(pTableMeta != NULL); - uint32_t size = tscGetTableMetaSize(pTableMeta); + size_t size = tscGetTableMetaSize(pTableMeta); + STableMeta* p = calloc(1, size); memcpy(p, pTableMeta, size); return p; } -static int32_t createSecondaryExpr(SQueryAttr* pQueryAttr, SQueryInfo* pQueryInfo, STableMetaInfo* pTableMetaInfo) { - if (!tscIsSecondStageQuery(pQueryInfo)) { +SVgroupsInfo* tscVgroupsInfoDup(SVgroupsInfo* pVgroupsInfo) { + assert(pVgroupsInfo != NULL); + + size_t size = sizeof(SVgroupInfo) * pVgroupsInfo->numOfVgroups + sizeof(SVgroupsInfo); + SVgroupsInfo* pInfo = calloc(1, size); + memcpy(pInfo, pVgroupsInfo, size); + return pInfo; +} + +int32_t createProjectionExpr(SQueryInfo* pQueryInfo, STableMetaInfo* pTableMetaInfo, SExprInfo*** pExpr, int32_t* num) { + if (!pQueryInfo->arithmeticOnAgg) { return TSDB_CODE_SUCCESS; } - pQueryAttr->numOfExpr2 = tscNumOfFields(pQueryInfo); - pQueryAttr->pExpr2 = calloc(pQueryAttr->numOfExpr2, sizeof(SExprInfo)); - if (pQueryAttr->pExpr2 == NULL) { + *num = tscNumOfFields(pQueryInfo); + *pExpr = calloc(*(num), POINTER_BYTES); + if ((*pExpr) == NULL) { return TSDB_CODE_TSC_OUT_OF_MEMORY; } - for (int32_t i = 0; i < pQueryAttr->numOfExpr2; ++i) { + for (int32_t i = 0; i < (*num); ++i) { SInternalField* pField = tscFieldInfoGetInternalField(&pQueryInfo->fieldsInfo, i); - SExprInfo* pExpr = pField->pExpr; + SExprInfo* pSource = pField->pExpr; + + SExprInfo* px = calloc(1, sizeof(SExprInfo)); + (*pExpr)[i] = px; - SSqlExpr *pse = &pQueryAttr->pExpr2[i].base; + SSqlExpr *pse = &px->base; pse->uid = pTableMetaInfo->pTableMeta->id.uid; - pse->resColId = pExpr->base.resColId; + pse->resColId = pSource->base.resColId; + strncpy(pse->aliasName, pSource->base.aliasName, tListLen(pse->aliasName)); + strncpy(pse->token, pSource->base.token, tListLen(pse->token)); - if (pExpr->base.functionId != TSDB_FUNC_ARITHM) { // this should be switched to projection query + if (pSource->base.functionId != TSDB_FUNC_ARITHM) { // this should be switched to projection query pse->numOfParams = 0; // no params for projection query pse->functionId = TSDB_FUNC_PRJ; - pse->colInfo.colId = pExpr->base.resColId; + pse->colInfo.colId = pSource->base.resColId; - for (int32_t j = 0; j < pQueryAttr->numOfOutput; ++j) { - if (pQueryAttr->pExpr1[j].base.resColId == pse->colInfo.colId) { + int32_t numOfOutput = (int32_t) taosArrayGetSize(pQueryInfo->exprList); + for (int32_t j = 0; j < numOfOutput; ++j) { + SExprInfo* p = taosArrayGetP(pQueryInfo->exprList, j); + if (p->base.resColId == pse->colInfo.colId) { pse->colInfo.colIndex = j; + break; } } pse->colInfo.flag = TSDB_COL_NORMAL; - pse->resType = pExpr->base.resType; - pse->resBytes = pExpr->base.resBytes; + pse->resType = pSource->base.resType; + pse->resBytes = pSource->base.resBytes; + strncpy(pse->colInfo.name, pSource->base.aliasName, tListLen(pse->colInfo.name)); // TODO restore refactor - int32_t functionId = pExpr->base.functionId; - if (pExpr->base.functionId == TSDB_FUNC_FIRST_DST) { + int32_t functionId = pSource->base.functionId; + if (pSource->base.functionId == TSDB_FUNC_FIRST_DST) { functionId = TSDB_FUNC_FIRST; - } else if (pExpr->base.functionId == TSDB_FUNC_LAST_DST) { + } else if (pSource->base.functionId == TSDB_FUNC_LAST_DST) { functionId = TSDB_FUNC_LAST; - } else if (pExpr->base.functionId == TSDB_FUNC_STDDEV_DST) { + } else if (pSource->base.functionId == TSDB_FUNC_STDDEV_DST) { functionId = TSDB_FUNC_STDDEV; } int32_t inter = 0; - getResultDataInfo(pExpr->base.colType, pExpr->base.colBytes, functionId, 0, &pse->resType, + getResultDataInfo(pSource->base.colType, pSource->base.colBytes, functionId, 0, &pse->resType, &pse->resBytes, &inter, 0, false); pse->colType = pse->resType; pse->colBytes = pse->resBytes; } else { // arithmetic expression - pse->colInfo.colId = pExpr->base.colInfo.colId; - pse->colType = pExpr->base.colType; - pse->colBytes = pExpr->base.colBytes; + pse->colInfo.colId = pSource->base.colInfo.colId; + pse->colType = pSource->base.colType; + pse->colBytes = pSource->base.colBytes; pse->resBytes = sizeof(double); pse->resType = TSDB_DATA_TYPE_DOUBLE; - pse->functionId = pExpr->base.functionId; - pse->numOfParams = pExpr->base.numOfParams; + pse->functionId = pSource->base.functionId; + pse->numOfParams = pSource->base.numOfParams; - for (int32_t j = 0; j < pExpr->base.numOfParams; ++j) { - tVariantAssign(&pse->param[j], &pExpr->base.param[j]); - buildArithmeticExprFromMsg(&pQueryAttr->pExpr2[i], NULL); + for (int32_t j = 0; j < pSource->base.numOfParams; ++j) { + tVariantAssign(&pse->param[j], &pSource->base.param[j]); + buildArithmeticExprFromMsg(px, NULL); } } } @@ -3549,7 +4109,7 @@ static int32_t createSecondaryExpr(SQueryAttr* pQueryAttr, SQueryInfo* pQueryInf static int32_t createGlobalAggregateExpr(SQueryAttr* pQueryAttr, SQueryInfo* pQueryInfo) { assert(tscIsTwoStageSTableQuery(pQueryInfo, 0)); - pQueryAttr->numOfExpr3 = (int32_t) tscSqlExprNumOfExprs(pQueryInfo); + pQueryAttr->numOfExpr3 = (int32_t) tscNumOfExprs(pQueryInfo); pQueryAttr->pExpr3 = calloc(pQueryAttr->numOfExpr3, sizeof(SExprInfo)); if (pQueryAttr->pExpr3 == NULL) { return TSDB_CODE_TSC_OUT_OF_MEMORY; @@ -3559,7 +4119,7 @@ static int32_t createGlobalAggregateExpr(SQueryAttr* pQueryAttr, SQueryInfo* pQu SExprInfo* pExpr = &pQueryAttr->pExpr1[i]; SSqlExpr* pse = &pQueryAttr->pExpr3[i].base; - tscSqlExprAssign(&pQueryAttr->pExpr3[i], pExpr); + tscExprAssign(&pQueryAttr->pExpr3[i], pExpr); pse->colInfo.colId = pExpr->base.resColId; pse->colInfo.colIndex = i; @@ -3623,7 +4183,7 @@ static int32_t createTagColumnInfo(SQueryAttr* pQueryAttr, SQueryInfo* pQueryInf if ((pCol->columnIndex >= numOfTagColumns || pCol->columnIndex < TSDB_TBNAME_COLUMN_INDEX) || (!isValidDataType(pColSchema->type))) { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } SColumnInfo* pTagCol = &pQueryAttr->tagColList[i]; @@ -3641,7 +4201,7 @@ int32_t tscCreateQueryFromQueryInfo(SQueryInfo* pQueryInfo, SQueryAttr* pQueryAt memset(pQueryAttr, 0, sizeof(SQueryAttr)); int16_t numOfCols = (int16_t) taosArrayGetSize(pQueryInfo->colList); - int16_t numOfOutput = (int16_t) tscSqlExprNumOfExprs(pQueryInfo); + int16_t numOfOutput = (int16_t) tscNumOfExprs(pQueryInfo); pQueryAttr->topBotQuery = tscIsTopBotQuery(pQueryInfo); pQueryAttr->hasTagResults = hasTagValOutput(pQueryInfo); @@ -3676,7 +4236,7 @@ int32_t tscCreateQueryFromQueryInfo(SQueryInfo* pQueryInfo, SQueryAttr* pQueryAt STableMetaInfo* pTableMetaInfo = pQueryInfo->pTableMetaInfo[0]; - pQueryAttr->pGroupbyExpr = calloc(1, sizeof(SSqlGroupbyExpr)); + pQueryAttr->pGroupbyExpr = calloc(1, sizeof(SGroupbyExpr)); *(pQueryAttr->pGroupbyExpr) = pQueryInfo->groupbyExpr; if (pQueryInfo->groupbyExpr.numOfGroupCols > 0) { @@ -3687,8 +4247,8 @@ int32_t tscCreateQueryFromQueryInfo(SQueryInfo* pQueryInfo, SQueryAttr* pQueryAt pQueryAttr->pExpr1 = calloc(pQueryAttr->numOfOutput, sizeof(SExprInfo)); for(int32_t i = 0; i < pQueryAttr->numOfOutput; ++i) { - SExprInfo* pExpr = tscSqlExprGet(pQueryInfo, i); - tscSqlExprAssign(&pQueryAttr->pExpr1[i], pExpr); + SExprInfo* pExpr = tscExprGet(pQueryInfo, i); + tscExprAssign(&pQueryAttr->pExpr1[i], pExpr); if (pQueryAttr->pExpr1[i].base.functionId == TSDB_FUNC_ARITHM) { for (int32_t j = 0; j < pQueryAttr->pExpr1[i].base.numOfParams; ++j) { @@ -3714,13 +4274,17 @@ int32_t tscCreateQueryFromQueryInfo(SQueryInfo* pQueryInfo, SQueryAttr* pQueryAt } // for simple table, not for super table - int32_t code = createSecondaryExpr(pQueryAttr, pQueryInfo, pTableMetaInfo); - if (code != TSDB_CODE_SUCCESS) { - return code; + if (pQueryInfo->arithmeticOnAgg) { + pQueryAttr->numOfExpr2 = (int32_t) taosArrayGetSize(pQueryInfo->exprList1); + pQueryAttr->pExpr2 = calloc(pQueryAttr->numOfExpr2, sizeof(SExprInfo)); + for(int32_t i = 0; i < pQueryAttr->numOfExpr2; ++i) { + SExprInfo* p = taosArrayGetP(pQueryInfo->exprList1, i); + tscExprAssign(&pQueryAttr->pExpr2[i], p); + } } // tag column info - code = createTagColumnInfo(pQueryAttr, pQueryInfo, pTableMetaInfo); + int32_t code = createTagColumnInfo(pQueryAttr, pQueryInfo, pTableMetaInfo); if (code != TSDB_CODE_SUCCESS) { return code; } @@ -3745,19 +4309,106 @@ int32_t tscCreateQueryFromQueryInfo(SQueryInfo* pQueryInfo, SQueryAttr* pQueryAt tscError("%p illegal value of numOfCols in query msg: %" PRIu64 ", table cols:%d", addr, (uint64_t)pQueryAttr->numOfCols, numOfCols); - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } if (pQueryAttr->interval.interval < 0) { tscError("%p illegal value of aggregation time interval in query msg: %" PRId64, addr, (int64_t)pQueryInfo->interval.interval); - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } if (pQueryAttr->pGroupbyExpr->numOfGroupCols < 0) { tscError("%p illegal value of numOfGroupCols in query msg: %d", addr, pQueryInfo->groupbyExpr.numOfGroupCols); - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; + } + + return TSDB_CODE_SUCCESS; +} + +static int32_t doAddTableName(char* nextStr, char** str, SArray* pNameArray, SSqlObj* pSql) { + int32_t code = TSDB_CODE_SUCCESS; + SSqlCmd* pCmd = &pSql->cmd; + + char tablename[TSDB_TABLE_FNAME_LEN] = {0}; + int32_t len = 0; + + if (nextStr == NULL) { + strncpy(tablename, *str, TSDB_TABLE_FNAME_LEN); + len = (int32_t) strlen(tablename); + } else { + memcpy(tablename, *str, nextStr - (*str)); + len = (int32_t)(nextStr - (*str)); + tablename[len] = '\0'; + } + + (*str) = nextStr + 1; + len = (int32_t)strtrim(tablename); + + SStrToken sToken = {.n = len, .type = TK_ID, .z = tablename}; + tGetToken(tablename, &sToken.type); + + // Check if the table name available or not + if (tscValidateName(&sToken) != TSDB_CODE_SUCCESS) { + code = TSDB_CODE_TSC_INVALID_TABLE_ID_LENGTH; + sprintf(pCmd->payload, "table name is invalid"); + return code; } + SName name = {0}; + if ((code = tscSetTableFullName(&name, &sToken, pSql)) != TSDB_CODE_SUCCESS) { + return code; + } + + memset(tablename, 0, tListLen(tablename)); + tNameExtractFullName(&name, tablename); + + char* p = strdup(tablename); + taosArrayPush(pNameArray, &p); return TSDB_CODE_SUCCESS; } + +int tscTransferTableNameList(SSqlObj *pSql, const char *pNameList, int32_t length, SArray* pNameArray) { + SSqlCmd *pCmd = &pSql->cmd; + + pCmd->command = TSDB_SQL_MULTI_META; + pCmd->msgType = TSDB_MSG_TYPE_CM_TABLES_META; + + int code = TSDB_CODE_TSC_INVALID_TABLE_ID_LENGTH; + char *str = (char *)pNameList; + + SQueryInfo *pQueryInfo = tscGetQueryInfoS(pCmd); + if (pQueryInfo == NULL) { + pSql->res.code = terrno; + return terrno; + } + + char *nextStr; + while (1) { + nextStr = strchr(str, ','); + if (nextStr == NULL) { + code = doAddTableName(nextStr, &str, pNameArray, pSql); + break; + } + + code = doAddTableName(nextStr, &str, pNameArray, pSql); + if (code != TSDB_CODE_SUCCESS) { + return code; + } + + if (taosArrayGetSize(pNameArray) > TSDB_MULTI_TABLEMETA_MAX_NUM) { + code = TSDB_CODE_TSC_INVALID_TABLE_ID_LENGTH; + sprintf(pCmd->payload, "tables over the max number"); + return code; + } + } + + if (taosArrayGetSize(pNameArray) > TSDB_MULTI_TABLEMETA_MAX_NUM) { + code = TSDB_CODE_TSC_INVALID_TABLE_ID_LENGTH; + sprintf(pCmd->payload, "tables over the max number"); + return code; + } + + return TSDB_CODE_SUCCESS; +} + diff --git a/src/common/inc/tdataformat.h b/src/common/inc/tdataformat.h index 88d5b850102cc2ecc796ead0fd75408da428c431..fcae7a415fae8bc1b5897bf6865f1cc2004ba731 100644 --- a/src/common/inc/tdataformat.h +++ b/src/common/inc/tdataformat.h @@ -234,6 +234,7 @@ typedef struct SDataCol { int len; // column data length VarDataOffsetT *dataOff; // For binary and nchar data, the offset in the data column void * pData; // Actual data pointer + TSKEY ts; // only used in last NULL column } SDataCol; static FORCE_INLINE void dataColReset(SDataCol *pDataCol) { pDataCol->len = 0; } diff --git a/src/common/inc/tname.h b/src/common/inc/tname.h index 48bec7fe4dfd04aa0080fd0b75ccdd84fe837761..ec2d76147ae71ce1e57abb606ceb73d3ef277248 100644 --- a/src/common/inc/tname.h +++ b/src/common/inc/tname.h @@ -44,8 +44,8 @@ typedef struct SResPair { // the structure for sql function in select clause typedef struct SSqlExpr { char aliasName[TSDB_COL_NAME_LEN]; // as aliasName + char token[TSDB_COL_NAME_LEN]; // original token SColIndex colInfo; - uint64_t uid; // refactor use the pointer int16_t functionId; // function id in aAgg array @@ -92,8 +92,6 @@ size_t tableIdPrefix(const char* name, char* prefix, int32_t len); void extractTableNameFromToken(SStrToken *pToken, SStrToken* pTable); -//SSchema tGetTbnameColumnSchema(); - SSchema tGetBlockDistColumnSchema(); SSchema tGetUserSpecifiedColumnSchema(tVariant* pVal, SStrToken* exprStr, const char* name); diff --git a/src/common/src/tarithoperator.c b/src/common/src/tarithoperator.c index 1cb667d259f040cfab0656562f7c97444fc48d8a..b37e358b9c8688b888fb6122289dbe766f6306c9 100644 --- a/src/common/src/tarithoperator.c +++ b/src/common/src/tarithoperator.c @@ -2569,6 +2569,7 @@ _arithmetic_operator_fn_t getArithmeticOperatorFn(int32_t arithmeticOptr) { case TSDB_BINARY_OP_REMAINDER: return vectorRemainder; default: + assert(0); return NULL; } } diff --git a/src/common/src/texpr.c b/src/common/src/texpr.c index 4334a0de263e9857385222462724d4b8c758593e..f2dd3890a163ec1e1e25d2ac8b55f23aa9d5a79d 100644 --- a/src/common/src/texpr.c +++ b/src/common/src/texpr.c @@ -13,6 +13,7 @@ * along with this program. If not, see . */ +#include #include "os.h" #include "texpr.h" @@ -465,27 +466,29 @@ tExprNode* exprTreeFromTableName(const char* tbnameCond) { return expr; } -tExprNode* exprdup(tExprNode* pTree) { - if (pTree == NULL) { +tExprNode* exprdup(tExprNode* pNode) { + if (pNode == NULL) { return NULL; } - tExprNode* pNode = calloc(1, sizeof(tExprNode)); - if (pTree->nodeType == TSQL_NODE_EXPR) { - tExprNode* pLeft = exprdup(pTree->_node.pLeft); - tExprNode* pRight = exprdup(pTree->_node.pRight); - - pNode->nodeType = TSQL_NODE_EXPR; - pNode->_node.pLeft = pLeft; - pNode->_node.pRight = pRight; - } else if (pTree->nodeType == TSQL_NODE_VALUE) { - pNode->pVal = calloc(1, sizeof(tVariant)); - tVariantAssign(pNode->pVal, pTree->pVal); - } else if (pTree->nodeType == TSQL_NODE_COL) { - pNode->pSchema = calloc(1, sizeof(SSchema)); - *pNode->pSchema = *pTree->pSchema; + tExprNode* pCloned = calloc(1, sizeof(tExprNode)); + if (pNode->nodeType == TSQL_NODE_EXPR) { + tExprNode* pLeft = exprdup(pNode->_node.pLeft); + tExprNode* pRight = exprdup(pNode->_node.pRight); + + pCloned->_node.pLeft = pLeft; + pCloned->_node.pRight = pRight; + pCloned->_node.optr = pNode->_node.optr; + pCloned->_node.hasPK = pNode->_node.hasPK; + } else if (pNode->nodeType == TSQL_NODE_VALUE) { + pCloned->pVal = calloc(1, sizeof(tVariant)); + tVariantAssign(pCloned->pVal, pNode->pVal); + } else if (pNode->nodeType == TSQL_NODE_COL) { + pCloned->pSchema = calloc(1, sizeof(SSchema)); + *pCloned->pSchema = *pNode->pSchema; } - return pNode; + pCloned->nodeType = pNode->nodeType; + return pCloned; } diff --git a/src/dnode/src/dnodeMain.c b/src/dnode/src/dnodeMain.c index ee4dfcf85210d389651224796727d5b72ea8b9ac..cce382b15f5c7e1396ecce543fbd1355bfdf5ac5 100644 --- a/src/dnode/src/dnodeMain.c +++ b/src/dnode/src/dnodeMain.c @@ -219,14 +219,20 @@ static int32_t dnodeInitStorage() { if (tsCompactMnodeWal == 1) { sprintf(tsMnodeTmpDir, "%s/mnode_tmp", tsDataDir); - tfsRmdir(tsMnodeTmpDir); + if (taosDirExist(tsMnodeTmpDir)) { + dError("mnode_tmp dir already exist in %s,quit compact job", tsMnodeTmpDir); + return -1; + } if (dnodeCreateDir(tsMnodeTmpDir) < 0) { dError("failed to create dir: %s, reason: %s", tsMnodeTmpDir, strerror(errno)); return -1; } sprintf(tsMnodeBakDir, "%s/mnode_bak", tsDataDir); - //tfsRmdir(tsMnodeBakDir); + if (taosDirExist(tsMnodeBakDir)) { + dError("mnode_bak dir already exist in %s,quit compact job", tsMnodeBakDir); + return -1; + } } //TODO(dengyihao): no need to init here if (dnodeCreateDir(tsMnodeDir) < 0) { diff --git a/src/inc/taosdef.h b/src/inc/taosdef.h index e596ee67ecf1ab736246056d4e510661eee677fe..c00fde18162a1c7006706286ac26bc5cd6e7e547 100644 --- a/src/inc/taosdef.h +++ b/src/inc/taosdef.h @@ -33,6 +33,8 @@ extern "C" { #endif #define TSWINDOW_INITIALIZER ((STimeWindow) {INT64_MIN, INT64_MAX}) +#define TSWINDOW_DESC_INITIALIZER ((STimeWindow) {INT64_MAX, INT64_MIN}) + #define TSKEY_INITIAL_VAL INT64_MIN // Bytes for each type. @@ -298,7 +300,7 @@ do { \ #define TSDB_DEFAULT_DB_UPDATE_OPTION 0 #define TSDB_MIN_DB_CACHE_LAST_ROW 0 -#define TSDB_MAX_DB_CACHE_LAST_ROW 1 +#define TSDB_MAX_DB_CACHE_LAST_ROW 3 #define TSDB_DEFAULT_CACHE_LAST_ROW 0 #define TSDB_MIN_FSYNC_PERIOD 0 @@ -345,6 +347,7 @@ do { \ #define TSDB_QUERY_TYPE_TAG_FILTER_QUERY 0x400u #define TSDB_QUERY_TYPE_INSERT 0x100u // insert type #define TSDB_QUERY_TYPE_MULTITABLE_QUERY 0x200u +#define TSDB_QUERY_TYPE_FILE_INSERT 0x400u // insert data from file #define TSDB_QUERY_TYPE_STMT_INSERT 0x800u // stmt insert type #define TSDB_QUERY_HAS_TYPE(x, _type) (((x) & (_type)) != 0) diff --git a/src/inc/taoserror.h b/src/inc/taoserror.h index d3ac0ce28004c2b189ba98ac6266ec5dd1a7c935..09f632ea32077b3e9bd3fcddcff418f280192c3f 100644 --- a/src/inc/taoserror.h +++ b/src/inc/taoserror.h @@ -74,7 +74,7 @@ int32_t* taosGetErrno(); #define TSDB_CODE_REF_NOT_EXIST TAOS_DEF_ERROR_CODE(0, 0x010A) //"Ref is not there") //client -#define TSDB_CODE_TSC_INVALID_SQL TAOS_DEF_ERROR_CODE(0, 0x0200) //"Invalid SQL statement") +#define TSDB_CODE_TSC_INVALID_OPERATION TAOS_DEF_ERROR_CODE(0, 0x0200) //"Invalid Operation") #define TSDB_CODE_TSC_INVALID_QHANDLE TAOS_DEF_ERROR_CODE(0, 0x0201) //"Invalid qhandle") #define TSDB_CODE_TSC_INVALID_TIME_STAMP TAOS_DEF_ERROR_CODE(0, 0x0202) //"Invalid combination of client/service time") #define TSDB_CODE_TSC_INVALID_VALUE TAOS_DEF_ERROR_CODE(0, 0x0203) //"Invalid value in client") @@ -219,6 +219,7 @@ int32_t* taosGetErrno(); #define TSDB_CODE_VND_NO_WRITE_AUTH TAOS_DEF_ERROR_CODE(0, 0x0512) //"Database write operation denied") #define TSDB_CODE_VND_IS_SYNCING TAOS_DEF_ERROR_CODE(0, 0x0513) //"Database is syncing") #define TSDB_CODE_VND_INVALID_TSDB_STATE TAOS_DEF_ERROR_CODE(0, 0x0514) //"Invalid tsdb state") +#define TSDB_CODE_VND_IS_CLOSING TAOS_DEF_ERROR_CODE(0, 0x0515) //"Database is closing") // tsdb #define TSDB_CODE_TDB_INVALID_TABLE_ID TAOS_DEF_ERROR_CODE(0, 0x0600) //"Invalid table ID") diff --git a/src/inc/taosmsg.h b/src/inc/taosmsg.h index 3b7022fb88d34e0e71a3f70ace85d769e338b11e..ff3cc6e956293a6be52f5aff99316f72bdf5254b 100644 --- a/src/inc/taosmsg.h +++ b/src/inc/taosmsg.h @@ -84,7 +84,7 @@ TAOS_DEFINE_MESSAGE_TYPE( TSDB_MSG_TYPE_CM_DROP_TABLE, "drop-table" ) TAOS_DEFINE_MESSAGE_TYPE( TSDB_MSG_TYPE_CM_ALTER_TABLE, "alter-table" ) TAOS_DEFINE_MESSAGE_TYPE( TSDB_MSG_TYPE_CM_TABLE_META, "table-meta" ) TAOS_DEFINE_MESSAGE_TYPE( TSDB_MSG_TYPE_CM_STABLE_VGROUP, "stable-vgroup" ) -TAOS_DEFINE_MESSAGE_TYPE( TSDB_MSG_TYPE_CM_TABLES_META, "tables-meta" ) +TAOS_DEFINE_MESSAGE_TYPE( TSDB_MSG_TYPE_CM_TABLES_META, "multiTable-meta" ) TAOS_DEFINE_MESSAGE_TYPE( TSDB_MSG_TYPE_CM_ALTER_STREAM, "alter-stream" ) TAOS_DEFINE_MESSAGE_TYPE( TSDB_MSG_TYPE_CM_SHOW, "show" ) TAOS_DEFINE_MESSAGE_TYPE( TSDB_MSG_TYPE_CM_RETRIEVE, "retrieve" ) @@ -294,6 +294,8 @@ typedef struct { typedef struct { char name[TSDB_TABLE_FNAME_LEN]; + // if user specify DROP STABLE, this flag will be set. And an error will be returned if it is not a super table + int8_t supertable; int8_t igNotExists; } SCMDropTableMsg; @@ -703,8 +705,9 @@ typedef struct { } STableInfoMsg; typedef struct { + int32_t numOfVgroups; int32_t numOfTables; - char tableIds[]; + char tableNames[]; } SMultiTableInfoMsg; typedef struct SSTableVgroupMsg { @@ -753,8 +756,9 @@ typedef struct STableMetaMsg { typedef struct SMultiTableMeta { int32_t numOfTables; + int32_t numOfVgroup; int32_t contLen; - char metas[]; + char meta[]; } SMultiTableMeta; typedef struct { diff --git a/src/inc/tsdb.h b/src/inc/tsdb.h index 1ba5131f6db82cf7f0fdd512b7dbf94bd68914dc..21d55d5d0fea0312b07e17eee2cbfc91e2175fc1 100644 --- a/src/inc/tsdb.h +++ b/src/inc/tsdb.h @@ -69,9 +69,13 @@ typedef struct { int8_t precision; int8_t compression; int8_t update; - int8_t cacheLastRow; + int8_t cacheLastRow; // 0:no cache, 1: cache last row, 2: cache last NULL column 3: 1&2 } STsdbCfg; +#define CACHE_NO_LAST(c) ((c)->cacheLastRow == 0) +#define CACHE_LAST_ROW(c) (((c)->cacheLastRow & 1) > 0) +#define CACHE_LAST_NULL_COLUMN(c) (((c)->cacheLastRow & 2) > 0) + // --------- TSDB REPOSITORY USAGE STATISTICS typedef struct { int64_t totalStorage; // total bytes occupie @@ -261,6 +265,12 @@ TsdbQueryHandleT *tsdbQueryTables(STsdbRepo *tsdb, STsdbQueryCond *pCond, STable TsdbQueryHandleT tsdbQueryLastRow(STsdbRepo *tsdb, STsdbQueryCond *pCond, STableGroupInfo *tableInfo, uint64_t qId, SMemRef *pRef); + +TsdbQueryHandleT tsdbQueryCacheLast(STsdbRepo *tsdb, STsdbQueryCond *pCond, STableGroupInfo *groupList, uint64_t qId, SMemRef* pMemRef); + +bool isTsdbCacheLastRow(TsdbQueryHandleT* pQueryHandle); + + /** * get the queried table object list * @param pHandle diff --git a/src/kit/taosdemo/taosdemo.c b/src/kit/taosdemo/taosdemo.c index bae0476b17a65224910725d6e46fed343a912693..ec15a99d14f040df13d06afc29275381b65d306b 100644 --- a/src/kit/taosdemo/taosdemo.c +++ b/src/kit/taosdemo/taosdemo.c @@ -53,6 +53,8 @@ #include "taoserror.h" #include "tutil.h" +#define STMT_IFACE_ENABLED 1 + #define REQ_EXTRA_BUF_LEN 1024 #define RESP_BUF_LEN 4096 @@ -253,12 +255,13 @@ typedef struct SColumn_S { typedef struct SSuperTable_S { char sTblName[MAX_TB_NAME_SIZE+1]; + char dataSource[MAX_TB_NAME_SIZE+1]; // rand_gen or sample + char childTblPrefix[MAX_TB_NAME_SIZE]; + char insertMode[MAX_TB_NAME_SIZE]; // taosc, rest + uint16_t childTblExists; int64_t childTblCount; - bool childTblExists; // 0: no, 1: yes uint64_t batchCreateTableNum; // 0: no batch, > 0: batch table number in one sql uint8_t autoCreateTable; // 0: create sub table, 1: auto create sub table - char childTblPrefix[MAX_TB_NAME_SIZE]; - char dataSource[MAX_TB_NAME_SIZE+1]; // rand_gen or sample uint16_t iface; // 0: taosc, 1: rest, 2: stmt int64_t childTblLimit; uint64_t childTblOffset; @@ -377,7 +380,7 @@ typedef struct SDbs_S { typedef struct SpecifiedQueryInfo_S { uint64_t queryInterval; // 0: unlimit > 0 loop/s uint32_t concurrent; - uint64_t sqlCount; + int sqlCount; uint32_t asyncMode; // 0: sync, 1: async uint64_t subscribeInterval; // ms uint64_t queryTimes; @@ -386,6 +389,7 @@ typedef struct SpecifiedQueryInfo_S { char sql[MAX_QUERY_SQL_COUNT][MAX_QUERY_SQL_LENGTH+1]; char result[MAX_QUERY_SQL_COUNT][MAX_FILE_NAME_LEN+1]; int resubAfterConsume[MAX_QUERY_SQL_COUNT]; + int endAfterConsume[MAX_QUERY_SQL_COUNT]; TAOS_SUB* tsub[MAX_QUERY_SQL_COUNT]; char topic[MAX_QUERY_SQL_COUNT][32]; int consumed[MAX_QUERY_SQL_COUNT]; @@ -404,10 +408,11 @@ typedef struct SuperQueryInfo_S { uint64_t queryTimes; int64_t childTblCount; char childTblPrefix[MAX_TB_NAME_SIZE]; - uint64_t sqlCount; + int sqlCount; char sql[MAX_QUERY_SQL_COUNT][MAX_QUERY_SQL_LENGTH+1]; char result[MAX_QUERY_SQL_COUNT][MAX_FILE_NAME_LEN+1]; int resubAfterConsume; + int endAfterConsume; TAOS_SUB* tsub[MAX_QUERY_SQL_COUNT]; char* childTblName; @@ -470,6 +475,7 @@ typedef struct SThreadInfo_S { // seq of query or subscribe uint64_t querySeq; // sequence number of sql command + TAOS_SUB* tsub; } threadInfo; @@ -825,7 +831,7 @@ static void parse_args(int argc, char *argv[], SArguments *arguments) { if ((argc == i+1) || (!isStringNumber(argv[i+1]))) { printHelp(); - errorPrint("%s", "\n\t-q need a number following!\nQuery mode -- 0: SYNC, 1: ASYNC. Default is SYNC.\n"); + errorPrint("%s", "\n\t-q need a number following!\nQuery mode -- 0: SYNC, not-0: ASYNC. Default is SYNC.\n"); exit(EXIT_FAILURE); } arguments->async_mode = atoi(argv[++i]); @@ -929,6 +935,7 @@ static void parse_args(int argc, char *argv[], SArguments *arguments) { && strcasecmp(argv[i], "BIGINT") && strcasecmp(argv[i], "DOUBLE") && strcasecmp(argv[i], "BINARY") + && strcasecmp(argv[i], "TIMESTAMP") && strcasecmp(argv[i], "NCHAR")) { printHelp(); errorPrint("%s", "-b: Invalid data_type!\n"); @@ -950,6 +957,7 @@ static void parse_args(int argc, char *argv[], SArguments *arguments) { && strcasecmp(token, "BIGINT") && strcasecmp(token, "DOUBLE") && strcasecmp(token, "BINARY") + && strcasecmp(token, "TIMESTAMP") && strcasecmp(token, "NCHAR")) { printHelp(); free(g_dupstr); @@ -1051,6 +1059,19 @@ static void parse_args(int argc, char *argv[], SArguments *arguments) { } } + int columnCount; + for (columnCount = 0; columnCount < MAX_NUM_DATATYPE; columnCount ++) { + if (g_args.datatype[columnCount] == NULL) { + break; + } + } + + if (0 == columnCount) { + perror("data type error!"); + exit(-1); + } + g_args.num_of_CPR = columnCount; + if (((arguments->debug_print) && (arguments->metaFile == NULL)) || arguments->verbose_print) { printf("###################################################################\n"); @@ -1073,7 +1094,7 @@ static void parse_args(int argc, char *argv[], SArguments *arguments) { } printf("# Insertion interval: %"PRIu64"\n", arguments->insert_interval); - printf("# Number of records per req: %ud\n", + printf("# Number of records per req: %u\n", arguments->num_of_RPR); printf("# Max SQL length: %"PRIu64"\n", arguments->max_sql_len); @@ -1132,11 +1153,11 @@ static int queryDbExec(TAOS *taos, char *command, QUERY_TYPE type, bool quiet) { } } + verbosePrint("%s() LN%d - command: %s\n", __func__, __LINE__, command); if (code != 0) { if (!quiet) { - debugPrint("%s() LN%d - command: %s\n", __func__, __LINE__, command); - errorPrint("Failed to execute %s, command length: %d, reason: %s\n", - command, (int)strlen(command), taos_errstr(res)); + errorPrint("Failed to execute %s, reason: %s\n", + command, taos_errstr(res)); } taos_free_result(res); //taos_close(taos); @@ -1365,7 +1386,7 @@ static int printfInsertMeta() { g_Dbs.threadCountByCreateTbl); printf("top insert interval: \033[33m%"PRIu64"\033[0m\n", g_args.insert_interval); - printf("number of records per req: \033[33m%ud\033[0m\n", + printf("number of records per req: \033[33m%u\033[0m\n", g_args.num_of_RPR); printf("max sql length: \033[33m%"PRIu64"\033[0m\n", g_args.max_sql_len); @@ -1451,7 +1472,8 @@ static int printfInsertMeta() { if (PRE_CREATE_SUBTBL == g_Dbs.db[i].superTbls[j].autoCreateTable) { printf(" autoCreateTable: \033[33m%s\033[0m\n", "no"); - } else if (AUTO_CREATE_SUBTBL == g_Dbs.db[i].superTbls[j].autoCreateTable) { + } else if (AUTO_CREATE_SUBTBL == + g_Dbs.db[i].superTbls[j].autoCreateTable) { printf(" autoCreateTable: \033[33m%s\033[0m\n", "yes"); } else { printf(" autoCreateTable: \033[33m%s\033[0m\n", "error"); @@ -1491,7 +1513,7 @@ static int printfInsertMeta() { printf(" multiThreadWriteOneTbl: \033[33myes\033[0m\n"); } */ - printf(" interlaceRows: \033[33m%ud\033[0m\n", + printf(" interlaceRows: \033[33m%u\033[0m\n", g_Dbs.db[i].superTbls[j].interlaceRows); if (g_Dbs.db[i].superTbls[j].interlaceRows > 0) { @@ -1569,7 +1591,7 @@ static void printfInsertMetaToFile(FILE* fp) { fprintf(fp, "resultFile: %s\n", g_Dbs.resultFile); fprintf(fp, "thread num of insert data: %d\n", g_Dbs.threadCount); fprintf(fp, "thread num of create table: %d\n", g_Dbs.threadCountByCreateTbl); - fprintf(fp, "number of records per req: %ud\n", g_args.num_of_RPR); + fprintf(fp, "number of records per req: %u\n", g_args.num_of_RPR); fprintf(fp, "max sql length: %"PRIu64"\n", g_args.max_sql_len); fprintf(fp, "database count: %d\n", g_Dbs.dbCount); @@ -1666,7 +1688,7 @@ static void printfInsertMetaToFile(FILE* fp) { (g_Dbs.db[i].superTbls[j].iface==REST_IFACE)?"rest":"stmt"); fprintf(fp, " insertRows: %"PRId64"\n", g_Dbs.db[i].superTbls[j].insertRows); - fprintf(fp, " interlace rows: %ud\n", + fprintf(fp, " interlace rows: %u\n", g_Dbs.db[i].superTbls[j].interlaceRows); if (g_Dbs.db[i].superTbls[j].interlaceRows > 0) { fprintf(fp, " stable insert interval: %"PRIu64"\n", @@ -1679,7 +1701,7 @@ static void printfInsertMetaToFile(FILE* fp) { fprintf(fp, " multiThreadWriteOneTbl: yes\n"); } */ - fprintf(fp, " interlaceRows: %ud\n", + fprintf(fp, " interlaceRows: %u\n", g_Dbs.db[i].superTbls[j].interlaceRows); fprintf(fp, " disorderRange: %d\n", g_Dbs.db[i].superTbls[j].disorderRange); @@ -1755,7 +1777,7 @@ static void printfQueryMeta() { if ((SUBSCRIBE_TEST == g_args.test_mode) || (QUERY_TEST == g_args.test_mode)) { printf("specified table query info: \n"); - printf("sqlCount: \033[33m%"PRIu64"\033[0m\n", + printf("sqlCount: \033[33m%d\033[0m\n", g_queryInfo.specifiedQueryInfo.sqlCount); if (g_queryInfo.specifiedQueryInfo.sqlCount > 0) { printf("specified tbl query times:\n"); @@ -1775,15 +1797,15 @@ static void printfQueryMeta() { printf("keepProgress: \033[33m%d\033[0m\n", g_queryInfo.specifiedQueryInfo.subscribeKeepProgress); - for (uint64_t i = 0; i < g_queryInfo.specifiedQueryInfo.sqlCount; i++) { - printf(" sql[%"PRIu64"]: \033[33m%s\033[0m\n", + for (int i = 0; i < g_queryInfo.specifiedQueryInfo.sqlCount; i++) { + printf(" sql[%d]: \033[33m%s\033[0m\n", i, g_queryInfo.specifiedQueryInfo.sql[i]); } printf("\n"); } printf("super table query info:\n"); - printf("sqlCount: \033[33m%"PRIu64"\033[0m\n", + printf("sqlCount: \033[33m%d\033[0m\n", g_queryInfo.superQueryInfo.sqlCount); if (g_queryInfo.superQueryInfo.sqlCount > 0) { @@ -2732,8 +2754,6 @@ static int createSuperTable( snprintf(command, BUFFER_SIZE, "create table if not exists %s.%s (ts timestamp%s) tags %s", dbName, superTbl->sTblName, cols, tags); - verbosePrint("%s() LN%d: %s\n", __func__, __LINE__, command); - if (0 != queryDbExec(taos, command, NO_INSERT_TYPE, false)) { errorPrint( "create supertable %s failed!\n\n", superTbl->sTblName); @@ -2756,7 +2776,6 @@ static int createDatabasesAndStables() { for (int i = 0; i < g_Dbs.dbCount; i++) { if (g_Dbs.db[i].drop) { sprintf(command, "drop database if exists %s;", g_Dbs.db[i].dbName); - verbosePrint("%s() %d command: %s\n", __func__, __LINE__, command); if (0 != queryDbExec(taos, command, NO_INSERT_TYPE, false)) { taos_close(taos); return -1; @@ -2829,7 +2848,6 @@ static int createDatabasesAndStables() { " precision \'%s\';", g_Dbs.db[i].dbCfg.precision); } - debugPrint("%s() %d command: %s\n", __func__, __LINE__, command); if (0 != queryDbExec(taos, command, NO_INSERT_TYPE, false)) { taos_close(taos); errorPrint( "\ncreate database %s failed!\n\n", g_Dbs.db[i].dbName); @@ -2846,8 +2864,6 @@ static int createDatabasesAndStables() { for (uint64_t j = 0; j < g_Dbs.db[i].superTblCount; j++) { sprintf(command, "describe %s.%s;", g_Dbs.db[i].dbName, g_Dbs.db[i].superTbls[j].sTblName); - verbosePrint("%s() %d command: %s\n", __func__, __LINE__, command); - ret = queryDbExec(taos, command, NO_INSERT_TYPE, true); if ((ret != 0) || (g_Dbs.db[i].drop)) { @@ -2951,7 +2967,6 @@ static void* createTable(void *sarg) } len = 0; - verbosePrint("%s() LN%d %s\n", __func__, __LINE__, buffer); if (0 != queryDbExec(pThreadInfo->taos, buffer, NO_INSERT_TYPE, false)){ errorPrint( "queryDbExec() failed. buffer:\n%s\n", buffer); free(buffer); @@ -2967,7 +2982,6 @@ static void* createTable(void *sarg) } if (0 != len) { - verbosePrint("%s() %d buffer: %s\n", __func__, __LINE__, buffer); if (0 != queryDbExec(pThreadInfo->taos, buffer, NO_INSERT_TYPE, false)) { errorPrint( "queryDbExec() failed. buffer:\n%s\n", buffer); } @@ -2978,7 +2992,7 @@ static void* createTable(void *sarg) } static int startMultiThreadCreateChildTable( - char* cols, int threads, uint64_t startFrom, int64_t ntables, + char* cols, int threads, uint64_t tableFrom, int64_t ntables, char* db_name, SSuperTable* superTblInfo) { pthread_t *pids = malloc(threads * sizeof(pthread_t)); @@ -3022,10 +3036,10 @@ static int startMultiThreadCreateChildTable( return -1; } - pThreadInfo->start_table_from = startFrom; + pThreadInfo->start_table_from = tableFrom; pThreadInfo->ntables = iend_table_to = i < b ? startFrom + a : startFrom + a - 1; - startFrom = pThreadInfo->end_table_to + 1; + pThreadInfo->end_table_to = i < b ? tableFrom + a : tableFrom + a - 1; + tableFrom = pThreadInfo->end_table_to + 1; pThreadInfo->use_metric = true; pThreadInfo->cols = cols; pThreadInfo->minDelay = UINT64_MAX; @@ -3051,61 +3065,61 @@ static void createChildTables() { char tblColsBuf[MAX_SQL_SIZE]; int len; - for (int i = 0; i < g_Dbs.dbCount; i++) { - if (g_Dbs.use_metric) { - if (g_Dbs.db[i].superTblCount > 0) { - // with super table - for (uint64_t j = 0; j < g_Dbs.db[i].superTblCount; j++) { - if ((AUTO_CREATE_SUBTBL == g_Dbs.db[i].superTbls[j].autoCreateTable) - || (TBL_ALREADY_EXISTS == g_Dbs.db[i].superTbls[j].childTblExists)) { - continue; - } + for (int i = 0; i < g_Dbs.dbCount; i++) { + if (g_Dbs.use_metric) { + if (g_Dbs.db[i].superTblCount > 0) { + // with super table + for (int j = 0; j < g_Dbs.db[i].superTblCount; j++) { + if ((AUTO_CREATE_SUBTBL == g_Dbs.db[i].superTbls[j].autoCreateTable) + || (TBL_ALREADY_EXISTS == g_Dbs.db[i].superTbls[j].childTblExists)) { + continue; + } + verbosePrint("%s() LN%d: %s\n", __func__, __LINE__, + g_Dbs.db[i].superTbls[j].colsOfCreateChildTable); + uint64_t startFrom = 0; + g_totalChildTables += g_Dbs.db[i].superTbls[j].childTblCount; + + verbosePrint("%s() LN%d: create %"PRId64" child tables from %"PRIu64"\n", + __func__, __LINE__, g_totalChildTables, startFrom); + + startMultiThreadCreateChildTable( + g_Dbs.db[i].superTbls[j].colsOfCreateChildTable, + g_Dbs.threadCountByCreateTbl, + startFrom, + g_Dbs.db[i].superTbls[j].childTblCount, + g_Dbs.db[i].dbName, &(g_Dbs.db[i].superTbls[j])); + } + } + } else { + // normal table + len = snprintf(tblColsBuf, MAX_SQL_SIZE, "(TS TIMESTAMP"); + for (int j = 0; j < g_args.num_of_CPR; j++) { + if ((strncasecmp(g_args.datatype[j], "BINARY", strlen("BINARY")) == 0) + || (strncasecmp(g_args.datatype[j], + "NCHAR", strlen("NCHAR")) == 0)) { + snprintf(tblColsBuf + len, MAX_SQL_SIZE - len, + ", COL%d %s(%d)", j, g_args.datatype[j], g_args.len_of_binary); + } else { + snprintf(tblColsBuf + len, MAX_SQL_SIZE - len, + ", COL%d %s", j, g_args.datatype[j]); + } + len = strlen(tblColsBuf); + } - verbosePrint("%s() LN%d: %s\n", __func__, __LINE__, - g_Dbs.db[i].superTbls[j].colsOfCreateChildTable); - uint64_t startFrom = 0; - g_totalChildTables += g_Dbs.db[i].superTbls[j].childTblCount; - - verbosePrint("%s() LN%d: create %"PRId64" child tables from %"PRIu64"\n", - __func__, __LINE__, g_totalChildTables, startFrom); - startMultiThreadCreateChildTable( - g_Dbs.db[i].superTbls[j].colsOfCreateChildTable, - g_Dbs.threadCountByCreateTbl, - startFrom, - g_Dbs.db[i].superTbls[j].childTblCount, - g_Dbs.db[i].dbName, &(g_Dbs.db[i].superTbls[j])); + snprintf(tblColsBuf + len, MAX_SQL_SIZE - len, ")"); + + verbosePrint("%s() LN%d: dbName: %s num of tb: %"PRId64" schema: %s\n", + __func__, __LINE__, + g_Dbs.db[i].dbName, g_args.num_of_tables, tblColsBuf); + startMultiThreadCreateChildTable( + tblColsBuf, + g_Dbs.threadCountByCreateTbl, + 0, + g_args.num_of_tables, + g_Dbs.db[i].dbName, + NULL); } - } - } else { - // normal table - len = snprintf(tblColsBuf, MAX_SQL_SIZE, "(TS TIMESTAMP"); - for (int j = 0; j < g_args.num_of_CPR; j++) { - if ((strncasecmp(g_args.datatype[j], "BINARY", strlen("BINARY")) == 0) - || (strncasecmp(g_args.datatype[j], - "NCHAR", strlen("NCHAR")) == 0)) { - snprintf(tblColsBuf + len, MAX_SQL_SIZE - len, - ", COL%d %s(%d)", j, g_args.datatype[j], g_args.len_of_binary); - } else { - snprintf(tblColsBuf + len, MAX_SQL_SIZE - len, - ", COL%d %s", j, g_args.datatype[j]); - } - len = strlen(tblColsBuf); - } - - snprintf(tblColsBuf + len, MAX_SQL_SIZE - len, ")"); - - verbosePrint("%s() LN%d: dbName: %s num of tb: %"PRId64" schema: %s\n", - __func__, __LINE__, - g_Dbs.db[i].dbName, g_args.num_of_tables, tblColsBuf); - startMultiThreadCreateChildTable( - tblColsBuf, - g_Dbs.threadCountByCreateTbl, - 0, - g_args.num_of_tables, - g_Dbs.db[i].dbName, - NULL); } - } } /* @@ -3566,9 +3580,9 @@ static bool getMetaFromInsertJsonFile(cJSON* root) { // rows per table need be less than insert batch if (g_args.interlace_rows > g_args.num_of_RPR) { - printf("NOTICE: interlace rows value %ud > num_of_records_per_req %ud\n\n", + printf("NOTICE: interlace rows value %u > num_of_records_per_req %u\n\n", g_args.interlace_rows, g_args.num_of_RPR); - printf(" interlace rows value will be set to num_of_records_per_req %ud\n\n", + printf(" interlace rows value will be set to num_of_records_per_req %u\n\n", g_args.num_of_RPR); prompt(); g_args.interlace_rows = g_args.num_of_RPR; @@ -3799,36 +3813,40 @@ static bool getMetaFromInsertJsonFile(cJSON* root) { // dbinfo cJSON *stbName = cJSON_GetObjectItem(stbInfo, "name"); - if (!stbName || stbName->type != cJSON_String || stbName->valuestring == NULL) { + if (!stbName || stbName->type != cJSON_String + || stbName->valuestring == NULL) { errorPrint("%s() LN%d, failed to read json, stb name not found\n", __func__, __LINE__); goto PARSE_OVER; } - tstrncpy(g_Dbs.db[i].superTbls[j].sTblName, stbName->valuestring, MAX_TB_NAME_SIZE); + tstrncpy(g_Dbs.db[i].superTbls[j].sTblName, stbName->valuestring, + MAX_TB_NAME_SIZE); cJSON *prefix = cJSON_GetObjectItem(stbInfo, "childtable_prefix"); if (!prefix || prefix->type != cJSON_String || prefix->valuestring == NULL) { printf("ERROR: failed to read json, childtable_prefix not found\n"); goto PARSE_OVER; } - tstrncpy(g_Dbs.db[i].superTbls[j].childTblPrefix, prefix->valuestring, MAX_DB_NAME_SIZE); + tstrncpy(g_Dbs.db[i].superTbls[j].childTblPrefix, prefix->valuestring, + MAX_DB_NAME_SIZE); - cJSON *autoCreateTbl = cJSON_GetObjectItem(stbInfo, "auto_create_table"); // yes, no, null + cJSON *autoCreateTbl = cJSON_GetObjectItem(stbInfo, "auto_create_table"); if (autoCreateTbl && autoCreateTbl->type == cJSON_String && autoCreateTbl->valuestring != NULL) { - if (0 == strncasecmp(autoCreateTbl->valuestring, "yes", 3)) { - g_Dbs.db[i].superTbls[j].autoCreateTable = AUTO_CREATE_SUBTBL; - } else if (0 == strncasecmp(autoCreateTbl->valuestring, "no", 2)) { - g_Dbs.db[i].superTbls[j].autoCreateTable = PRE_CREATE_SUBTBL; - } else { - g_Dbs.db[i].superTbls[j].autoCreateTable = PRE_CREATE_SUBTBL; - } + if ((0 == strncasecmp(autoCreateTbl->valuestring, "yes", 3)) + && (TBL_ALREADY_EXISTS != g_Dbs.db[i].superTbls[j].childTblExists)) { + g_Dbs.db[i].superTbls[j].autoCreateTable = AUTO_CREATE_SUBTBL; + } else if (0 == strncasecmp(autoCreateTbl->valuestring, "no", 2)) { + g_Dbs.db[i].superTbls[j].autoCreateTable = PRE_CREATE_SUBTBL; + } else { + g_Dbs.db[i].superTbls[j].autoCreateTable = PRE_CREATE_SUBTBL; + } } else if (!autoCreateTbl) { - g_Dbs.db[i].superTbls[j].autoCreateTable = PRE_CREATE_SUBTBL; + g_Dbs.db[i].superTbls[j].autoCreateTable = PRE_CREATE_SUBTBL; } else { - printf("ERROR: failed to read json, auto_create_table not found\n"); - goto PARSE_OVER; + printf("ERROR: failed to read json, auto_create_table not found\n"); + goto PARSE_OVER; } cJSON* batchCreateTbl = cJSON_GetObjectItem(stbInfo, "batch_create_tbl_num"); @@ -3862,6 +3880,10 @@ static bool getMetaFromInsertJsonFile(cJSON* root) { goto PARSE_OVER; } + if (TBL_ALREADY_EXISTS == g_Dbs.db[i].superTbls[j].childTblExists) { + g_Dbs.db[i].superTbls[j].autoCreateTable = PRE_CREATE_SUBTBL; + } + cJSON* count = cJSON_GetObjectItem(stbInfo, "childtable_count"); if (!count || count->type != cJSON_Number || 0 >= count->valueint) { errorPrint("%s() LN%d, failed to read json, childtable_count input mistake\n", @@ -3919,7 +3941,8 @@ static bool getMetaFromInsertJsonFile(cJSON* root) { cJSON* childTbl_offset = cJSON_GetObjectItem(stbInfo, "childtable_offset"); if ((childTbl_offset) && (g_Dbs.db[i].drop != true) && (g_Dbs.db[i].superTbls[j].childTblExists == TBL_ALREADY_EXISTS)) { - if (childTbl_offset->type != cJSON_Number || 0 > childTbl_offset->valueint) { + if ((childTbl_offset->type != cJSON_Number) + || (0 > childTbl_offset->valueint)) { printf("ERROR: failed to read json, childtable_offset\n"); goto PARSE_OVER; } @@ -3975,7 +3998,8 @@ static bool getMetaFromInsertJsonFile(cJSON* root) { } cJSON *tagsFile = cJSON_GetObjectItem(stbInfo, "tags_file"); - if (tagsFile && tagsFile->type == cJSON_String && tagsFile->valuestring != NULL) { + if ((tagsFile && tagsFile->type == cJSON_String) + && (tagsFile->valuestring != NULL)) { tstrncpy(g_Dbs.db[i].superTbls[j].tagsFile, tagsFile->valuestring, MAX_FILE_NAME_LEN); if (0 == g_Dbs.db[i].superTbls[j].tagsFile[0]) { @@ -4035,10 +4059,10 @@ static bool getMetaFromInsertJsonFile(cJSON* root) { g_Dbs.db[i].superTbls[j].interlaceRows = stbInterlaceRows->valueint; // rows per table need be less than insert batch if (g_Dbs.db[i].superTbls[j].interlaceRows > g_args.num_of_RPR) { - printf("NOTICE: db[%d].superTbl[%d]'s interlace rows value %ud > num_of_records_per_req %ud\n\n", + printf("NOTICE: db[%d].superTbl[%d]'s interlace rows value %u > num_of_records_per_req %u\n\n", i, j, g_Dbs.db[i].superTbls[j].interlaceRows, g_args.num_of_RPR); - printf(" interlace rows value will be set to num_of_records_per_req %ud\n\n", + printf(" interlace rows value will be set to num_of_records_per_req %u\n\n", g_args.num_of_RPR); prompt(); g_Dbs.db[i].superTbls[j].interlaceRows = g_args.num_of_RPR; @@ -4255,7 +4279,7 @@ static bool getMetaFromQueryJsonFile(cJSON* root) { if (concurrent && concurrent->type == cJSON_Number) { if (concurrent->valueint <= 0) { errorPrint( - "%s() LN%d, query sqlCount %"PRIu64" or concurrent %d is not correct.\n", + "%s() LN%d, query sqlCount %d or concurrent %d is not correct.\n", __func__, __LINE__, g_queryInfo.specifiedQueryInfo.sqlCount, g_queryInfo.specifiedQueryInfo.concurrent); @@ -4354,6 +4378,17 @@ static bool getMetaFromQueryJsonFile(cJSON* root) { tstrncpy(g_queryInfo.specifiedQueryInfo.sql[j], sqlStr->valuestring, MAX_QUERY_SQL_LENGTH); + cJSON* endAfterConsume = + cJSON_GetObjectItem(specifiedQuery, "endAfterConsume"); + if (endAfterConsume + && endAfterConsume->type == cJSON_Number) { + g_queryInfo.specifiedQueryInfo.endAfterConsume[j] + = endAfterConsume->valueint; + } else if (!endAfterConsume) { + // default value is -1, which mean infinite loop + g_queryInfo.specifiedQueryInfo.endAfterConsume[j] = -1; + } + cJSON* resubAfterConsume = cJSON_GetObjectItem(specifiedQuery, "resubAfterConsume"); if (resubAfterConsume @@ -4361,9 +4396,8 @@ static bool getMetaFromQueryJsonFile(cJSON* root) { g_queryInfo.specifiedQueryInfo.resubAfterConsume[j] = resubAfterConsume->valueint; } else if (!resubAfterConsume) { - //printf("failed to read json, subscribe interval no found\n"); - //goto PARSE_OVER; - g_queryInfo.specifiedQueryInfo.resubAfterConsume[j] = 1; + // default value is -1, which mean do not resub + g_queryInfo.specifiedQueryInfo.resubAfterConsume[j] = -1; } cJSON *result = cJSON_GetObjectItem(sql, "result"); @@ -4507,16 +4541,26 @@ static bool getMetaFromQueryJsonFile(cJSON* root) { g_queryInfo.superQueryInfo.subscribeKeepProgress = 0; } + cJSON* superEndAfterConsume = + cJSON_GetObjectItem(superQuery, "endAfterConsume"); + if (superEndAfterConsume + && superEndAfterConsume->type == cJSON_Number) { + g_queryInfo.superQueryInfo.endAfterConsume = + superEndAfterConsume->valueint; + } else if (!superEndAfterConsume) { + // default value is -1, which mean do not resub + g_queryInfo.superQueryInfo.endAfterConsume = -1; + } + cJSON* superResubAfterConsume = - cJSON_GetObjectItem(superQuery, "resubAfterConsume"); + cJSON_GetObjectItem(superQuery, "endAfterConsume"); if (superResubAfterConsume && superResubAfterConsume->type == cJSON_Number) { - g_queryInfo.superQueryInfo.resubAfterConsume = + g_queryInfo.superQueryInfo.endAfterConsume = superResubAfterConsume->valueint; } else if (!superResubAfterConsume) { - //printf("failed to read json, subscribe interval no found\n"); - ////goto PARSE_OVER; - g_queryInfo.superQueryInfo.resubAfterConsume = 1; + // default value is -1, which mean do not resub + g_queryInfo.superQueryInfo.endAfterConsume = -1; } // supert table sqls @@ -4699,8 +4743,8 @@ static int getRowDataFromSample( static int64_t generateStbRowData( SSuperTable* stbInfo, - char* recBuf, int64_t timestamp - ) { + char* recBuf, int64_t timestamp) +{ int64_t dataLen = 0; char *pstr = recBuf; int64_t maxLen = MAX_DATA_SIZE; @@ -4728,23 +4772,23 @@ static int64_t generateStbRowData( dataLen += snprintf(pstr + dataLen, maxLen - dataLen, "\'%s\',", buf); tmfree(buf); } else if (0 == strncasecmp(stbInfo->columns[i].dataType, - "INT", 3)) { + "INT", strlen("INT"))) { dataLen += snprintf(pstr + dataLen, maxLen - dataLen, "%d,", rand_int()); } else if (0 == strncasecmp(stbInfo->columns[i].dataType, - "BIGINT", 6)) { + "BIGINT", strlen("BIGINT"))) { dataLen += snprintf(pstr + dataLen, maxLen - dataLen, "%"PRId64",", rand_bigint()); } else if (0 == strncasecmp(stbInfo->columns[i].dataType, - "FLOAT", 5)) { + "FLOAT", strlen("FLOAT"))) { dataLen += snprintf(pstr + dataLen, maxLen - dataLen, "%f,", rand_float()); } else if (0 == strncasecmp(stbInfo->columns[i].dataType, - "DOUBLE", 6)) { + "DOUBLE", strlen("DOUBLE"))) { dataLen += snprintf(pstr + dataLen, maxLen - dataLen, "%f,", rand_double()); } else if (0 == strncasecmp(stbInfo->columns[i].dataType, - "SMALLINT", 8)) { + "SMALLINT", strlen("SMALLINT"))) { dataLen += snprintf(pstr + dataLen, maxLen - dataLen, "%d,", rand_smallint()); } else if (0 == strncasecmp(stbInfo->columns[i].dataType, @@ -4778,42 +4822,34 @@ static int64_t generateData(char *recBuf, char **data_type, memset(recBuf, 0, MAX_DATA_SIZE); char *pstr = recBuf; pstr += sprintf(pstr, "(%" PRId64, timestamp); - int c = 0; - for (; c < MAX_NUM_DATATYPE; c++) { - if (data_type[c] == NULL) { - break; - } - } + int columnCount = g_args.num_of_CPR; - if (0 == c) { - perror("data type error!"); - exit(-1); - } - - for (int i = 0; i < c; i++) { - if (strcasecmp(data_type[i % c], "TINYINT") == 0) { + for (int i = 0; i < columnCount; i++) { + if (strcasecmp(data_type[i % columnCount], "TINYINT") == 0) { pstr += sprintf(pstr, ",%d", rand_tinyint() ); - } else if (strcasecmp(data_type[i % c], "SMALLINT") == 0) { + } else if (strcasecmp(data_type[i % columnCount], "SMALLINT") == 0) { pstr += sprintf(pstr, ",%d", rand_smallint()); - } else if (strcasecmp(data_type[i % c], "INT") == 0) { + } else if (strcasecmp(data_type[i % columnCount], "INT") == 0) { pstr += sprintf(pstr, ",%d", rand_int()); - } else if (strcasecmp(data_type[i % c], "BIGINT") == 0) { + } else if (strcasecmp(data_type[i % columnCount], "BIGINT") == 0) { + pstr += sprintf(pstr, ",%" PRId64, rand_bigint()); + } else if (strcasecmp(data_type[i % columnCount], "TIMESTAMP") == 0) { pstr += sprintf(pstr, ",%" PRId64, rand_bigint()); - } else if (strcasecmp(data_type[i % c], "FLOAT") == 0) { + } else if (strcasecmp(data_type[i % columnCount], "FLOAT") == 0) { pstr += sprintf(pstr, ",%10.4f", rand_float()); - } else if (strcasecmp(data_type[i % c], "DOUBLE") == 0) { + } else if (strcasecmp(data_type[i % columnCount], "DOUBLE") == 0) { double t = rand_double(); pstr += sprintf(pstr, ",%20.8f", t); - } else if (strcasecmp(data_type[i % c], "BOOL") == 0) { + } else if (strcasecmp(data_type[i % columnCount], "BOOL") == 0) { bool b = rand_bool() & 1; pstr += sprintf(pstr, ",%s", b ? "true" : "false"); - } else if (strcasecmp(data_type[i % c], "BINARY") == 0) { + } else if (strcasecmp(data_type[i % columnCount], "BINARY") == 0) { char *s = malloc(lenOfBinary); rand_string(s, lenOfBinary); pstr += sprintf(pstr, ",\"%s\"", s); free(s); - } else if (strcasecmp(data_type[i % c], "NCHAR") == 0) { + } else if (strcasecmp(data_type[i % columnCount], "NCHAR") == 0) { char *s = malloc(lenOfBinary); rand_string(s, lenOfBinary); pstr += sprintf(pstr, ",\"%s\"", s); @@ -4860,77 +4896,94 @@ static int prepareSampleDataForSTable(SSuperTable *superTblInfo) { return 0; } -static int64_t execInsert(threadInfo *pThreadInfo, uint64_t k) +static int32_t execInsert(threadInfo *pThreadInfo, uint32_t k) { - int affectedRows; - SSuperTable* superTblInfo = pThreadInfo->superTblInfo; + int32_t affectedRows; + SSuperTable* superTblInfo = pThreadInfo->superTblInfo; - verbosePrint("[%d] %s() LN%d %s\n", pThreadInfo->threadID, + verbosePrint("[%d] %s() LN%d %s\n", pThreadInfo->threadID, __func__, __LINE__, pThreadInfo->buffer); - if (superTblInfo) { - if (superTblInfo->iface == TAOSC_IFACE) { - affectedRows = queryDbExec( - pThreadInfo->taos, - pThreadInfo->buffer, INSERT_TYPE, false); - } else if (superTblInfo->iface == REST_IFACE) { - if (0 != postProceSql(g_Dbs.host, &g_Dbs.serv_addr, g_Dbs.port, - pThreadInfo->buffer, NULL /* not set result file */)) { - affectedRows = -1; - printf("========restful return fail, threadID[%d]\n", - pThreadInfo->threadID); - } else { - affectedRows = k; - } - } else if (superTblInfo->iface == STMT_IFACE) { - debugPrint("%s() LN%d, stmt=%p", __func__, __LINE__, pThreadInfo->stmt); - if (0 != taos_stmt_execute(pThreadInfo->stmt)) { - errorPrint("%s() LN%d, failied to execute insert statement\n", - __func__, __LINE__); - exit(-1); - } - affectedRows = k; - } else { - errorPrint("%s() LN%d: unknown insert mode: %d\n", - __func__, __LINE__, superTblInfo->iface); - affectedRows = 0; + uint16_t iface; + if (superTblInfo) + iface = superTblInfo->iface; + else + iface = g_args.iface; + + debugPrint("[%d] %s() LN%d %s\n", pThreadInfo->threadID, + __func__, __LINE__, + (g_args.iface==TAOSC_IFACE)? + "taosc":(g_args.iface==REST_IFACE)?"rest":"stmt"); + + switch(iface) { + case TAOSC_IFACE: + affectedRows = queryDbExec( + pThreadInfo->taos, + pThreadInfo->buffer, INSERT_TYPE, false); + break; + + case REST_IFACE: + if (0 != postProceSql(g_Dbs.host, &g_Dbs.serv_addr, g_Dbs.port, + pThreadInfo->buffer, NULL /* not set result file */)) { + affectedRows = -1; + printf("========restful return fail, threadID[%d]\n", + pThreadInfo->threadID); + } else { + affectedRows = k; + } + break; + + case STMT_IFACE: + debugPrint("%s() LN%d, stmt=%p", __func__, __LINE__, pThreadInfo->stmt); + if (0 != taos_stmt_execute(pThreadInfo->stmt)) { + errorPrint("%s() LN%d, failied to execute insert statement\n", + __func__, __LINE__); + exit(-1); + } + affectedRows = k; + break; + + default: + errorPrint("%s() LN%d: unknown insert mode: %d\n", + __func__, __LINE__, superTblInfo->iface); + affectedRows = 0; } - } else { - affectedRows = queryDbExec(pThreadInfo->taos, pThreadInfo->buffer, INSERT_TYPE, false); - } - return affectedRows; + return affectedRows; } static void getTableName(char *pTblName, threadInfo* pThreadInfo, uint64_t tableSeq) { - SSuperTable* superTblInfo = pThreadInfo->superTblInfo; - if ((superTblInfo) - && (AUTO_CREATE_SUBTBL != superTblInfo->autoCreateTable)) { - if (superTblInfo->childTblLimit > 0) { - snprintf(pTblName, TSDB_TABLE_NAME_LEN, "%s", - superTblInfo->childTblName + - (tableSeq - superTblInfo->childTblOffset) * TSDB_TABLE_NAME_LEN); + SSuperTable* superTblInfo = pThreadInfo->superTblInfo; + if (superTblInfo) { + if (AUTO_CREATE_SUBTBL != superTblInfo->autoCreateTable) { + if (superTblInfo->childTblLimit > 0) { + snprintf(pTblName, TSDB_TABLE_NAME_LEN, "%s", + superTblInfo->childTblName + + (tableSeq - superTblInfo->childTblOffset) * TSDB_TABLE_NAME_LEN); + } else { + verbosePrint("[%d] %s() LN%d: from=%"PRIu64" count=%"PRId64" seq=%"PRIu64"\n", + pThreadInfo->threadID, __func__, __LINE__, + pThreadInfo->start_table_from, + pThreadInfo->ntables, tableSeq); + snprintf(pTblName, TSDB_TABLE_NAME_LEN, "%s", + superTblInfo->childTblName + tableSeq * TSDB_TABLE_NAME_LEN); + } + } else { + snprintf(pTblName, TSDB_TABLE_NAME_LEN, "%s%"PRIu64"", + superTblInfo->childTblPrefix, tableSeq); + } } else { - - verbosePrint("[%d] %s() LN%d: from=%"PRIu64" count=%"PRId64" seq=%"PRIu64"\n", - pThreadInfo->threadID, __func__, __LINE__, - pThreadInfo->start_table_from, - pThreadInfo->ntables, tableSeq); - snprintf(pTblName, TSDB_TABLE_NAME_LEN, "%s", - superTblInfo->childTblName + tableSeq * TSDB_TABLE_NAME_LEN); + snprintf(pTblName, TSDB_TABLE_NAME_LEN, "%s%"PRIu64"", + g_args.tb_prefix, tableSeq); } - } else { - snprintf(pTblName, TSDB_TABLE_NAME_LEN, "%s%"PRIu64"", - g_args.tb_prefix, tableSeq); - } } -static int64_t generateDataTailWithoutStb( +static int32_t generateDataTailWithoutStb( uint32_t batch, char* buffer, int64_t remainderBufLen, int64_t insertRows, - uint64_t startFrom, int64_t startTime, + uint64_t recordFrom, int64_t startTime, /* int64_t *pSamplePos, */int64_t *dataLen) { uint64_t len = 0; @@ -4938,7 +4991,7 @@ static int64_t generateDataTailWithoutStb( verbosePrint("%s() LN%d batch=%d\n", __func__, __LINE__, batch); - int64_t k = 0; + int32_t k = 0; for (k = 0; k < batch;) { char data[MAX_DATA_SIZE]; memset(data, 0, MAX_DATA_SIZE); @@ -4950,7 +5003,7 @@ static int64_t generateDataTailWithoutStb( retLen = generateData(data, data_type, startTime + getTSRandTail( - (int64_t)DEFAULT_TIMESTAMP_STEP, k, + (int64_t) DEFAULT_TIMESTAMP_STEP, k, g_args.disorderRatio, g_args.disorderRange), lenOfBinary); @@ -4963,12 +5016,12 @@ static int64_t generateDataTailWithoutStb( len += retLen; remainderBufLen -= retLen; - verbosePrint("%s() LN%d len=%"PRIu64" k=%"PRIu64" \nbuffer=%s\n", + verbosePrint("%s() LN%d len=%"PRIu64" k=%d \nbuffer=%s\n", __func__, __LINE__, len, k, buffer); - startFrom ++; + recordFrom ++; - if (startFrom >= insertRows) { + if (recordFrom >= insertRows) { break; } } @@ -4997,7 +5050,7 @@ static int32_t generateStbDataTail( SSuperTable* superTblInfo, uint32_t batch, char* buffer, int64_t remainderBufLen, int64_t insertRows, - uint64_t startFrom, int64_t startTime, + uint64_t recordFrom, int64_t startTime, int64_t *pSamplePos, int64_t *dataLen) { uint64_t len = 0; @@ -5009,7 +5062,7 @@ static int32_t generateStbDataTail( } else { tsRand = false; } - verbosePrint("%s() LN%d batch=%ud\n", __func__, __LINE__, batch); + verbosePrint("%s() LN%d batch=%u\n", __func__, __LINE__, batch); int32_t k = 0; for (k = 0; k < batch;) { @@ -5043,12 +5096,12 @@ static int32_t generateStbDataTail( len += retLen; remainderBufLen -= retLen; - verbosePrint("%s() LN%d len=%"PRIu64" k=%ud \nbuffer=%s\n", + verbosePrint("%s() LN%d len=%"PRIu64" k=%u \nbuffer=%s\n", __func__, __LINE__, len, k, buffer); - startFrom ++; + recordFrom ++; - if (startFrom >= insertRows) { + if (recordFrom >= insertRows) { break; } } @@ -5171,7 +5224,7 @@ static int32_t generateStbInterlaceData( int64_t dataLen = 0; - verbosePrint("[%d] %s() LN%d i=%"PRIu64" batchPerTblTimes=%ud batchPerTbl = %ud\n", + verbosePrint("[%d] %s() LN%d i=%"PRIu64" batchPerTblTimes=%u batchPerTbl = %u\n", pThreadInfo->threadID, __func__, __LINE__, i, batchPerTblTimes, batchPerTbl); @@ -5189,7 +5242,7 @@ static int32_t generateStbInterlaceData( pstr += dataLen; *pRemainderBufLen -= dataLen; } else { - debugPrint("%s() LN%d, generated data tail: %ud, not equal batch per table: %ud\n", + debugPrint("%s() LN%d, generated data tail: %u, not equal batch per table: %u\n", __func__, __LINE__, k, batchPerTbl); pstr -= headLen; pstr[0] = '\0'; @@ -5200,10 +5253,11 @@ static int32_t generateStbInterlaceData( } static int64_t generateInterlaceDataWithoutStb( - char *tableName, uint32_t batchPerTbl, + char *tableName, uint32_t batch, uint64_t tableSeq, char *dbName, char *buffer, int64_t insertRows, + int64_t startTime, uint64_t *pRemainderBufLen) { assert(buffer); @@ -5222,18 +5276,17 @@ static int64_t generateInterlaceDataWithoutStb( int64_t dataLen = 0; - int64_t startTime = 1500000000000; - int64_t k = generateDataTailWithoutStb( - batchPerTbl, pstr, *pRemainderBufLen, insertRows, 0, + int32_t k = generateDataTailWithoutStb( + batch, pstr, *pRemainderBufLen, insertRows, 0, startTime, &dataLen); - if (k == batchPerTbl) { + if (k == batch) { pstr += dataLen; *pRemainderBufLen -= dataLen; } else { - debugPrint("%s() LN%d, generated data tail: %"PRIu64", not equal batch per table: %ud\n", - __func__, __LINE__, k, batchPerTbl); + debugPrint("%s() LN%d, generated data tail: %d, not equal batch per table: %u\n", + __func__, __LINE__, k, batch); pstr -= headLen; pstr[0] = '\0'; k = 0; @@ -5242,31 +5295,238 @@ static int64_t generateInterlaceDataWithoutStb( return k; } -static int32_t prepareStbStmt(SSuperTable *stbInfo, +#if STMT_IFACE_ENABLED == 1 +static int32_t prepareStmtBindArrayByType(TAOS_BIND *bind, + char *dataType, int32_t dataLen, char **ptr) +{ + if (0 == strncasecmp(dataType, + "BINARY", strlen("BINARY"))) { + if (dataLen > TSDB_MAX_BINARY_LEN) { + errorPrint( "binary length overflow, max size:%u\n", + (uint32_t)TSDB_MAX_BINARY_LEN); + return -1; + } + char *bind_binary = (char *)*ptr; + rand_string(bind_binary, dataLen); + + bind->buffer_type = TSDB_DATA_TYPE_BINARY; + bind->buffer_length = dataLen; + bind->buffer = bind_binary; + bind->length = &bind->buffer_length; + bind->is_null = NULL; + + *ptr += bind->buffer_length; + } else if (0 == strncasecmp(dataType, + "NCHAR", strlen("NCHAR"))) { + if (dataLen > TSDB_MAX_BINARY_LEN) { + errorPrint( "nchar length overflow, max size:%u\n", + (uint32_t)TSDB_MAX_BINARY_LEN); + return -1; + } + char *bind_nchar = (char *)*ptr; + rand_string(bind_nchar, dataLen); + + bind->buffer_type = TSDB_DATA_TYPE_NCHAR; + bind->buffer_length = strlen(bind_nchar); + bind->buffer = bind_nchar; + bind->length = &bind->buffer_length; + bind->is_null = NULL; + + *ptr += bind->buffer_length; + } else if (0 == strncasecmp(dataType, + "INT", strlen("INT"))) { + int32_t *bind_int = (int32_t *)*ptr; + + *bind_int = rand_int(); + bind->buffer_type = TSDB_DATA_TYPE_INT; + bind->buffer_length = sizeof(int32_t); + bind->buffer = bind_int; + bind->length = &bind->buffer_length; + bind->is_null = NULL; + + *ptr += bind->buffer_length; + } else if (0 == strncasecmp(dataType, + "BIGINT", strlen("BIGINT"))) { + int64_t *bind_bigint = (int64_t *)*ptr; + + *bind_bigint = rand_bigint(); + bind->buffer_type = TSDB_DATA_TYPE_BIGINT; + bind->buffer_length = sizeof(int64_t); + bind->buffer = bind_bigint; + bind->length = &bind->buffer_length; + bind->is_null = NULL; + + *ptr += bind->buffer_length; + } else if (0 == strncasecmp(dataType, + "FLOAT", strlen("FLOAT"))) { + float *bind_float = (float *) *ptr; + + *bind_float = rand_float(); + bind->buffer_type = TSDB_DATA_TYPE_FLOAT; + bind->buffer_length = sizeof(float); + bind->buffer = bind_float; + bind->length = &bind->buffer_length; + bind->is_null = NULL; + + *ptr += bind->buffer_length; + } else if (0 == strncasecmp(dataType, + "DOUBLE", strlen("DOUBLE"))) { + double *bind_double = (double *)*ptr; + + *bind_double = rand_double(); + bind->buffer_type = TSDB_DATA_TYPE_DOUBLE; + bind->buffer_length = sizeof(double); + bind->buffer = bind_double; + bind->length = &bind->buffer_length; + bind->is_null = NULL; + + *ptr += bind->buffer_length; + } else if (0 == strncasecmp(dataType, + "SMALLINT", strlen("SMALLINT"))) { + int16_t *bind_smallint = (int16_t *)*ptr; + + *bind_smallint = rand_smallint(); + bind->buffer_type = TSDB_DATA_TYPE_SMALLINT; + bind->buffer_length = sizeof(int16_t); + bind->buffer = bind_smallint; + bind->length = &bind->buffer_length; + bind->is_null = NULL; + + *ptr += bind->buffer_length; + } else if (0 == strncasecmp(dataType, + "TINYINT", strlen("TINYINT"))) { + int8_t *bind_tinyint = (int8_t *)*ptr; + + *bind_tinyint = rand_tinyint(); + bind->buffer_type = TSDB_DATA_TYPE_TINYINT; + bind->buffer_length = sizeof(int8_t); + bind->buffer = bind_tinyint; + bind->length = &bind->buffer_length; + bind->is_null = NULL; + *ptr += bind->buffer_length; + } else if (0 == strncasecmp(dataType, + "BOOL", strlen("BOOL"))) { + int8_t *bind_bool = (int8_t *)*ptr; + + *bind_bool = rand_bool(); + bind->buffer_type = TSDB_DATA_TYPE_BOOL; + bind->buffer_length = sizeof(int8_t); + bind->buffer = bind_bool; + bind->length = &bind->buffer_length; + bind->is_null = NULL; + + *ptr += bind->buffer_length; + } else if (0 == strncasecmp(dataType, + "TIMESTAMP", strlen("TIMESTAMP"))) { + int64_t *bind_ts2 = (int64_t *) *ptr; + + *bind_ts2 = rand_bigint(); + bind->buffer_type = TSDB_DATA_TYPE_TIMESTAMP; + bind->buffer_length = sizeof(int64_t); + bind->buffer = bind_ts2; + bind->length = &bind->buffer_length; + bind->is_null = NULL; + + *ptr += bind->buffer_length; + } else { + errorPrint( "No support data type: %s\n", + dataType); + return -1; + } + + return 0; +} + +static int32_t prepareStmtWithoutStb( TAOS_STMT *stmt, - char *tableName, uint32_t batch, uint64_t insertRows, - int64_t startTime, char *buffer) + char *tableName, + uint32_t batch, + int64_t insertRows, + int64_t recordFrom, + int64_t startTime) { - uint32_t k; - int ret; - char *pstr = buffer; - pstr += sprintf(pstr, "INSERT INTO %s values(?", tableName); + int ret = taos_stmt_set_tbname(stmt, tableName); + if (ret != 0) { + errorPrint("failed to execute taos_stmt_set_tbname(%s). return 0x%x. reason: %s\n", + tableName, ret, taos_errstr(NULL)); + return ret; + } + + char **data_type = g_args.datatype; + + char *bindArray = malloc(sizeof(TAOS_BIND) * (g_args.num_of_CPR + 1)); + if (bindArray == NULL) { + errorPrint("Failed to allocate %d bind params\n", + (g_args.num_of_CPR + 1)); + return -1; + } + + int32_t k = 0; + for (k = 0; k < batch;) { + /* columnCount + 1 (ts) */ + char data[MAX_DATA_SIZE]; + memset(data, 0, MAX_DATA_SIZE); + + char *ptr = data; + TAOS_BIND *bind = (TAOS_BIND *)(bindArray + 0); + + int64_t *bind_ts; + + bind_ts = (int64_t *)ptr; + bind->buffer_type = TSDB_DATA_TYPE_TIMESTAMP; + *bind_ts = startTime + getTSRandTail( + (int64_t)DEFAULT_TIMESTAMP_STEP, k, + g_args.disorderRatio, + g_args.disorderRange); + bind->buffer_length = sizeof(int64_t); + bind->buffer = bind_ts; + bind->length = &bind->buffer_length; + bind->is_null = NULL; + + ptr += bind->buffer_length; + + for (int i = 0; i < g_args.num_of_CPR; i ++) { + bind = (TAOS_BIND *)((char *)bindArray + (sizeof(TAOS_BIND) * (i + 1))); + if ( -1 == prepareStmtBindArrayByType( + bind, + data_type[i], + g_args.len_of_binary, + &ptr)) { + return -1; + } + } + taos_stmt_bind_param(stmt, (TAOS_BIND *)bindArray); + // if msg > 3MB, break + taos_stmt_add_batch(stmt); - for (int i = 0; i < stbInfo->columnCount; i++) { - pstr += sprintf(pstr, ",?"); + k++; + recordFrom ++; + if (recordFrom >= insertRows) { + break; + } } - pstr += sprintf(pstr, ")"); - ret = taos_stmt_prepare(stmt, buffer, 0); - if (ret != 0){ - errorPrint("failed to execute taos_stmt_prepare. return 0x%x. reason: %s\n", - ret, taos_errstr(NULL)); + return k; +} + +static int32_t prepareStbStmt(SSuperTable *stbInfo, + TAOS_STMT *stmt, + char *tableName, uint32_t batch, + uint64_t insertRows, + uint64_t recordFrom, + int64_t startTime, char *buffer) +{ + int ret = taos_stmt_set_tbname(stmt, tableName); + if (ret != 0) { + errorPrint("failed to execute taos_stmt_set_tbname(%s). return 0x%x. reason: %s\n", + tableName, ret, taos_errstr(NULL)); return ret; } - void *bindArray = malloc(sizeof(TAOS_BIND) * (stbInfo->columnCount + 1)); + char *bindArray = malloc(sizeof(TAOS_BIND) * (stbInfo->columnCount + 1)); if (bindArray == NULL) { - errorPrint("Failed to allocate %d bind params\n", batch); + errorPrint("Failed to allocate %d bind params\n", + (stbInfo->columnCount + 1)); return -1; } @@ -5276,35 +5536,59 @@ static int32_t prepareStbStmt(SSuperTable *stbInfo, } else { tsRand = false; } - for (k = 0; k < batch; k++) { + + uint32_t k; + for (k = 0; k < batch;) { /* columnCount + 1 (ts) */ - for (int i = 0; i <= stbInfo->columnCount; i ++) { - TAOS_BIND *bind = (TAOS_BIND *)bindArray + (sizeof(TAOS_BIND) * i); - if (i == 0) { - bind->buffer_type = TSDB_DATA_TYPE_TIMESTAMP; - int64_t ts; - if (tsRand) { - ts = startTime + getTSRandTail( - stbInfo->timeStampStep, k, - stbInfo->disorderRatio, - stbInfo->disorderRange); - } else { - ts = startTime + stbInfo->timeStampStep * k; - } - bind->buffer = &ts; - - } else { + char data[MAX_DATA_SIZE]; + memset(data, 0, MAX_DATA_SIZE); + + char *ptr = data; + TAOS_BIND *bind = (TAOS_BIND *)(bindArray + 0); + + int64_t *bind_ts; + bind_ts = (int64_t *)ptr; + bind->buffer_type = TSDB_DATA_TYPE_TIMESTAMP; + if (tsRand) { + *bind_ts = startTime + getTSRandTail( + stbInfo->timeStampStep, k, + stbInfo->disorderRatio, + stbInfo->disorderRange); + } else { + *bind_ts = startTime + stbInfo->timeStampStep * k; + } + bind->buffer_length = sizeof(int64_t); + bind->buffer = bind_ts; + bind->length = &bind->buffer_length; + bind->is_null = NULL; + + ptr += bind->buffer_length; + + for (int i = 0; i < stbInfo->columnCount; i ++) { + bind = (TAOS_BIND *)((char *)bindArray + (sizeof(TAOS_BIND) * (i + 1))); + if ( -1 == prepareStmtBindArrayByType( + bind, + stbInfo->columns[i].dataType, + stbInfo->columns[i].dataLen, + &ptr)) { + return -1; } } + taos_stmt_bind_param(stmt, (TAOS_BIND *)bindArray); // if msg > 3MB, break - } + taos_stmt_add_batch(stmt); - taos_stmt_bind_param(stmt, bindArray); - taos_stmt_add_batch(stmt); + k++; + recordFrom ++; + if (recordFrom >= insertRows) { + break; + } + } return k; } +#endif static int32_t generateStbProgressiveData( SSuperTable *superTblInfo, @@ -5312,7 +5596,7 @@ static int32_t generateStbProgressiveData( int64_t tableSeq, char *dbName, char *buffer, int64_t insertRows, - uint64_t startFrom, int64_t startTime, int64_t *pSamplePos, + uint64_t recordFrom, int64_t startTime, int64_t *pSamplePos, int64_t *pRemainderBufLen) { assert(buffer != NULL); @@ -5335,22 +5619,17 @@ static int32_t generateStbProgressiveData( return generateStbDataTail(superTblInfo, g_args.num_of_RPR, pstr, *pRemainderBufLen, - insertRows, startFrom, + insertRows, recordFrom, startTime, pSamplePos, &dataLen); } -static int64_t prepareStmtWithoutStb(char *tableName) -{ - return -1; -} - -static int64_t generateProgressiveDataWithoutStb( +static int32_t generateProgressiveDataWithoutStb( char *tableName, /* int64_t tableSeq, */ threadInfo *pThreadInfo, char *buffer, int64_t insertRows, - uint64_t startFrom, int64_t startTime, /*int64_t *pSamplePos, */ + uint64_t recordFrom, int64_t startTime, /*int64_t *pSamplePos, */ int64_t *pRemainderBufLen) { assert(buffer != NULL); @@ -5371,14 +5650,14 @@ static int64_t generateProgressiveDataWithoutStb( int64_t dataLen; return generateDataTailWithoutStb( - g_args.num_of_RPR, pstr, *pRemainderBufLen, insertRows, startFrom, + g_args.num_of_RPR, pstr, *pRemainderBufLen, insertRows, recordFrom, startTime, /*pSamplePos, */&dataLen); } static void printStatPerThread(threadInfo *pThreadInfo) { - fprintf(stderr, "====thread[%d] completed total inserted rows: %"PRIu64 ", total affected rows: %"PRIu64". %.2f records/second====\n", + fprintf(stderr, "====thread[%d] completed total inserted rows: %"PRIu64 ", total affected rows: %"PRIu64". %.2f records/second====\n", pThreadInfo->threadID, pThreadInfo->totalInsertRows, pThreadInfo->totalAffectedRows, @@ -5496,22 +5775,53 @@ static void* syncWriteInterlace(threadInfo *pThreadInfo) { int32_t generated; if (superTblInfo) { - generated = generateStbInterlaceData( - superTblInfo, - tableName, batchPerTbl, i, - batchPerTblTimes, - tableSeq, - pThreadInfo, pstr, - insertRows, - startTime, - &remainderBufLen); + if (superTblInfo->iface == STMT_IFACE) { +#if STMT_IFACE_ENABLED == 1 + generated = prepareStbStmt(superTblInfo, + pThreadInfo->stmt, + tableName, + batchPerTbl, + insertRows, i, + startTime, + pThreadInfo->buffer); +#else + generated = -1; +#endif + } else { + generated = generateStbInterlaceData( + superTblInfo, + tableName, batchPerTbl, i, + batchPerTblTimes, + tableSeq, + pThreadInfo, pstr, + insertRows, + startTime, + &remainderBufLen); + } } else { - generated = generateInterlaceDataWithoutStb( - tableName, batchPerTbl, - tableSeq, - pThreadInfo->db_name, pstr, - insertRows, - &remainderBufLen); + if (g_args.iface == STMT_IFACE) { + debugPrint("[%d] %s() LN%d, tableName:%s, batch:%d startTime:%"PRId64"\n", + pThreadInfo->threadID, + __func__, __LINE__, + tableName, batchPerTbl, startTime); +#if STMT_IFACE_ENABLED == 1 + generated = prepareStmtWithoutStb( + pThreadInfo->stmt, tableName, + batchPerTbl, + insertRows, i, + startTime); +#else + generated = -1; +#endif + } else { + generated = generateInterlaceDataWithoutStb( + tableName, batchPerTbl, + tableSeq, + pThreadInfo->db_name, pstr, + insertRows, + startTime, + &remainderBufLen); + } } debugPrint("[%d] %s() LN%d, generated records is %d\n", @@ -5526,9 +5836,10 @@ static void* syncWriteInterlace(threadInfo *pThreadInfo) { tableSeq ++; recOfBatch += batchPerTbl; + pstr += (oldRemainderLen - remainderBufLen); -// startTime += batchPerTbl * superTblInfo->timeStampStep; pThreadInfo->totalInsertRows += batchPerTbl; + verbosePrint("[%d] %s() LN%d batchPerTbl=%d recOfBatch=%d\n", pThreadInfo->threadID, __func__, __LINE__, batchPerTbl, recOfBatch); @@ -5682,10 +5993,16 @@ static void* syncWriteProgressive(threadInfo *pThreadInfo) { int32_t generated; if (superTblInfo) { if (superTblInfo->iface == STMT_IFACE) { - generated = prepareStbStmt(superTblInfo, +#if STMT_IFACE_ENABLED == 1 + generated = prepareStbStmt( + superTblInfo, pThreadInfo->stmt, - tableName, g_args.num_of_RPR, - insertRows, start_time, pstr); + tableName, + g_args.num_of_RPR, + insertRows, i, start_time, pstr); +#else + generated = -1; +#endif } else { generated = generateStbProgressiveData( superTblInfo, @@ -5696,7 +6013,16 @@ static void* syncWriteProgressive(threadInfo *pThreadInfo) { } } else { if (g_args.iface == STMT_IFACE) { - generated = prepareStmtWithoutStb(tableName); +#if STMT_IFACE_ENABLED == 1 + generated = prepareStmtWithoutStb( + pThreadInfo->stmt, + tableName, + g_args.num_of_RPR, + insertRows, i, + start_time); +#else + generated = -1; +#endif } else { generated = generateProgressiveDataWithoutStb( tableName, @@ -5717,13 +6043,13 @@ static void* syncWriteProgressive(threadInfo *pThreadInfo) { startTs = taosGetTimestampMs(); - int64_t affectedRows = execInsert(pThreadInfo, generated); + int32_t affectedRows = execInsert(pThreadInfo, generated); endTs = taosGetTimestampMs(); uint64_t delay = endTs - startTs; performancePrint("%s() LN%d, insert execution time is %"PRId64"ms\n", __func__, __LINE__, delay); - verbosePrint("[%d] %s() LN%d affectedRows=%"PRId64"\n", + verbosePrint("[%d] %s() LN%d affectedRows=%d\n", pThreadInfo->threadID, __func__, __LINE__, affectedRows); @@ -5733,7 +6059,7 @@ static void* syncWriteProgressive(threadInfo *pThreadInfo) { pThreadInfo->totalDelay += delay; if (affectedRows < 0) { - errorPrint("%s() LN%d, affected rows: %"PRId64"\n", + errorPrint("%s() LN%d, affected rows: %d\n", __func__, __LINE__, affectedRows); goto free_of_progressive; } @@ -5903,15 +6229,6 @@ static int convertHostToServAddr(char *host, uint16_t port, struct sockaddr_in * static void startMultiThreadInsertData(int threads, char* db_name, char* precision,SSuperTable* superTblInfo) { - //TAOS* taos; - //if (0 == strncasecmp(superTblInfo->iface, "taosc", 5)) { - // taos = taos_connect(g_Dbs.host, g_Dbs.user, g_Dbs.password, db_name, g_Dbs.port); - // if (NULL == taos) { - // printf("connect to server fail, reason: %s\n", taos_errstr(NULL)); - // exit(-1); - // } - //} - int32_t timePrec = TSDB_TIME_PRECISION_MILLI; if (0 != precision[0]) { if (0 == strncasecmp(precision, "ms", 2)) { @@ -5973,7 +6290,7 @@ static void startMultiThreadInsertData(int threads, char* db_name, } int64_t ntables = 0; - uint64_t startFrom; + uint64_t tableFrom; if (superTblInfo) { int64_t limit; @@ -6000,7 +6317,7 @@ static void startMultiThreadInsertData(int threads, char* db_name, } ntables = limit; - startFrom = offset; + tableFrom = offset; if ((superTblInfo->childTblExists != TBL_NO_EXISTS) && ((superTblInfo->childTblOffset + superTblInfo->childTblLimit ) @@ -6032,7 +6349,7 @@ static void startMultiThreadInsertData(int threads, char* db_name, offset); } else { ntables = g_args.num_of_tables; - startFrom = 0; + tableFrom = 0; } taos_close(taos0); @@ -6090,7 +6407,16 @@ static void startMultiThreadInsertData(int threads, char* db_name, exit(-1); } - if ((superTblInfo) && (superTblInfo->iface == STMT_IFACE)) { + if ((g_args.iface == STMT_IFACE) + || ((superTblInfo) && (superTblInfo->iface == STMT_IFACE))) { + + int columnCount; + if (superTblInfo) { + columnCount = superTblInfo->columnCount; + } else { + columnCount = g_args.num_of_CPR; + } + pThreadInfo->stmt = taos_stmt_init(pThreadInfo->taos); if (NULL == pThreadInfo->stmt) { errorPrint( @@ -6101,6 +6427,24 @@ static void startMultiThreadInsertData(int threads, char* db_name, free(infos); exit(-1); } + + char buffer[3000]; + char *pstr = buffer; + pstr += sprintf(pstr, "INSERT INTO ? values(?"); + + for (int col = 0; col < columnCount; col ++) { + pstr += sprintf(pstr, ",?"); + } + pstr += sprintf(pstr, ")"); + + int ret = taos_stmt_prepare(pThreadInfo->stmt, buffer, 0); + if (ret != 0){ + errorPrint("failed to execute taos_stmt_prepare. return 0x%x. reason: %s\n", + ret, taos_errstr(NULL)); + free(pids); + free(infos); + exit(-1); + } } } else { pThreadInfo->taos = NULL; @@ -6109,10 +6453,10 @@ static void startMultiThreadInsertData(int threads, char* db_name, /* if ((NULL == superTblInfo) || (0 == superTblInfo->multiThreadWriteOneTbl)) { */ - pThreadInfo->start_table_from = startFrom; + pThreadInfo->start_table_from = tableFrom; pThreadInfo->ntables = iend_table_to = i < b ? startFrom + a : startFrom + a - 1; - startFrom = pThreadInfo->end_table_to + 1; + pThreadInfo->end_table_to = i < b ? tableFrom + a : tableFrom + a - 1; + tableFrom = pThreadInfo->end_table_to + 1; /* } else { pThreadInfo->start_table_from = 0; pThreadInfo->ntables = superTblInfo->childTblCount; @@ -6421,7 +6765,6 @@ static int insertTestProcess() { } } - taosMsleep(1000); // create sub threads for inserting data //start = taosGetTimestampMs(); for (int i = 0; i < g_Dbs.dbCount; i++) { @@ -6500,7 +6843,7 @@ static void *specifiedTableQuery(void *sarg) { uint64_t lastPrintTime = taosGetTimestampMs(); uint64_t startTs = taosGetTimestampMs(); - if (g_queryInfo.specifiedQueryInfo.result[pThreadInfo->querySeq] != NULL) { + if (g_queryInfo.specifiedQueryInfo.result[pThreadInfo->querySeq][0] != '\0') { sprintf(pThreadInfo->filePath, "%s-%d", g_queryInfo.specifiedQueryInfo.result[pThreadInfo->querySeq], pThreadInfo->threadID); @@ -6601,7 +6944,7 @@ static void *superTableQuery(void *sarg) { for (int j = 0; j < g_queryInfo.superQueryInfo.sqlCount; j++) { memset(sqlstr,0,sizeof(sqlstr)); replaceChildTblName(g_queryInfo.superQueryInfo.sql[j], sqlstr, i); - if (g_queryInfo.superQueryInfo.result[j] != NULL) { + if (g_queryInfo.superQueryInfo.result[j][0] != '\0') { sprintf(pThreadInfo->filePath, "%s-%d", g_queryInfo.superQueryInfo.result[j], pThreadInfo->threadID); @@ -6700,7 +7043,6 @@ static int queryTestProcess() { char sqlStr[MAX_TB_NAME_SIZE*2]; sprintf(sqlStr, "use %s", g_queryInfo.dbName); - verbosePrint("%s() %d sqlStr: %s\n", __func__, __LINE__, sqlStr); if (0 != queryDbExec(taos, sqlStr, NO_INSERT_TYPE, false)) { taos_close(taos); free(infos); @@ -6752,15 +7094,15 @@ static int queryTestProcess() { b = ntables % threads; } - uint64_t startFrom = 0; + uint64_t tableFrom = 0; for (int i = 0; i < threads; i++) { threadInfo *pThreadInfo = infosOfSub + i; pThreadInfo->threadID = i; - pThreadInfo->start_table_from = startFrom; + pThreadInfo->start_table_from = tableFrom; pThreadInfo->ntables = iend_table_to = i < b ? startFrom + a : startFrom + a - 1; - startFrom = pThreadInfo->end_table_to + 1; + pThreadInfo->end_table_to = i < b ? tableFrom + a : tableFrom + a - 1; + tableFrom = pThreadInfo->end_table_to + 1; pThreadInfo->taos = NULL; // TODO: workaround to use separate taos connection; pthread_create(pidsOfSub + i, NULL, superTableQuery, pThreadInfo); } @@ -6941,7 +7283,10 @@ static void *superSubscribe(void *sarg) { uint64_t st = 0, et = 0; - while(1) { + while ((g_queryInfo.superQueryInfo.endAfterConsume == -1) + || (g_queryInfo.superQueryInfo.endAfterConsume < + consumed[pThreadInfo->end_table_to - pThreadInfo->start_table_from])) { + for (uint64_t i = pThreadInfo->start_table_from; i <= pThreadInfo->end_table_to; i++) { tsubSeq = i - pThreadInfo->start_table_from; @@ -6970,7 +7315,7 @@ static void *superSubscribe(void *sarg) { } consumed[tsubSeq] ++; - if ((g_queryInfo.superQueryInfo.subscribeKeepProgress) + if ((g_queryInfo.superQueryInfo.resubAfterConsume != -1) && (consumed[tsubSeq] >= g_queryInfo.superQueryInfo.resubAfterConsume)) { printf("keepProgress:%d, resub super table query: %"PRIu64"\n", @@ -7024,7 +7369,6 @@ static void *specifiedSubscribe(void *sarg) { char sqlStr[MAX_TB_NAME_SIZE*2]; sprintf(sqlStr, "use %s", g_queryInfo.dbName); - debugPrint("%s() %d sqlStr: %s\n", __func__, __LINE__, sqlStr); if (0 != queryDbExec(pThreadInfo->taos, sqlStr, NO_INSERT_TYPE, false)) { taos_close(pThreadInfo->taos); return NULL; @@ -7034,7 +7378,7 @@ static void *specifiedSubscribe(void *sarg) { "taosdemo-subscribe-%"PRIu64"-%d", pThreadInfo->querySeq, pThreadInfo->threadID); - if (g_queryInfo.specifiedQueryInfo.result[pThreadInfo->querySeq] != NULL) { + if (g_queryInfo.specifiedQueryInfo.result[pThreadInfo->querySeq][0] != '\0') { sprintf(pThreadInfo->filePath, "%s-%d", g_queryInfo.specifiedQueryInfo.result[pThreadInfo->querySeq], pThreadInfo->threadID); @@ -7053,7 +7397,10 @@ static void *specifiedSubscribe(void *sarg) { // start loop to consume result g_queryInfo.specifiedQueryInfo.consumed[pThreadInfo->threadID] = 0; - while(1) { + while((g_queryInfo.specifiedQueryInfo.endAfterConsume[pThreadInfo->querySeq] == -1) + || (g_queryInfo.specifiedQueryInfo.consumed[pThreadInfo->threadID] < + g_queryInfo.specifiedQueryInfo.endAfterConsume[pThreadInfo->querySeq])) { + if (ASYNC_MODE == g_queryInfo.specifiedQueryInfo.asyncMode) { continue; } @@ -7061,15 +7408,18 @@ static void *specifiedSubscribe(void *sarg) { g_queryInfo.specifiedQueryInfo.res[pThreadInfo->threadID] = taos_consume( g_queryInfo.specifiedQueryInfo.tsub[pThreadInfo->threadID]); if (g_queryInfo.specifiedQueryInfo.res[pThreadInfo->threadID]) { - if (g_queryInfo.specifiedQueryInfo.result[pThreadInfo->querySeq][0] != 0) { + if (g_queryInfo.specifiedQueryInfo.result[pThreadInfo->querySeq][0] + != 0) { sprintf(pThreadInfo->filePath, "%s-%d", g_queryInfo.specifiedQueryInfo.result[pThreadInfo->querySeq], pThreadInfo->threadID); - fetchResult(g_queryInfo.specifiedQueryInfo.res[pThreadInfo->threadID], pThreadInfo); + fetchResult( + g_queryInfo.specifiedQueryInfo.res[pThreadInfo->threadID], + pThreadInfo); } g_queryInfo.specifiedQueryInfo.consumed[pThreadInfo->threadID] ++; - if ((g_queryInfo.specifiedQueryInfo.subscribeKeepProgress) + if ((g_queryInfo.specifiedQueryInfo.resubAfterConsume[pThreadInfo->querySeq] != -1) && (g_queryInfo.specifiedQueryInfo.consumed[pThreadInfo->threadID] >= g_queryInfo.specifiedQueryInfo.resubAfterConsume[pThreadInfo->querySeq])) { printf("keepProgress:%d, resub specified query: %"PRIu64"\n", @@ -7078,16 +7428,17 @@ static void *specifiedSubscribe(void *sarg) { g_queryInfo.specifiedQueryInfo.consumed[pThreadInfo->threadID] = 0; taos_unsubscribe(g_queryInfo.specifiedQueryInfo.tsub[pThreadInfo->threadID], g_queryInfo.specifiedQueryInfo.subscribeKeepProgress); - g_queryInfo.specifiedQueryInfo.tsub[pThreadInfo->threadID] = subscribeImpl( - SPECIFIED_CLASS, - pThreadInfo, - g_queryInfo.specifiedQueryInfo.sql[pThreadInfo->querySeq], - g_queryInfo.specifiedQueryInfo.topic[pThreadInfo->threadID], - g_queryInfo.specifiedQueryInfo.subscribeRestart, - g_queryInfo.specifiedQueryInfo.subscribeInterval); + g_queryInfo.specifiedQueryInfo.tsub[pThreadInfo->threadID] = + subscribeImpl( + SPECIFIED_CLASS, + pThreadInfo, + g_queryInfo.specifiedQueryInfo.sql[pThreadInfo->querySeq], + g_queryInfo.specifiedQueryInfo.topic[pThreadInfo->threadID], + g_queryInfo.specifiedQueryInfo.subscribeRestart, + g_queryInfo.specifiedQueryInfo.subscribeInterval); if (NULL == g_queryInfo.specifiedQueryInfo.tsub[pThreadInfo->threadID]) { - taos_close(pThreadInfo->taos); - return NULL; + taos_close(pThreadInfo->taos); + return NULL; } } } @@ -7136,12 +7487,12 @@ static int subscribeTestProcess() { //==== create threads for query for specified table if (g_queryInfo.specifiedQueryInfo.sqlCount <= 0) { - debugPrint("%s() LN%d, sepcified query sqlCount %"PRIu64".\n", + debugPrint("%s() LN%d, sepcified query sqlCount %d.\n", __func__, __LINE__, g_queryInfo.specifiedQueryInfo.sqlCount); } else { if (g_queryInfo.specifiedQueryInfo.concurrent <= 0) { - errorPrint("%s() LN%d, sepcified query sqlCount %"PRIu64".\n", + errorPrint("%s() LN%d, sepcified query sqlCount %d.\n", __func__, __LINE__, g_queryInfo.specifiedQueryInfo.sqlCount); exit(-1); @@ -7174,7 +7525,7 @@ static int subscribeTestProcess() { //==== create threads for super table query if (g_queryInfo.superQueryInfo.sqlCount <= 0) { - debugPrint("%s() LN%d, super table query sqlCount %"PRIu64".\n", + debugPrint("%s() LN%d, super table query sqlCount %d.\n", __func__, __LINE__, g_queryInfo.superQueryInfo.sqlCount); } else { @@ -7210,17 +7561,17 @@ static int subscribeTestProcess() { } for (uint64_t i = 0; i < g_queryInfo.superQueryInfo.sqlCount; i++) { - uint64_t startFrom = 0; + uint64_t tableFrom = 0; for (int j = 0; j < threads; j++) { uint64_t seq = i * threads + j; threadInfo *pThreadInfo = infosOfStable + seq; pThreadInfo->threadID = seq; pThreadInfo->querySeq = i; - pThreadInfo->start_table_from = startFrom; + pThreadInfo->start_table_from = tableFrom; pThreadInfo->ntables = jend_table_to = jend_table_to + 1; + pThreadInfo->end_table_to = jend_table_to + 1; pThreadInfo->taos = NULL; // TODO: workaround to use separate taos connection; pthread_create(pidsOfStable + seq, NULL, superSubscribe, pThreadInfo); @@ -7300,7 +7651,7 @@ static void setParaFromArg(){ g_Dbs.threadCountByCreateTbl = g_args.num_of_threads; g_Dbs.dbCount = 1; - g_Dbs.db[0].drop = 1; + g_Dbs.db[0].drop = true; tstrncpy(g_Dbs.db[0].dbName, g_args.database, MAX_DB_NAME_SIZE); g_Dbs.db[0].dbCfg.replica = g_args.replica; @@ -7452,7 +7803,6 @@ static void querySqlFile(TAOS* taos, char* sqlFile) } memcpy(cmd + cmd_len, line, read_len); - verbosePrint("%s() LN%d cmd: %s\n", __func__, __LINE__, cmd); if (0 != queryDbExec(taos, cmd, NO_INSERT_TYPE, false)) { errorPrint("%s() LN%d, queryDbExec %s failed!\n", __func__, __LINE__, cmd); diff --git a/src/mnode/src/mnodeSdb.c b/src/mnode/src/mnodeSdb.c index 9e3b87671e35e53ad5faa6ec7313ba2af3f6c247..84e4f33ca7efe2cafca9ec64d8b7872cac827403 100644 --- a/src/mnode/src/mnodeSdb.c +++ b/src/mnode/src/mnodeSdb.c @@ -719,13 +719,13 @@ static int32_t sdbProcessWrite(void *wparam, void *hparam, int32_t qtype, void * if (action == SDB_ACTION_INSERT) { return sdbPerformInsertAction(pHead, pTable); } else if (action == SDB_ACTION_DELETE) { - if (qtype == TAOS_QTYPE_FWD) { + //if (qtype == TAOS_QTYPE_FWD) { // Drop database/stable may take a long time and cause a timeout, so we confirm first then reput it into queue - sdbWriteFwdToQueue(1, hparam, TAOS_QTYPE_QUERY, unused); - return TSDB_CODE_SUCCESS; - } else { + // sdbWriteFwdToQueue(1, hparam, TAOS_QTYPE_QUERY, unused); + // return TSDB_CODE_SUCCESS; + //} else { return sdbPerformDeleteAction(pHead, pTable); - } + //} } else if (action == SDB_ACTION_UPDATE) { return sdbPerformUpdateAction(pHead, pTable); } else { diff --git a/src/mnode/src/mnodeTable.c b/src/mnode/src/mnodeTable.c index 4d3125a2d1ebe872e992846792e2b8042fb223a9..1584df16fe1b0af84ae1bcc5123a9d71be447ca0 100644 --- a/src/mnode/src/mnodeTable.c +++ b/src/mnode/src/mnodeTable.c @@ -966,6 +966,11 @@ static int32_t mnodeProcessDropTableMsg(SMnodeMsg *pMsg) { pMsg->rpcMsg.ahandle, pDrop->name, pSTable->uid, pSTable->numOfTables, taosHashGetSize(pSTable->vgHash)); return mnodeProcessDropSuperTableMsg(pMsg); } else { + // user specify the "DROP STABLE" sql statement, but it is actually a normal table, return error msg. + if (pDrop->supertable) { + return TSDB_CODE_MND_INVALID_TABLE_TYPE; + } + SCTableObj *pCTable = (SCTableObj *)pMsg->pTable; mInfo("msg:%p, app:%p table:%s, start to drop ctable, vgId:%d tid:%d uid:%" PRIu64, pMsg, pMsg->rpcMsg.ahandle, pDrop->name, pCTable->vgId, pCTable->tid, pCTable->uid); @@ -1189,8 +1194,8 @@ static int32_t mnodeFindSuperTableTagIndex(SSTableObj *pStable, const char *tagN static int32_t mnodeAddSuperTableTagCb(SMnodeMsg *pMsg, int32_t code) { SSTableObj *pStable = (SSTableObj *)pMsg->pTable; - mLInfo("msg:%p, app:%p stable %s, add tag result:%s", pMsg, pMsg->rpcMsg.ahandle, pStable->info.tableId, - tstrerror(code)); + mLInfo("msg:%p, app:%p stable %s, add tag result:%s, numOfTags:%d", pMsg, pMsg->rpcMsg.ahandle, pStable->info.tableId, + tstrerror(code), pStable->numOfTags); return code; } @@ -1674,12 +1679,9 @@ static int32_t mnodeSetSchemaFromSuperTable(SSchema *pSchema, SSTableObj *pTable return (pTable->numOfColumns + pTable->numOfTags) * sizeof(SSchema); } -static int32_t mnodeGetSuperTableMeta(SMnodeMsg *pMsg) { +static int32_t mnodeDoGetSuperTableMeta(SMnodeMsg *pMsg, STableMetaMsg* pMeta) { SSTableObj *pTable = (SSTableObj *)pMsg->pTable; - STableMetaMsg *pMeta = rpcMallocCont(sizeof(STableMetaMsg) + sizeof(SSchema) * (TSDB_MAX_TAGS + TSDB_MAX_COLUMNS + 16)); - if (pMeta == NULL) { - return TSDB_CODE_MND_OUT_OF_MEMORY; - } + pMeta->uid = htobe64(pTable->uid); pMeta->sversion = htons(pTable->sversion); pMeta->tversion = htons(pTable->tversion); @@ -1690,6 +1692,18 @@ static int32_t mnodeGetSuperTableMeta(SMnodeMsg *pMsg) { pMeta->contLen = sizeof(STableMetaMsg) + mnodeSetSchemaFromSuperTable(pMeta->schema, pTable); tstrncpy(pMeta->tableFname, pTable->info.tableId, sizeof(pMeta->tableFname)); + return TSDB_CODE_SUCCESS; +} + +static int32_t mnodeGetSuperTableMeta(SMnodeMsg *pMsg) { + SSTableObj *pTable = (SSTableObj *)pMsg->pTable; + STableMetaMsg *pMeta = rpcMallocCont(sizeof(STableMetaMsg) + sizeof(SSchema) * (TSDB_MAX_TAGS + TSDB_MAX_COLUMNS + 16)); + if (pMeta == NULL) { + return TSDB_CODE_MND_OUT_OF_MEMORY; + } + + mnodeDoGetSuperTableMeta(pMsg, pMeta); + pMsg->rpcRsp.len = pMeta->contLen; pMeta->contLen = htons(pMeta->contLen); @@ -1700,11 +1714,7 @@ static int32_t mnodeGetSuperTableMeta(SMnodeMsg *pMsg) { return TSDB_CODE_SUCCESS; } -static int32_t mnodeProcessSuperTableVgroupMsg(SMnodeMsg *pMsg) { - SSTableVgroupMsg *pInfo = pMsg->rpcMsg.pCont; - int32_t numOfTable = htonl(pInfo->numOfTables); - - // reserve space +static int32_t calculateVgroupMsgLength(SSTableVgroupMsg* pInfo, int32_t numOfTable) { int32_t contLen = sizeof(SSTableVgroupRspMsg) + 32 * sizeof(SVgroupMsg) + sizeof(SVgroupsMsg); for (int32_t i = 0; i < numOfTable; ++i) { char *stableName = (char *)pInfo + sizeof(SSTableVgroupMsg) + (TSDB_TABLE_FNAME_LEN)*i; @@ -1716,6 +1726,75 @@ static int32_t mnodeProcessSuperTableVgroupMsg(SMnodeMsg *pMsg) { mnodeDecTableRef(pTable); } + return contLen; +} + +static char* serializeVgroupInfo(SSTableObj *pTable, char* name, char* msg, SMnodeMsg* pMsgBody, void* handle) { + SName sn = {0}; + tNameFromString(&sn, name, T_NAME_ACCT | T_NAME_DB | T_NAME_TABLE); + const char* tableName = tNameGetTableName(&sn); + + strncpy(msg, tableName, TSDB_TABLE_NAME_LEN); + msg += TSDB_TABLE_NAME_LEN; + + if (pTable->vgHash == NULL) { + mDebug("msg:%p, app:%p stable:%s, no vgroup exist while get stable vgroup info", pMsgBody, handle, name); + mnodeDecTableRef(pTable); + + // even this super table has no corresponding table, still return + SVgroupsMsg *pVgroupMsg = (SVgroupsMsg *)msg; + pVgroupMsg->numOfVgroups = 0; + + msg += sizeof(SVgroupsMsg); + } else { + SVgroupsMsg *pVgroupMsg = (SVgroupsMsg *)msg; + mDebug("msg:%p, app:%p stable:%s, hash:%p sizeOfVgList:%d will be returned", pMsgBody, handle, + pTable->info.tableId, pTable->vgHash, taosHashGetSize(pTable->vgHash)); + + int32_t *pVgId = taosHashIterate(pTable->vgHash, NULL); + int32_t vgSize = 0; + while (pVgId) { + SVgObj *pVgroup = mnodeGetVgroup(*pVgId); + pVgId = taosHashIterate(pTable->vgHash, pVgId); + if (pVgroup == NULL) { + continue; + } + + pVgroupMsg->vgroups[vgSize].vgId = htonl(pVgroup->vgId); + pVgroupMsg->vgroups[vgSize].numOfEps = 0; + + for (int32_t vn = 0; vn < pVgroup->numOfVnodes; ++vn) { + SDnodeObj *pDnode = pVgroup->vnodeGid[vn].pDnode; + if (pDnode == NULL) break; + + tstrncpy(pVgroupMsg->vgroups[vgSize].epAddr[vn].fqdn, pDnode->dnodeFqdn, TSDB_FQDN_LEN); + pVgroupMsg->vgroups[vgSize].epAddr[vn].port = htons(pDnode->dnodePort); + + pVgroupMsg->vgroups[vgSize].numOfEps++; + } + + vgSize++; + mnodeDecVgroupRef(pVgroup); + } + + taosHashCancelIterate(pTable->vgHash, pVgId); + mnodeDecTableRef(pTable); + + pVgroupMsg->numOfVgroups = htonl(vgSize); + + // one table is done, try the next table + msg += sizeof(SVgroupsMsg) + vgSize * sizeof(SVgroupMsg); + } + + return msg; +} + +static int32_t mnodeProcessSuperTableVgroupMsg(SMnodeMsg *pMsg) { + SSTableVgroupMsg *pInfo = pMsg->rpcMsg.pCont; + int32_t numOfTable = htonl(pInfo->numOfTables); + + // calculate the required space. + int32_t contLen = calculateVgroupMsgLength(pInfo, numOfTable); SSTableVgroupRspMsg *pRsp = rpcMallocCont(contLen); if (pRsp == NULL) { return TSDB_CODE_MND_OUT_OF_MEMORY; @@ -1726,62 +1805,16 @@ static int32_t mnodeProcessSuperTableVgroupMsg(SMnodeMsg *pMsg) { for (int32_t i = 0; i < numOfTable; ++i) { char *stableName = (char *)pInfo + sizeof(SSTableVgroupMsg) + (TSDB_TABLE_FNAME_LEN)*i; + SSTableObj *pTable = mnodeGetSuperTable(stableName); if (pTable == NULL) { mError("msg:%p, app:%p stable:%s, not exist while get stable vgroup info", pMsg, pMsg->rpcMsg.ahandle, stableName); mnodeDecTableRef(pTable); continue; } - if (pTable->vgHash == NULL) { - mDebug("msg:%p, app:%p stable:%s, no vgroup exist while get stable vgroup info", pMsg, pMsg->rpcMsg.ahandle, - stableName); - mnodeDecTableRef(pTable); - - // even this super table has no corresponding table, still return - pRsp->numOfTables++; - - SVgroupsMsg *pVgroupMsg = (SVgroupsMsg *)msg; - pVgroupMsg->numOfVgroups = 0; - - msg += sizeof(SVgroupsMsg); - } else { - SVgroupsMsg *pVgroupMsg = (SVgroupsMsg *)msg; - mDebug("msg:%p, app:%p stable:%s, hash:%p sizeOfVgList:%d will be returned", pMsg, pMsg->rpcMsg.ahandle, - pTable->info.tableId, pTable->vgHash, taosHashGetSize(pTable->vgHash)); - - int32_t *pVgId = taosHashIterate(pTable->vgHash, NULL); - int32_t vgSize = 0; - while (pVgId) { - SVgObj *pVgroup = mnodeGetVgroup(*pVgId); - pVgId = taosHashIterate(pTable->vgHash, pVgId); - if (pVgroup == NULL) continue; - - pVgroupMsg->vgroups[vgSize].vgId = htonl(pVgroup->vgId); - pVgroupMsg->vgroups[vgSize].numOfEps = 0; - for (int32_t vn = 0; vn < pVgroup->numOfVnodes; ++vn) { - SDnodeObj *pDnode = pVgroup->vnodeGid[vn].pDnode; - if (pDnode == NULL) break; - - tstrncpy(pVgroupMsg->vgroups[vgSize].epAddr[vn].fqdn, pDnode->dnodeFqdn, TSDB_FQDN_LEN); - pVgroupMsg->vgroups[vgSize].epAddr[vn].port = htons(pDnode->dnodePort); - - pVgroupMsg->vgroups[vgSize].numOfEps++; - } - - vgSize++; - mnodeDecVgroupRef(pVgroup); - } - - taosHashCancelIterate(pTable->vgHash, pVgId); - mnodeDecTableRef(pTable); - - pVgroupMsg->numOfVgroups = htonl(vgSize); - - // one table is done, try the next table - msg += sizeof(SVgroupsMsg) + vgSize * sizeof(SVgroupMsg); - pRsp->numOfTables++; - } + msg = serializeVgroupInfo(pTable, stableName, msg, pMsg, pMsg->rpcMsg.ahandle); + pRsp->numOfTables++; } if (pRsp->numOfTables != numOfTable) { @@ -2415,9 +2448,9 @@ static int32_t mnodeDoGetChildTableMeta(SMnodeMsg *pMsg, STableMetaMsg *pMeta) { pMeta->vgroup.numOfEps++; mnodeDecDnodeRef(pDnode); } - pMeta->vgroup.vgId = htonl(pMsg->pVgroup->vgId); - mDebug("msg:%p, app:%p table:%s, uid:%" PRIu64 " table meta is retrieved, vgId:%d sid:%d", pMsg, pMsg->rpcMsg.ahandle, + pMeta->vgroup.vgId = htonl(pMsg->pVgroup->vgId); + mDebug("msg:%p, app:%p table:%s, uid:%" PRIu64 " table meta is retrieved, vgId:%d tid:%d", pMsg, pMsg->rpcMsg.ahandle, pTable->info.tableId, pTable->uid, pTable->vgId, pTable->tid); return TSDB_CODE_SUCCESS; @@ -2811,56 +2844,128 @@ static void mnodeProcessAlterTableRsp(SRpcMsg *rpcMsg) { static int32_t mnodeProcessMultiTableMetaMsg(SMnodeMsg *pMsg) { SMultiTableInfoMsg *pInfo = pMsg->rpcMsg.pCont; - pInfo->numOfTables = htonl(pInfo->numOfTables); - int32_t totalMallocLen = 4 * 1024 * 1024; // first malloc 4 MB, subsequent reallocation as twice - SMultiTableMeta *pMultiMeta = rpcMallocCont(totalMallocLen); + pInfo->numOfTables = htonl(pInfo->numOfTables); + pInfo->numOfVgroups = htonl(pInfo->numOfVgroups); + + int32_t contLen = pMsg->rpcMsg.contLen - sizeof(SMultiTableInfoMsg); + + int32_t num = 0; + int32_t code = TSDB_CODE_SUCCESS; + char* str = strndup(pInfo->tableNames, contLen); + char** nameList = strsplit(str, ",", &num); + SArray* pList = taosArrayInit(4, POINTER_BYTES); + SMultiTableMeta *pMultiMeta = NULL; + + if (num != pInfo->numOfTables + pInfo->numOfVgroups) { + mError("msg:%p, app:%p, failed to get multi-tableMeta, msg inconsistent", pMsg, pMsg->rpcMsg.ahandle); + code = TSDB_CODE_MND_INVALID_TABLE_NAME; + goto _error; + } + + // first malloc 80KB, subsequent reallocation will expand the size as twice of the original size + int32_t totalMallocLen = sizeof(STableMetaMsg) + sizeof(SSchema) * (TSDB_MAX_TAGS + TSDB_MAX_COLUMNS + 16); + pMultiMeta = rpcMallocCont(totalMallocLen); if (pMultiMeta == NULL) { - return TSDB_CODE_MND_OUT_OF_MEMORY; + code = TSDB_CODE_MND_OUT_OF_MEMORY; + goto _error; } pMultiMeta->contLen = sizeof(SMultiTableMeta); pMultiMeta->numOfTables = 0; - for (int32_t t = 0; t < pInfo->numOfTables; ++t) { - char * tableId = (char *)(pInfo->tableIds + t * TSDB_TABLE_FNAME_LEN); - SCTableObj *pTable = mnodeGetChildTable(tableId); - if (pTable == NULL) continue; + int32_t t = 0; + for (; t < pInfo->numOfTables; ++t) { + char *fullName = nameList[t]; + + pMsg->pVgroup = NULL; + pMsg->pTable = mnodeGetTable(fullName); + if (pMsg->pTable == NULL) { + mError("msg:%p, app:%p table:%s, failed to get table meta, table not exist", pMsg, pMsg->rpcMsg.ahandle, fullName); + code = TSDB_CODE_MND_INVALID_TABLE_NAME; + goto _error; + } + + if (pMsg->pDb == NULL) { + pMsg->pDb = mnodeGetDbByTableName(fullName); + } - if (pMsg->pDb == NULL) pMsg->pDb = mnodeGetDbByTableName(tableId); if (pMsg->pDb == NULL || pMsg->pDb->status != TSDB_DB_STATUS_READY) { - mnodeDecTableRef(pTable); - continue; + mnodeDecTableRef(pMsg->pTable); + code = TSDB_CODE_APP_NOT_READY; + goto _error; } - int availLen = totalMallocLen - pMultiMeta->contLen; - if (availLen <= sizeof(STableMetaMsg) + sizeof(SSchema) * (TSDB_MAX_TAGS + TSDB_MAX_COLUMNS + 16)) { + int remain = totalMallocLen - pMultiMeta->contLen; + if (remain <= sizeof(STableMetaMsg) + sizeof(SSchema) * (TSDB_MAX_TAGS + TSDB_MAX_COLUMNS + 16)) { totalMallocLen *= 2; pMultiMeta = rpcReallocCont(pMultiMeta, totalMallocLen); if (pMultiMeta == NULL) { - mnodeDecTableRef(pTable); - return TSDB_CODE_MND_OUT_OF_MEMORY; - } else { - t--; - mnodeDecTableRef(pTable); - continue; + mnodeDecTableRef(pMsg->pTable); + code = TSDB_CODE_MND_OUT_OF_MEMORY; + goto _error; } } - STableMetaMsg *pMeta = (STableMetaMsg *)(pMultiMeta->metas + pMultiMeta->contLen); - int32_t code = mnodeDoGetChildTableMeta(pMsg, pMeta); + STableMetaMsg *pMeta = (STableMetaMsg *)((char*) pMultiMeta + pMultiMeta->contLen); + + if (pMsg->pTable->type == TSDB_SUPER_TABLE) { + code = mnodeDoGetSuperTableMeta(pMsg, pMeta); + taosArrayPush(pList, &fullName);// keep the full name for each super table for retrieve vgroup list + } else { + code = mnodeDoGetChildTableMeta(pMsg, pMeta); + } + if (code == TSDB_CODE_SUCCESS) { pMultiMeta->numOfTables ++; pMultiMeta->contLen += pMeta->contLen; } - mnodeDecTableRef(pTable); + mnodeDecTableRef(pMsg->pTable); + assert(((SCTableObj*)pMsg->pTable)->refCount >= 1); + pMsg->pTable = NULL; } + char* msg = (char*) pMultiMeta + pMultiMeta->contLen; + + // add the additional super table names that needs the vgroup info + for(;t < num; ++t) { + taosArrayPush(pList, &nameList[t]); + } + + // add the pVgroupList into the pList + int32_t numOfVgroupList = (int32_t) taosArrayGetSize(pList); + pMultiMeta->numOfVgroup = htonl(numOfVgroupList); + + for(int32_t i = 0; i < numOfVgroupList; ++i) { + char* name = taosArrayGetP(pList, i); + + SSTableObj *pTable = mnodeGetSuperTable(name); + if (pTable == NULL) { + mError("msg:%p, app:%p stable:%s, not exist while get stable vgroup info", pMsg, pMsg->rpcMsg.ahandle, name); + code = TSDB_CODE_MND_INVALID_TABLE_NAME; + goto _error; + } + + msg = serializeVgroupInfo(pTable, name, msg, pMsg, pMsg->rpcMsg.ahandle); + } + + pMultiMeta->contLen = (int32_t) (msg - (char*) pMultiMeta); + + pMultiMeta->numOfTables = htonl(pMultiMeta->numOfTables); pMsg->rpcRsp.rsp = pMultiMeta; pMsg->rpcRsp.len = pMultiMeta->contLen; return TSDB_CODE_SUCCESS; + + _error: + tfree(str); + tfree(nameList); + rpcFreeCont(pMultiMeta); + taosArrayDestroy(pList); + pMsg->pTable = NULL; + + return code; } static int32_t mnodeGetShowTableMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) { diff --git a/src/mnode/src/mnodeVgroup.c b/src/mnode/src/mnodeVgroup.c index 67532ad85a965a18e5bed3f4c2bc40763bedcefc..a64d2564841241c7f49b7da82a48b51d27783c1a 100644 --- a/src/mnode/src/mnodeVgroup.c +++ b/src/mnode/src/mnodeVgroup.c @@ -121,7 +121,7 @@ static int32_t mnodeVgroupActionDelete(SSdbRow *pRow) { SVgObj *pVgroup = pRow->pObj; if (pVgroup->pDb == NULL) { - mError("vgId:%d, db:%s is not exist while insert into hash", pVgroup->vgId, pVgroup->dbName); + mError("vgId:%d, db:%s is not exist while delete from hash", pVgroup->vgId, pVgroup->dbName); return TSDB_CODE_MND_VGROUP_NOT_EXIST; } diff --git a/src/os/inc/osDir.h b/src/os/inc/osDir.h index bdc65e452068cf34fd3969ada6ac69fbf5035217..7afe1264512bbffd34d9278ffb37034a473b827f 100644 --- a/src/os/inc/osDir.h +++ b/src/os/inc/osDir.h @@ -21,6 +21,7 @@ extern "C" { #endif void taosRemoveDir(char *rootDir); +bool taosDirExist(const char* dirname); int32_t taosMkDir(const char *pathname, mode_t mode); void taosRemoveOldLogFiles(char *rootDir, int32_t keepDays); int32_t taosRename(char *oldName, char *newName); diff --git a/src/os/src/detail/osDir.c b/src/os/src/detail/osDir.c index 2a2e2519b5dadf94ee8641064735e92267394a87..c467c64872d4b660af5cebb19b017f1528b55055 100644 --- a/src/os/src/detail/osDir.c +++ b/src/os/src/detail/osDir.c @@ -45,6 +45,10 @@ void taosRemoveDir(char *rootDir) { uInfo("dir:%s is removed", rootDir); } +bool taosDirExist(const char* dirname) { + return access(dirname, F_OK) == 0; +} + int taosMkDir(const char *path, mode_t mode) { int code = mkdir(path, 0755); if (code < 0 && errno == EEXIST) code = 0; diff --git a/src/os/src/detail/osTime.c b/src/os/src/detail/osTime.c index d9d070218e2a64e5f2535c073db4852b0a8960ef..67e0c2642e42f66229c437e603f7062edf571f34 100644 --- a/src/os/src/detail/osTime.c +++ b/src/os/src/detail/osTime.c @@ -87,12 +87,12 @@ static int32_t (*parseLocaltimeFp[]) (char* timestr, int64_t* time, int32_t time int32_t taosGetTimestampSec() { return (int32_t)time(NULL); } -int32_t taosParseTime(char* timestr, int64_t* time, int32_t len, int32_t timePrec, int8_t daylight) { +int32_t taosParseTime(char* timestr, int64_t* time, int32_t len, int32_t timePrec, int8_t day_light) { /* parse datatime string in with tz */ if (strnchr(timestr, 'T', len, false) != NULL) { return parseTimeWithTz(timestr, time, timePrec); } else { - return (*parseLocaltimeFp[daylight])(timestr, time, timePrec); + return (*parseLocaltimeFp[day_light])(timestr, time, timePrec); } } diff --git a/src/plugins/http/src/httpResp.c b/src/plugins/http/src/httpResp.c index 063f2bb04e4760d683434d9e6a600791c2a57888..4fcf236fce94da5062ae5063d081ca71aa8f2a98 100644 --- a/src/plugins/http/src/httpResp.c +++ b/src/plugins/http/src/httpResp.c @@ -165,7 +165,7 @@ void httpSendTaosdInvalidSqlErrorResp(HttpContext *pContext, char *errMsg) { } } - httpSendErrorRespImp(pContext, httpCode, "Bad Request", TSDB_CODE_TSC_INVALID_SQL & 0XFFFF, temp); + httpSendErrorRespImp(pContext, httpCode, "Bad Request", TSDB_CODE_TSC_INVALID_OPERATION & 0XFFFF, temp); } void httpSendSuccResp(HttpContext *pContext, char *desc) { diff --git a/src/plugins/http/src/httpSql.c b/src/plugins/http/src/httpSql.c index b345c1531f1f6904470a2e9dbae361834ef32fff..5a0480b6943d385b94acc0860534db09a5c24353 100644 --- a/src/plugins/http/src/httpSql.c +++ b/src/plugins/http/src/httpSql.c @@ -263,7 +263,7 @@ void httpProcessSingleSqlCallBackImp(void *param, TAOS_RES *result, int32_t code if (code != TSDB_CODE_SUCCESS) { SSqlObj *pObj = (SSqlObj *)result; - if (code == TSDB_CODE_TSC_INVALID_SQL) { + if (code == TSDB_CODE_TSC_INVALID_OPERATION) { terrno = code; httpError("context:%p, fd:%d, user:%s, query error, code:%s, sqlObj:%p, error:%s", pContext, pContext->fd, pContext->user, tstrerror(code), pObj, taos_errstr(pObj)); diff --git a/src/query/inc/qExecutor.h b/src/query/inc/qExecutor.h index b9361650e99a0531033cf4f680860730266f68f3..4ab718e09de962d74b792d12aed6bb5844008b9f 100644 --- a/src/query/inc/qExecutor.h +++ b/src/query/inc/qExecutor.h @@ -70,13 +70,13 @@ typedef struct SResultRowPool { SArray* pData; // SArray } SResultRowPool; -typedef struct SSqlGroupbyExpr { +typedef struct SGroupbyExpr { int16_t tableIndex; SArray* columnInfo; // SArray, group by columns information - int16_t numOfGroupCols; + int16_t numOfGroupCols; // todo remove it int16_t orderIndex; // order by column index int16_t orderType; // order by type: asc/desc -} SSqlGroupbyExpr; +} SGroupbyExpr; typedef struct SResultRow { int32_t pageId; // pageId & rowId is the position of current result in disk-based output buffer @@ -216,7 +216,7 @@ typedef struct SQueryAttr { int32_t intermediateResultRowSize; // intermediate result row size, in case of top-k query. int32_t maxTableColumnWidth; int32_t tagLen; // tag value length of current query - SSqlGroupbyExpr* pGroupbyExpr; + SGroupbyExpr* pGroupbyExpr; SExprInfo* pExpr1; SExprInfo* pExpr2; @@ -302,6 +302,7 @@ enum OPERATOR_TYPE_E { OP_GlobalAggregate = 18, // global merge for the multi-way data sources. OP_Filter = 19, OP_Distinct = 20, + OP_Join = 21, }; typedef struct SOperatorInfo { @@ -314,7 +315,8 @@ typedef struct SOperatorInfo { SExprInfo *pExpr; SQueryRuntimeEnv *pRuntimeEnv; - struct SOperatorInfo *upstream; + struct SOperatorInfo **upstream; // upstream pointer list + int32_t numOfUpstream; // number of upstream. The value is always ONE expect for join operator __operator_fn_t exec; __optr_cleanup_fn_t cleanup; } SOperatorInfo; @@ -362,7 +364,7 @@ typedef struct SQueryParam { SColIndex *pGroupColIndex; SColumnInfo *pTagColumnInfo; - SSqlGroupbyExpr *pGroupbyExpr; + SGroupbyExpr *pGroupbyExpr; int32_t tableScanOperator; SArray *pOperator; } SQueryParam; @@ -494,6 +496,8 @@ typedef struct SMultiwayMergeInfo { bool groupMix; } SMultiwayMergeInfo; +void appendUpstream(SOperatorInfo* p, SOperatorInfo* pUpstream); + SOperatorInfo* createDataBlocksOptScanInfo(void* pTsdbQueryHandle, SQueryRuntimeEnv* pRuntimeEnv, int32_t repeatTime, int32_t reverseTime); SOperatorInfo* createTableScanOperator(void* pTsdbQueryHandle, SQueryRuntimeEnv* pRuntimeEnv, int32_t repeatTime); SOperatorInfo* createTableSeqScanOperator(void* pTsdbQueryHandle, SQueryRuntimeEnv* pRuntimeEnv); @@ -514,12 +518,20 @@ SOperatorInfo* createMultiwaySortOperatorInfo(SQueryRuntimeEnv* pRuntimeEnv, SEx int32_t numOfRows, void* merger, bool groupMix); SOperatorInfo* createGlobalAggregateOperatorInfo(SQueryRuntimeEnv* pRuntimeEnv, SOperatorInfo* upstream, SExprInfo* pExpr, int32_t numOfOutput, void* param); SOperatorInfo* createSLimitOperatorInfo(SQueryRuntimeEnv* pRuntimeEnv, SOperatorInfo* upstream, SExprInfo* pExpr, int32_t numOfOutput, void* merger); -SOperatorInfo* createFilterOperatorInfo(SQueryRuntimeEnv* pRuntimeEnv, SOperatorInfo* upstream, SExprInfo* pExpr, int32_t numOfOutput); +SOperatorInfo* createFilterOperatorInfo(SQueryRuntimeEnv* pRuntimeEnv, SOperatorInfo* upstream, SExprInfo* pExpr, + int32_t numOfOutput, SColumnInfo* pCols, int32_t numOfFilter); + +SOperatorInfo* createJoinOperatorInfo(SOperatorInfo** pUpstream, int32_t numOfUpstream, SSchema* pSchema, int32_t numOfOutput); SSDataBlock* doGlobalAggregate(void* param, bool* newgroup); SSDataBlock* doMultiwayMergeSort(void* param, bool* newgroup); SSDataBlock* doSLimit(void* param, bool* newgroup); +int32_t doCreateFilterInfo(SColumnInfo* pCols, int32_t numOfCols, int32_t numOfFilterCols, SSingleColumnFilterInfo** pFilterInfo, uint64_t qId); +void doSetFilterColumnInfo(SSingleColumnFilterInfo* pFilterInfo, int32_t numOfFilterCols, SSDataBlock* pBlock); +bool doFilterDataBlock(SSingleColumnFilterInfo* pFilterInfo, int32_t numOfFilterCols, int32_t numOfRows, int8_t* p); +void doCompactSDataBlock(SSDataBlock* pBlock, int32_t numOfRows, int8_t* p); + SSDataBlock* createOutputBuf(SExprInfo* pExpr, int32_t numOfOutput, int32_t numOfRows); void* destroyOutputBuf(SSDataBlock* pBlock); @@ -536,13 +548,14 @@ int32_t createQueryFunc(SQueriedTableInfo* pTableInfo, int32_t numOfOutput, SExp int32_t createIndirectQueryFuncExprFromMsg(SQueryTableMsg *pQueryMsg, int32_t numOfOutput, SExprInfo **pExprInfo, SSqlExpr **pExpr, SExprInfo *prevExpr); -SSqlGroupbyExpr *createGroupbyExprFromMsg(SQueryTableMsg *pQueryMsg, SColIndex *pColIndex, int32_t *code); -SQInfo *createQInfoImpl(SQueryTableMsg *pQueryMsg, SSqlGroupbyExpr *pGroupbyExpr, SExprInfo *pExprs, +SGroupbyExpr *createGroupbyExprFromMsg(SQueryTableMsg *pQueryMsg, SColIndex *pColIndex, int32_t *code); +SQInfo *createQInfoImpl(SQueryTableMsg *pQueryMsg, SGroupbyExpr *pGroupbyExpr, SExprInfo *pExprs, SExprInfo *pSecExprs, STableGroupInfo *pTableGroupInfo, SColumnInfo* pTagCols, int32_t vgId, char* sql, uint64_t *qId); int32_t initQInfo(STsBufInfo* pTsBufInfo, void* tsdb, void* sourceOptr, SQInfo* pQInfo, SQueryParam* param, char* start, int32_t prevResultLen, void* merger); +int32_t createFilterInfo(SQueryAttr* pQueryAttr, uint64_t qId); void freeColumnFilterInfo(SColumnFilterInfo* pFilter, int32_t numOfFilters); STableQueryInfo *createTableQueryInfo(SQueryAttr* pQueryAttr, void* pTable, bool groupbyColumn, STimeWindow win, void* buf); diff --git a/src/query/inc/qFill.h b/src/query/inc/qFill.h index 00ac86caf4a079e01ad239496370d97ac28e84f2..caa0c55b3ff1a805975a6258ca360e5099f4894e 100644 --- a/src/query/inc/qFill.h +++ b/src/query/inc/qFill.h @@ -62,7 +62,7 @@ typedef struct SFillInfo { SFillColInfo* pFillCol; // column info for fill operations SFillTagColInfo* pTags; // tags value for filling gap - void* handle; // for dubug purpose + void* handle; // for debug purpose } SFillInfo; typedef struct SPoint { @@ -82,8 +82,6 @@ void taosFillSetStartInfo(SFillInfo* pFillInfo, int32_t numOfRows, TSKEY endKey) void taosFillSetInputDataBlock(SFillInfo* pFillInfo, const struct SSDataBlock* pInput); -void taosFillCopyInputDataFromOneFilePage(SFillInfo* pFillInfo, const tFilePage* pInput); - bool taosFillHasMoreResults(SFillInfo* pFillInfo); int64_t getNumOfResultsAfterFillGap(SFillInfo* pFillInfo, int64_t ekey, int32_t maxNumOfRows); diff --git a/src/query/inc/qPlan.h b/src/query/inc/qPlan.h index 8f35565e4bccd4896ec49ddb30f7236ac4b4650c..30c0f8db4e43f33e8ac69e2bcde50ca77abac4d2 100644 --- a/src/query/inc/qPlan.h +++ b/src/query/inc/qPlan.h @@ -16,7 +16,38 @@ #ifndef TDENGINE_QPLAN_H #define TDENGINE_QPLAN_H -//TODO refactor +struct SQueryInfo; + +typedef struct SQueryNodeBasicInfo { + int32_t type; + char *name; +} SQueryNodeBasicInfo; + +typedef struct SQueryTableInfo { + char *tableName; + STableId id; +} SQueryTableInfo; + +typedef struct SQueryNode { + SQueryNodeBasicInfo info; + SQueryTableInfo tableInfo; + SSchema *pSchema; // the schema of the input SSDatablock + int32_t numOfCols; // number of input columns + SExprInfo *pExpr; // the query functions or sql aggregations + int32_t numOfOutput; // number of result columns, which is also the number of pExprs + + void *pExtInfo; // additional information + // previous operator to generated result for current node to process + // in case of join, multiple prev nodes exist. + SArray *pPrevNodes;// upstream nodes + struct SQueryNode *nextNode; +} SQueryNode; + +SQueryNode* qCreateQueryPlan(struct SQueryInfo* pQueryInfo); +void* qDestroyQueryPlan(SQueryNode* pQueryNode); + +char* queryPlanToString(SQueryNode* pQueryNode); + SArray* createTableScanPlan(SQueryAttr* pQueryAttr); SArray* createExecOperatorPlan(SQueryAttr* pQueryAttr); SArray* createGlobalMergePlan(SQueryAttr* pQueryAttr); diff --git a/src/query/inc/qSqlparser.h b/src/query/inc/qSqlparser.h index 2bdff1f94fa702aa9c1da0b4da0a5b0804a43cdd..bdbe3ac05099384adc7a15e4a7674f4c48d9c14b 100644 --- a/src/query/inc/qSqlparser.h +++ b/src/query/inc/qSqlparser.h @@ -107,14 +107,18 @@ typedef struct SSqlNode { struct tSqlExpr *pHaving; // having clause [optional] } SSqlNode; -typedef struct STableNamePair { - SStrToken name; +typedef struct SRelElementPair { + union { + SStrToken tableName; + SArray *pSubquery; + }; + SStrToken aliasName; -} STableNamePair; +} SRelElementPair; typedef struct SRelationInfo { int32_t type; // nested query|table name list - SArray *list; // SArray|SArray + SArray *list; // SArray } SRelationInfo; typedef struct SCreatedTableInfo { @@ -254,8 +258,9 @@ SArray *tVariantListInsert(SArray *pList, tVariant *pVar, uint8_t sortOrder, int SArray *tVariantListAppendToken(SArray *pList, SStrToken *pAliasToken, uint8_t sortOrder); SRelationInfo *setTableNameList(SRelationInfo* pFromInfo, SStrToken *pName, SStrToken* pAlias); -SRelationInfo *setSubquery(SRelationInfo* pFromInfo, SArray* pSqlNode); +//SRelationInfo *setSubquery(SRelationInfo* pFromInfo, SRelElementPair* p); void *destroyRelationInfo(SRelationInfo* pFromInfo); +SRelationInfo *addSubqueryElem(SRelationInfo* pRelationInfo, SArray* pSub, SStrToken* pAlias); // sql expr leaf node tSqlExpr *tSqlExprCreateIdValue(SStrToken *pToken, int32_t optrType); diff --git a/src/query/inc/qUtil.h b/src/query/inc/qUtil.h index 3ca6d967463ff86de82112bb86307715c73715cd..0756e4178598f09ecdc784b07e226b9054edcfef 100644 --- a/src/query/inc/qUtil.h +++ b/src/query/inc/qUtil.h @@ -47,6 +47,9 @@ void clearResultRow(SQueryRuntimeEnv* pRuntimeEnv, SResultRow* pResultRow, in SResultRowCellInfo* getResultCell(const SResultRow* pRow, int32_t index, int32_t* offset); +void* destroyQueryFuncExpr(SExprInfo* pExprInfo, int32_t numOfExpr); +void* freeColumnInfo(SColumnInfo* pColumnInfo, int32_t numOfCols); + static FORCE_INLINE SResultRow *getResultRow(SResultRowInfo *pResultRowInfo, int32_t slot) { assert(pResultRowInfo != NULL && slot >= 0 && slot < pResultRowInfo->size); return pResultRowInfo->pResult[slot]; diff --git a/src/query/inc/sql.y b/src/query/inc/sql.y index 8ef8ef0e2b3080ce334686f7fdbb7105596356e5..2b5126c2fd5d60e7bd8e40e8b313f0bb277b709f 100644 --- a/src/query/inc/sql.y +++ b/src/query/inc/sql.y @@ -28,7 +28,7 @@ #include #include "qSqlparser.h" #include "tcmdtype.h" -#include "tstoken.h" +#include "ttoken.h" #include "ttokendef.h" #include "tutil.h" #include "tvariant.h" @@ -512,7 +512,13 @@ distinct(X) ::= . { X.n = 0;} %type from {SRelationInfo*} %destructor from {destroyRelationInfo($$);} from(A) ::= FROM tablelist(X). {A = X;} -from(A) ::= FROM LP union(Y) RP. {A = setSubquery(NULL, Y);} +from(A) ::= FROM sub(X). {A = X;} + +%type sub {SRelationInfo*} +%destructor sub {destroyRelationInfo($$);} +sub(A) ::= LP union(Y) RP. {A = addSubqueryElem(NULL, Y, NULL);} +sub(A) ::= LP union(Y) RP ids(Z). {A = addSubqueryElem(NULL, Y, &Z);} +sub(A) ::= sub(X) COMMA LP union(Y) RP ids(Z).{A = addSubqueryElem(X, Y, &Z);} %type tablelist {SRelationInfo*} %destructor tablelist {destroyRelationInfo($$);} diff --git a/src/query/src/qAggMain.c b/src/query/src/qAggMain.c index 7b656d473afd0374ee48c6992a285a8fddec5b68..177cea8b62f8297586d3e39b1cfd2994734c1a62 100644 --- a/src/query/src/qAggMain.c +++ b/src/query/src/qAggMain.c @@ -166,7 +166,7 @@ int32_t getResultDataInfo(int32_t dataType, int32_t dataBytes, int32_t functionI int16_t *bytes, int32_t *interBytes, int16_t extLength, bool isSuperTable) { if (!isValidDataType(dataType)) { qError("Illegal data type %d or data type length %d", dataType, dataBytes); - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } if (functionId == TSDB_FUNC_TS || functionId == TSDB_FUNC_TS_DUMMY || functionId == TSDB_FUNC_TAG_DUMMY || @@ -353,7 +353,7 @@ int32_t getResultDataInfo(int32_t dataType, int32_t dataBytes, int32_t functionI *interBytes = (*bytes); } else { - return TSDB_CODE_TSC_INVALID_SQL; + return TSDB_CODE_TSC_INVALID_OPERATION; } return TSDB_CODE_SUCCESS; @@ -3700,7 +3700,7 @@ char *getArithColumnData(void *param, const char* name, int32_t colId) { } } - assert(index >= 0 /*&& colId >= 0*/); + assert(index >= 0); return pSupport->data[index] + pSupport->offset * pSupport->colList[index].bytes; } @@ -4839,7 +4839,7 @@ static void mergeTableBlockDist(STableBlockDist* pDist, const STableBlockDist* p pDist->dataBlockInfos = taosArrayInit(4, sizeof(SFileBlockInfo)); } - taosArrayPushBatch(pDist->dataBlockInfos, pSrc->dataBlockInfos->pData, (int32_t) taosArrayGetSize(pSrc->dataBlockInfos)); + taosArrayAddBatch(pDist->dataBlockInfos, pSrc->dataBlockInfos->pData, (int32_t) taosArrayGetSize(pSrc->dataBlockInfos)); } void block_func_merge(SQLFunctionCtx* pCtx) { diff --git a/src/query/src/qExecutor.c b/src/query/src/qExecutor.c index 40759c93c4f62721c2ba81d5c5eaed40492de328..cb94525614dc27d5905195d347818026ae7aa592 100644 --- a/src/query/src/qExecutor.c +++ b/src/query/src/qExecutor.c @@ -33,6 +33,8 @@ #define SET_MASTER_SCAN_FLAG(runtime) ((runtime)->scanFlag = MASTER_SCAN) #define SET_REVERSE_SCAN_FLAG(runtime) ((runtime)->scanFlag = REVERSE_SCAN) +#define TSWINDOW_IS_EQUAL(t1, t2) (((t1).skey == (t2).skey) && ((t1).ekey == (t2).ekey)) + #define SWITCH_ORDER(n) (((n) = ((n) == TSDB_ORDER_ASC) ? TSDB_ORDER_DESC : TSDB_ORDER_ASC)) #define SDATA_BLOCK_INITIALIZER (SDataBlockInfo) {{0}, 0} @@ -169,6 +171,8 @@ static void setBlockStatisInfo(SQLFunctionCtx *pCtx, SSDataBlock* pSDataBlock, S static void destroyTableQueryInfoImpl(STableQueryInfo *pTableQueryInfo); static bool hasMainOutput(SQueryAttr *pQueryAttr); +static SColumnInfo* extractColumnFilterInfo(SExprInfo* pExpr, int32_t numOfOutput, int32_t* numOfFilterCols); + static int32_t setTimestampListJoinInfo(SQueryRuntimeEnv* pRuntimeEnv, tVariant* pTag, STableQueryInfo *pTableQueryInfo); static void releaseQueryBuf(size_t numOfTables); static int32_t binarySearchForKey(char *pValue, int num, TSKEY key, int order); @@ -176,8 +180,6 @@ static STsdbQueryCond createTsdbQueryCond(SQueryAttr* pQueryAttr, STimeWindow* w static STableIdInfo createTableIdInfo(STableQueryInfo* pTableQueryInfo); static void setTableScanFilterOperatorInfo(STableScanInfo* pTableScanInfo, SOperatorInfo* pDownstream); -static int32_t doCreateFilterInfo(SColumnInfo* pCols, int32_t numOfCols, int32_t numOfFilterCols, - SSingleColumnFilterInfo** pFilterInfo, uint64_t qId); static void* doDestroyFilterInfo(SSingleColumnFilterInfo* pFilterInfo, int32_t numOfFilterCols); static int32_t getNumOfScanTimes(SQueryAttr* pQueryAttr); @@ -191,7 +193,7 @@ static void destroyOperatorInfo(SOperatorInfo* pOperator); static int32_t doCopyToSDataBlock(SQueryRuntimeEnv* pRuntimeEnv, SGroupResInfo* pGroupResInfo, int32_t orderType, SSDataBlock* pBlock); -static int32_t getGroupbyColumnIndex(SSqlGroupbyExpr *pGroupbyExpr, SSDataBlock* pDataBlock); +static int32_t getGroupbyColumnIndex(SGroupbyExpr *pGroupbyExpr, SSDataBlock* pDataBlock); static int32_t setGroupResultOutputBuf(SQueryRuntimeEnv *pRuntimeEnv, SGroupbyOperatorInfo *pInfo, int32_t numOfCols, char *pData, int16_t type, int16_t bytes, int32_t groupIndex); static void initCtxOutputBuffer(SQLFunctionCtx* pCtx, int32_t size); @@ -1420,7 +1422,7 @@ static int32_t setGroupResultOutputBuf(SQueryRuntimeEnv *pRuntimeEnv, SGroupbyOp return TSDB_CODE_SUCCESS; } -static int32_t getGroupbyColumnIndex(SSqlGroupbyExpr *pGroupbyExpr, SSDataBlock* pDataBlock) { +static int32_t getGroupbyColumnIndex(SGroupbyExpr *pGroupbyExpr, SSDataBlock* pDataBlock) { for (int32_t k = 0; k < pGroupbyExpr->numOfGroupCols; ++k) { SColIndex* pColIndex = taosArrayGet(pGroupbyExpr->columnInfo, k); if (TSDB_COL_IS_TAG(pColIndex->flag)) { @@ -1710,38 +1712,40 @@ static int32_t setupQueryRuntimeEnv(SQueryRuntimeEnv *pRuntimeEnv, int32_t numOf case OP_MultiTableTimeInterval: { pRuntimeEnv->proot = createMultiTableTimeIntervalOperatorInfo(pRuntimeEnv, pRuntimeEnv->proot, pQueryAttr->pExpr1, pQueryAttr->numOfOutput); - setTableScanFilterOperatorInfo(pRuntimeEnv->proot->upstream->info, pRuntimeEnv->proot); + setTableScanFilterOperatorInfo(pRuntimeEnv->proot->upstream[0]->info, pRuntimeEnv->proot); break; } case OP_TimeWindow: { pRuntimeEnv->proot = createTimeIntervalOperatorInfo(pRuntimeEnv, pRuntimeEnv->proot, pQueryAttr->pExpr1, pQueryAttr->numOfOutput); - setTableScanFilterOperatorInfo(pRuntimeEnv->proot->upstream->info, pRuntimeEnv->proot); + setTableScanFilterOperatorInfo(pRuntimeEnv->proot->upstream[0]->info, pRuntimeEnv->proot); break; } case OP_Groupby: { pRuntimeEnv->proot = createGroupbyOperatorInfo(pRuntimeEnv, pRuntimeEnv->proot, pQueryAttr->pExpr1, pQueryAttr->numOfOutput); - setTableScanFilterOperatorInfo(pRuntimeEnv->proot->upstream->info, pRuntimeEnv->proot); + setTableScanFilterOperatorInfo(pRuntimeEnv->proot->upstream[0]->info, pRuntimeEnv->proot); break; } case OP_SessionWindow: { pRuntimeEnv->proot = createSWindowOperatorInfo(pRuntimeEnv, pRuntimeEnv->proot, pQueryAttr->pExpr1, pQueryAttr->numOfOutput); - setTableScanFilterOperatorInfo(pRuntimeEnv->proot->upstream->info, pRuntimeEnv->proot); + setTableScanFilterOperatorInfo(pRuntimeEnv->proot->upstream[0]->info, pRuntimeEnv->proot); break; } case OP_MultiTableAggregate: { pRuntimeEnv->proot = createMultiTableAggOperatorInfo(pRuntimeEnv, pRuntimeEnv->proot, pQueryAttr->pExpr1, pQueryAttr->numOfOutput); - setTableScanFilterOperatorInfo(pRuntimeEnv->proot->upstream->info, pRuntimeEnv->proot); + setTableScanFilterOperatorInfo(pRuntimeEnv->proot->upstream[0]->info, pRuntimeEnv->proot); break; } case OP_Aggregate: { pRuntimeEnv->proot = createAggregateOperatorInfo(pRuntimeEnv, pRuntimeEnv->proot, pQueryAttr->pExpr1, pQueryAttr->numOfOutput); - if (pRuntimeEnv->proot->upstream->operatorType != OP_DummyInput) { - setTableScanFilterOperatorInfo(pRuntimeEnv->proot->upstream->info, pRuntimeEnv->proot); + + int32_t opType = pRuntimeEnv->proot->upstream[0]->operatorType; + if (opType != OP_DummyInput && opType != OP_Join) { + setTableScanFilterOperatorInfo(pRuntimeEnv->proot->upstream[0]->info, pRuntimeEnv->proot); } break; } @@ -1750,7 +1754,7 @@ static int32_t setupQueryRuntimeEnv(SQueryRuntimeEnv *pRuntimeEnv, int32_t numOf SOperatorInfo* prev = pRuntimeEnv->proot; if (i == 0) { pRuntimeEnv->proot = createArithOperatorInfo(pRuntimeEnv, prev, pQueryAttr->pExpr1, pQueryAttr->numOfOutput); - if (pRuntimeEnv->proot != NULL && pRuntimeEnv->proot->operatorType != OP_DummyInput) { // TODO refactor + if (pRuntimeEnv->proot != NULL && prev->operatorType != OP_DummyInput && prev->operatorType != OP_Join) { // TODO refactor setTableScanFilterOperatorInfo(prev->info, pRuntimeEnv->proot); } } else { @@ -1767,12 +1771,25 @@ static int32_t setupQueryRuntimeEnv(SQueryRuntimeEnv *pRuntimeEnv, int32_t numOf } case OP_Filter: { // todo refactor - assert(pQueryAttr->havingNum > 0); - if (pQueryAttr->stableQuery) { - pRuntimeEnv->proot = createFilterOperatorInfo(pRuntimeEnv, pRuntimeEnv->proot, pQueryAttr->pExpr3, pQueryAttr->numOfExpr3); - } else { - pRuntimeEnv->proot = createFilterOperatorInfo(pRuntimeEnv, pRuntimeEnv->proot, pQueryAttr->pExpr1, pQueryAttr->numOfOutput); - } + int32_t numOfFilterCols = 0; +// if (pQueryAttr->numOfFilterCols > 0) { +// pRuntimeEnv->proot = createFilterOperatorInfo(pRuntimeEnv, pRuntimeEnv->proot, pQueryAttr->pExpr1, +// pQueryAttr->numOfOutput, pQueryAttr->tableCols, pQueryAttr->numOfFilterCols); +// } else { + if (pQueryAttr->stableQuery) { + SColumnInfo* pColInfo = + extractColumnFilterInfo(pQueryAttr->pExpr3, pQueryAttr->numOfExpr3, &numOfFilterCols); + pRuntimeEnv->proot = createFilterOperatorInfo(pRuntimeEnv, pRuntimeEnv->proot, pQueryAttr->pExpr3, + pQueryAttr->numOfExpr3, pColInfo, numOfFilterCols); + freeColumnInfo(pColInfo, pQueryAttr->numOfExpr3); + } else { + SColumnInfo* pColInfo = + extractColumnFilterInfo(pQueryAttr->pExpr1, pQueryAttr->numOfOutput, &numOfFilterCols); + pRuntimeEnv->proot = createFilterOperatorInfo(pRuntimeEnv, pRuntimeEnv->proot, pQueryAttr->pExpr1, + pQueryAttr->numOfOutput, pColInfo, numOfFilterCols); + freeColumnInfo(pColInfo, pQueryAttr->numOfOutput); + } +// } break; } @@ -1979,6 +1996,37 @@ static bool isFirstLastRowQuery(SQueryAttr *pQueryAttr) { return false; } +static bool isCachedLastQuery(SQueryAttr *pQueryAttr) { + for (int32_t i = 0; i < pQueryAttr->numOfOutput; ++i) { + int32_t functionID = pQueryAttr->pExpr1[i].base.functionId; + if (functionID == TSDB_FUNC_LAST || functionID == TSDB_FUNC_LAST_DST) { + continue; + } + + return false; + } + + if (pQueryAttr->order.order != TSDB_ORDER_DESC || !TSWINDOW_IS_EQUAL(pQueryAttr->window, TSWINDOW_DESC_INITIALIZER)) { + return false; + } + + if (pQueryAttr->groupbyColumn) { + return false; + } + + if (pQueryAttr->interval.interval > 0) { + return false; + } + + if (pQueryAttr->numOfFilterCols > 0 || pQueryAttr->havingNum > 0) { + return false; + } + + return true; +} + + + /** * The following 4 kinds of query are treated as the tags query * tagprj, tid_tag query, count(tbname), 'abc' (user defined constant value column) query @@ -2084,7 +2132,7 @@ static int32_t updateBlockLoadStatus(SQueryAttr *pQuery, int32_t status) { if (hasFirstLastFunc && status == BLK_DATA_NO_NEEDED) { if(!hasOtherFunc) { return BLK_DATA_DISCARD; - } else{ + } else { return BLK_DATA_ALL_NEEDED; } } @@ -2359,6 +2407,105 @@ static int32_t doTSJoinFilter(SQueryRuntimeEnv *pRuntimeEnv, TSKEY key, bool asc return TS_JOIN_TS_EQUAL; } +bool doFilterDataBlock(SSingleColumnFilterInfo* pFilterInfo, int32_t numOfFilterCols, int32_t numOfRows, int8_t* p) { + bool all = true; + + for (int32_t i = 0; i < numOfRows; ++i) { + bool qualified = false; + + for (int32_t k = 0; k < numOfFilterCols; ++k) { + char* pElem = (char*)pFilterInfo[k].pData + pFilterInfo[k].info.bytes * i; + + qualified = false; + for (int32_t j = 0; j < pFilterInfo[k].numOfFilters; ++j) { + SColumnFilterElem* pFilterElem = &pFilterInfo[k].pFilters[j]; + + bool isnull = isNull(pElem, pFilterInfo[k].info.type); + if (isnull) { + if (pFilterElem->fp == isNullOperator) { + qualified = true; + break; + } else { + continue; + } + } else { + if (pFilterElem->fp == notNullOperator) { + qualified = true; + break; + } else if (pFilterElem->fp == isNullOperator) { + continue; + } + } + + if (pFilterElem->fp(pFilterElem, pElem, pElem, pFilterInfo[k].info.type)) { + qualified = true; + break; + } + } + + if (!qualified) { + break; + } + } + + p[i] = qualified ? 1 : 0; + if (!qualified) { + all = false; + } + } + + return all; +} + +void doCompactSDataBlock(SSDataBlock* pBlock, int32_t numOfRows, int8_t* p) { + int32_t len = 0; + int32_t start = 0; + for (int32_t j = 0; j < numOfRows; ++j) { + if (p[j] == 1) { + len++; + } else { + if (len > 0) { + int32_t cstart = j - len; + for (int32_t i = 0; i < pBlock->info.numOfCols; ++i) { + SColumnInfoData* pColumnInfoData = taosArrayGet(pBlock->pDataBlock, i); + + int16_t bytes = pColumnInfoData->info.bytes; + memmove(((char*)pColumnInfoData->pData) + start * bytes, pColumnInfoData->pData + cstart * bytes, + len * bytes); + } + + start += len; + len = 0; + } + } + } + + if (len > 0) { + int32_t cstart = numOfRows - len; + for (int32_t i = 0; i < pBlock->info.numOfCols; ++i) { + SColumnInfoData* pColumnInfoData = taosArrayGet(pBlock->pDataBlock, i); + + int16_t bytes = pColumnInfoData->info.bytes; + memmove(pColumnInfoData->pData + start * bytes, pColumnInfoData->pData + cstart * bytes, len * bytes); + } + + start += len; + len = 0; + } + + pBlock->info.rows = start; + pBlock->pBlockStatis = NULL; // clean the block statistics info + + if (start > 0) { + SColumnInfoData* pColumnInfoData = taosArrayGet(pBlock->pDataBlock, 0); + if (pColumnInfoData->info.type == TSDB_DATA_TYPE_TIMESTAMP && + pColumnInfoData->info.colId == PRIMARYKEY_TIMESTAMP_COL_INDEX) { + pBlock->info.window.skey = *(int64_t*)pColumnInfoData->pData; + pBlock->info.window.ekey = *(int64_t*)(pColumnInfoData->pData + TSDB_KEYSIZE * (start - 1)); + } + } +} + void filterRowsInDataBlock(SQueryRuntimeEnv* pRuntimeEnv, SSingleColumnFilterInfo* pFilterInfo, int32_t numOfFilterCols, SSDataBlock* pBlock, bool ascQuery) { int32_t numOfRows = pBlock->info.rows; @@ -2391,97 +2538,11 @@ void filterRowsInDataBlock(SQueryRuntimeEnv* pRuntimeEnv, SSingleColumnFilterInf // save the cursor status pRuntimeEnv->current->cur = tsBufGetCursor(pRuntimeEnv->pTsBuf); } else { - for (int32_t i = 0; i < numOfRows; ++i) { - bool qualified = false; - - for (int32_t k = 0; k < numOfFilterCols; ++k) { - char* pElem = (char*)pFilterInfo[k].pData + pFilterInfo[k].info.bytes * i; - - qualified = false; - for (int32_t j = 0; j < pFilterInfo[k].numOfFilters; ++j) { - SColumnFilterElem* pFilterElem = &pFilterInfo[k].pFilters[j]; - - bool isnull = isNull(pElem, pFilterInfo[k].info.type); - if (isnull) { - if (pFilterElem->fp == isNullOperator) { - qualified = true; - break; - } else { - continue; - } - } else { - if (pFilterElem->fp == notNullOperator) { - qualified = true; - break; - } else if (pFilterElem->fp == isNullOperator) { - continue; - } - } - - if (pFilterElem->fp(pFilterElem, pElem, pElem, pFilterInfo[k].info.type)) { - qualified = true; - break; - } - } - - if (!qualified) { - break; - } - } - - p[i] = qualified ? 1 : 0; - if (!qualified) { - all = false; - } - } + all = doFilterDataBlock(pFilterInfo, numOfFilterCols, numOfRows, p); } if (!all) { - int32_t start = 0; - int32_t len = 0; - for (int32_t j = 0; j < numOfRows; ++j) { - if (p[j] == 1) { - len++; - } else { - if (len > 0) { - int32_t cstart = j - len; - for (int32_t i = 0; i < pBlock->info.numOfCols; ++i) { - SColumnInfoData *pColumnInfoData = taosArrayGet(pBlock->pDataBlock, i); - - int16_t bytes = pColumnInfoData->info.bytes; - memmove(((char*)pColumnInfoData->pData) + start * bytes, pColumnInfoData->pData + cstart * bytes, len * bytes); - } - - start += len; - len = 0; - } - } - } - - if (len > 0) { - int32_t cstart = numOfRows - len; - for (int32_t i = 0; i < pBlock->info.numOfCols; ++i) { - SColumnInfoData *pColumnInfoData = taosArrayGet(pBlock->pDataBlock, i); - - int16_t bytes = pColumnInfoData->info.bytes; - memmove(pColumnInfoData->pData + start * bytes, pColumnInfoData->pData + cstart * bytes, len * bytes); - } - - start += len; - len = 0; - } - - pBlock->info.rows = start; - pBlock->pBlockStatis = NULL; // clean the block statistics info - - if (start > 0) { - SColumnInfoData* pColumnInfoData = taosArrayGet(pBlock->pDataBlock, 0); - if (pColumnInfoData->info.type == TSDB_DATA_TYPE_TIMESTAMP && - pColumnInfoData->info.colId == PRIMARYKEY_TIMESTAMP_COL_INDEX) { - pBlock->info.window.skey = *(int64_t*)pColumnInfoData->pData; - pBlock->info.window.ekey = *(int64_t*)(pColumnInfoData->pData + TSDB_KEYSIZE * (start - 1)); - } - } + doCompactSDataBlock(pBlock, numOfRows, p); } tfree(p); @@ -2509,7 +2570,7 @@ static uint32_t doFilterByBlockTimeWindow(STableScanInfo* pTableScanInfo, SSData return status; } -static void doSetFilterColumnInfo(SSingleColumnFilterInfo* pFilterInfo, int32_t numOfFilterCols, SSDataBlock* pBlock) { +void doSetFilterColumnInfo(SSingleColumnFilterInfo* pFilterInfo, int32_t numOfFilterCols, SSDataBlock* pBlock) { if (numOfFilterCols > 0 && pFilterInfo[0].pData != NULL) { return; } @@ -3914,6 +3975,15 @@ void queryCostStatis(SQInfo *pQInfo) { // return true; //} +void appendUpstream(SOperatorInfo* p, SOperatorInfo* pUpstream) { + if (p->upstream == NULL) { + assert(p->numOfUpstream == 0); + } + + p->upstream = realloc(p->upstream, POINTER_BYTES * (p->numOfUpstream + 1)); + p->upstream[p->numOfUpstream++] = pUpstream; +} + static void doDestroyTableQueryInfo(STableGroupInfo* pTableqinfoGroupInfo); static int32_t setupQueryHandle(void* tsdb, SQueryRuntimeEnv* pRuntimeEnv, int64_t qId, bool isSTableQuery) { @@ -3963,6 +4033,8 @@ static int32_t setupQueryHandle(void* tsdb, SQueryRuntimeEnv* pRuntimeEnv, int64 } } } + } else if (isCachedLastQuery(pQueryAttr)) { + pRuntimeEnv->pQueryHandle = tsdbQueryCacheLast(tsdb, &cond, &pQueryAttr->tableGroupInfo, qId, &pQueryAttr->memRef); } else if (pQueryAttr->pointInterpQuery) { pRuntimeEnv->pQueryHandle = tsdbQueryRowsInExternalWindow(tsdb, &cond, &pQueryAttr->tableGroupInfo, qId, &pQueryAttr->memRef); } else { @@ -4143,12 +4215,12 @@ static void doCloseAllTimeWindow(SQueryRuntimeEnv* pRuntimeEnv) { } static SSDataBlock* doTableScanImpl(void* param, bool* newgroup) { - SOperatorInfo* pOperator = (SOperatorInfo*) param; + SOperatorInfo *pOperator = (SOperatorInfo*) param; - STableScanInfo* pTableScanInfo = pOperator->info; - SSDataBlock* pBlock = &pTableScanInfo->block; + STableScanInfo *pTableScanInfo = pOperator->info; + SSDataBlock *pBlock = &pTableScanInfo->block; SQueryRuntimeEnv *pRuntimeEnv = pOperator->pRuntimeEnv; - SQueryAttr* pQueryAttr = pRuntimeEnv->pQueryAttr; + SQueryAttr *pQueryAttr = pRuntimeEnv->pQueryAttr; STableGroupInfo *pTableGroupInfo = &pOperator->pRuntimeEnv->tableqinfoGroupInfo; *newgroup = false; @@ -4208,7 +4280,7 @@ static SSDataBlock* doTableScan(void* param, bool *newgroup) { } if (++pTableScanInfo->current >= pTableScanInfo->times) { - if (pTableScanInfo->reverseTimes <= 0) { + if (pTableScanInfo->reverseTimes <= 0 || isTsdbCacheLastRow(pTableScanInfo->pQueryHandle)) { return NULL; } else { break; @@ -4591,13 +4663,14 @@ SOperatorInfo* createGlobalAggregateOperatorInfo(SQueryRuntimeEnv* pRuntimeEnv, pOperator->blockingOptr = true; pOperator->status = OP_IN_EXECUTING; pOperator->info = pInfo; - pOperator->upstream = upstream; pOperator->pExpr = pExpr; pOperator->numOfOutput = numOfOutput; pOperator->pRuntimeEnv = pRuntimeEnv; pOperator->exec = doGlobalAggregate; pOperator->cleanup = destroyGlobalAggOperatorInfo; + appendUpstream(pOperator, upstream); + return pOperator; } @@ -4662,7 +4735,7 @@ static SSDataBlock* doAggregate(void* param, bool* newgroup) { SQueryAttr* pQueryAttr = pRuntimeEnv->pQueryAttr; int32_t order = pQueryAttr->order.order; - SOperatorInfo* upstream = pOperator->upstream; + SOperatorInfo* upstream = pOperator->upstream[0]; while(1) { SSDataBlock* pBlock = upstream->exec(upstream, newgroup); @@ -4717,7 +4790,7 @@ static SSDataBlock* doSTableAggregate(void* param, bool* newgroup) { SQueryAttr* pQueryAttr = pRuntimeEnv->pQueryAttr; int32_t order = pQueryAttr->order.order; - SOperatorInfo* upstream = pOperator->upstream; + SOperatorInfo* upstream = pOperator->upstream[0]; while(1) { SSDataBlock* pBlock = upstream->exec(upstream, newgroup); @@ -4801,7 +4874,7 @@ static SSDataBlock* doArithmeticOperation(void* param, bool* newgroup) { bool prevVal = *newgroup; // The upstream exec may change the value of the newgroup, so use a local variable instead. - SSDataBlock* pBlock = pOperator->upstream->exec(pOperator->upstream, newgroup); + SSDataBlock* pBlock = pOperator->upstream[0]->exec(pOperator->upstream[0], newgroup); if (pBlock == NULL) { assert(*newgroup == false); @@ -4835,7 +4908,7 @@ static SSDataBlock* doArithmeticOperation(void* param, bool* newgroup) { } pRes->info.rows = getNumOfResult(pRuntimeEnv, pInfo->pCtx, pOperator->numOfOutput); - if (pRes->info.rows >= pRuntimeEnv->resultInfo.threshold) { + if (pRes->info.rows >= 1000/*pRuntimeEnv->resultInfo.threshold*/) { break; } } @@ -4855,7 +4928,7 @@ static SSDataBlock* doLimit(void* param, bool* newgroup) { SSDataBlock* pBlock = NULL; while (1) { - pBlock = pOperator->upstream->exec(pOperator->upstream, newgroup); + pBlock = pOperator->upstream[0]->exec(pOperator->upstream[0], newgroup); if (pBlock == NULL) { setQueryStatus(pOperator->pRuntimeEnv, QUERY_COMPLETED); pOperator->status = OP_EXEC_DONE; @@ -4895,27 +4968,6 @@ static SSDataBlock* doLimit(void* param, bool* newgroup) { return pBlock; } - -bool doFilterData(SColumnInfoData* p, int32_t rid, SColumnFilterElem *filterElem, __filter_func_t fp) { - char* input = p->pData + p->info.bytes * rid; - bool isnull = isNull(input, p->info.type); - if (isnull) { - return (fp == isNullOperator) ? true : false; - } else { - if (fp == notNullOperator) { - return true; - } else if (fp == isNullOperator) { - return false; - } - } - - if (fp(filterElem, input, input, p->info.type)) { - return true; - } - - return false; -} - static SSDataBlock* doFilter(void* param, bool* newgroup) { SOperatorInfo *pOperator = (SOperatorInfo *)param; if (pOperator->status == OP_EXEC_DONE) { @@ -4926,7 +4978,7 @@ static SSDataBlock* doFilter(void* param, bool* newgroup) { SQueryRuntimeEnv* pRuntimeEnv = pOperator->pRuntimeEnv; while (1) { - SSDataBlock *pBlock = pOperator->upstream->exec(pOperator->upstream, newgroup); + SSDataBlock *pBlock = pOperator->upstream[0]->exec(pOperator->upstream[0], newgroup); if (pBlock == NULL) { break; } @@ -4968,7 +5020,7 @@ static SSDataBlock* doIntervalAgg(void* param, bool* newgroup) { int32_t order = pQueryAttr->order.order; STimeWindow win = pQueryAttr->window; - SOperatorInfo* upstream = pOperator->upstream; + SOperatorInfo* upstream = pOperator->upstream[0]; while(1) { SSDataBlock* pBlock = upstream->exec(upstream, newgroup); @@ -5021,7 +5073,7 @@ static SSDataBlock* doSTableIntervalAgg(void* param, bool* newgroup) { SQueryAttr* pQueryAttr = pRuntimeEnv->pQueryAttr; int32_t order = pQueryAttr->order.order; - SOperatorInfo* upstream = pOperator->upstream; + SOperatorInfo* upstream = pOperator->upstream[0]; while(1) { SSDataBlock* pBlock = upstream->exec(upstream, newgroup); @@ -5076,7 +5128,7 @@ static SSDataBlock* doSessionWindowAgg(void* param, bool* newgroup) { int32_t order = pQueryAttr->order.order; STimeWindow win = pQueryAttr->window; - SOperatorInfo* upstream = pOperator->upstream; + SOperatorInfo* upstream = pOperator->upstream[0]; while(1) { SSDataBlock* pBlock = upstream->exec(upstream, newgroup); @@ -5127,7 +5179,7 @@ static SSDataBlock* hashGroupbyAggregate(void* param, bool* newgroup) { return pInfo->binfo.pRes; } - SOperatorInfo* upstream = pOperator->upstream; + SOperatorInfo* upstream = pOperator->upstream[0]; while(1) { SSDataBlock* pBlock = upstream->exec(upstream, newgroup); @@ -5196,7 +5248,7 @@ static SSDataBlock* doFill(void* param, bool* newgroup) { } while(1) { - SSDataBlock* pBlock = pOperator->upstream->exec(pOperator->upstream, newgroup); + SSDataBlock* pBlock = pOperator->upstream[0]->exec(pOperator->upstream[0], newgroup); if (*newgroup) { assert(pBlock != NULL); } @@ -5272,7 +5324,15 @@ static void destroyOperatorInfo(SOperatorInfo* pOperator) { pOperator->cleanup(pOperator->info, pOperator->numOfOutput); } - destroyOperatorInfo(pOperator->upstream); + if (pOperator->upstream != NULL) { + for(int32_t i = 0; i < pOperator->numOfUpstream; ++i) { + destroyOperatorInfo(pOperator->upstream[i]); + } + + tfree(pOperator->upstream); + pOperator->numOfUpstream = 0; + } + tfree(pOperator->info); tfree(pOperator); } @@ -5297,13 +5357,14 @@ SOperatorInfo* createAggregateOperatorInfo(SQueryRuntimeEnv* pRuntimeEnv, SOpera pOperator->blockingOptr = true; pOperator->status = OP_IN_EXECUTING; pOperator->info = pInfo; - pOperator->upstream = upstream; pOperator->pExpr = pExpr; pOperator->numOfOutput = numOfOutput; pOperator->pRuntimeEnv = pRuntimeEnv; pOperator->exec = doAggregate; pOperator->cleanup = destroyBasicOperatorInfo; + appendUpstream(pOperator, upstream); + return pOperator; } @@ -5370,13 +5431,13 @@ SOperatorInfo* createMultiTableAggOperatorInfo(SQueryRuntimeEnv* pRuntimeEnv, SO pOperator->blockingOptr = true; pOperator->status = OP_IN_EXECUTING; pOperator->info = pInfo; - pOperator->upstream = upstream; pOperator->pExpr = pExpr; pOperator->numOfOutput = numOfOutput; pOperator->pRuntimeEnv = pRuntimeEnv; pOperator->exec = doSTableAggregate; pOperator->cleanup = destroyBasicOperatorInfo; + appendUpstream(pOperator, upstream); return pOperator; } @@ -5400,63 +5461,62 @@ SOperatorInfo* createArithOperatorInfo(SQueryRuntimeEnv* pRuntimeEnv, SOperatorI pOperator->blockingOptr = false; pOperator->status = OP_IN_EXECUTING; pOperator->info = pInfo; - pOperator->upstream = upstream; pOperator->pExpr = pExpr; pOperator->numOfOutput = numOfOutput; pOperator->pRuntimeEnv = pRuntimeEnv; pOperator->exec = doArithmeticOperation; pOperator->cleanup = destroyArithOperatorInfo; + appendUpstream(pOperator, upstream); return pOperator; } -SOperatorInfo* createFilterOperatorInfo(SQueryRuntimeEnv* pRuntimeEnv, SOperatorInfo* upstream, SExprInfo* pExpr, - int32_t numOfOutput) { - SFilterOperatorInfo* pInfo = calloc(1, sizeof(SFilterOperatorInfo)); +SColumnInfo* extractColumnFilterInfo(SExprInfo* pExpr, int32_t numOfOutput, int32_t* numOfFilterCols) { + SColumnInfo* pCols = calloc(numOfOutput, sizeof(SColumnInfo)); - { - SColumnInfo* pCols = calloc(numOfOutput, sizeof(SColumnInfo)); + int32_t numOfFilter = 0; + for(int32_t i = 0; i < numOfOutput; ++i) { + if (pExpr[i].base.flist.numOfFilters > 0) { + numOfFilter += 1; + } - int32_t numOfFilter = 0; - for(int32_t i = 0; i < numOfOutput; ++i) { - if (pExpr[i].base.flist.numOfFilters > 0) { - numOfFilter += 1; - } + pCols[i].type = pExpr[i].base.resType; + pCols[i].bytes = pExpr[i].base.resBytes; + pCols[i].colId = pExpr[i].base.resColId; - pCols[i].type = pExpr[i].base.resType; - pCols[i].bytes = pExpr[i].base.resBytes; - pCols[i].colId = pExpr[i].base.resColId; + pCols[i].flist.numOfFilters = pExpr[i].base.flist.numOfFilters; + pCols[i].flist.filterInfo = calloc(pCols[i].flist.numOfFilters, sizeof(SColumnFilterInfo)); + memcpy(pCols[i].flist.filterInfo, pExpr[i].base.flist.filterInfo, pCols[i].flist.numOfFilters * sizeof(SColumnFilterInfo)); + } - pCols[i].flist.numOfFilters = pExpr[i].base.flist.numOfFilters; - pCols[i].flist.filterInfo = calloc(pCols[i].flist.numOfFilters, sizeof(SColumnFilterInfo)); - memcpy(pCols[i].flist.filterInfo, pExpr[i].base.flist.filterInfo, pCols[i].flist.numOfFilters * sizeof(SColumnFilterInfo)); - } + assert(numOfFilter > 0); - assert(numOfFilter > 0); - doCreateFilterInfo(pCols, numOfOutput, numOfFilter, &pInfo->pFilterInfo, 0); - pInfo->numOfFilterCols = numOfFilter; + *numOfFilterCols = numOfFilter; + return pCols; +} - for(int32_t i = 0; i < numOfOutput; ++i) { - tfree(pCols[i].flist.filterInfo); - } +SOperatorInfo* createFilterOperatorInfo(SQueryRuntimeEnv* pRuntimeEnv, SOperatorInfo* upstream, SExprInfo* pExpr, + int32_t numOfOutput, SColumnInfo* pCols, int32_t numOfFilter) { + SFilterOperatorInfo* pInfo = calloc(1, sizeof(SFilterOperatorInfo)); - tfree(pCols); - } + assert(numOfFilter > 0 && pCols != NULL); + doCreateFilterInfo(pCols, numOfOutput, numOfFilter, &pInfo->pFilterInfo, 0); + pInfo->numOfFilterCols = numOfFilter; SOperatorInfo* pOperator = calloc(1, sizeof(SOperatorInfo)); - pOperator->name = "ConditionOperator"; + pOperator->name = "FilterOperator"; pOperator->operatorType = OP_Filter; pOperator->blockingOptr = false; pOperator->status = OP_IN_EXECUTING; pOperator->numOfOutput = numOfOutput; pOperator->pExpr = pExpr; - pOperator->upstream = upstream; pOperator->exec = doFilter; pOperator->info = pInfo; pOperator->pRuntimeEnv = pRuntimeEnv; pOperator->cleanup = destroyConditionOperatorInfo; + appendUpstream(pOperator, upstream); return pOperator; } @@ -5471,10 +5531,10 @@ SOperatorInfo* createLimitOperatorInfo(SQueryRuntimeEnv* pRuntimeEnv, SOperatorI pOperator->operatorType = OP_Limit; pOperator->blockingOptr = false; pOperator->status = OP_IN_EXECUTING; - pOperator->upstream = upstream; pOperator->exec = doLimit; pOperator->info = pInfo; pOperator->pRuntimeEnv = pRuntimeEnv; + appendUpstream(pOperator, upstream); return pOperator; } @@ -5492,7 +5552,6 @@ SOperatorInfo* createTimeIntervalOperatorInfo(SQueryRuntimeEnv* pRuntimeEnv, SOp pOperator->operatorType = OP_TimeWindow; pOperator->blockingOptr = true; pOperator->status = OP_IN_EXECUTING; - pOperator->upstream = upstream; pOperator->pExpr = pExpr; pOperator->numOfOutput = numOfOutput; pOperator->info = pInfo; @@ -5500,6 +5559,7 @@ SOperatorInfo* createTimeIntervalOperatorInfo(SQueryRuntimeEnv* pRuntimeEnv, SOp pOperator->exec = doIntervalAgg; pOperator->cleanup = destroyBasicOperatorInfo; + appendUpstream(pOperator, upstream); return pOperator; } @@ -5517,7 +5577,6 @@ SOperatorInfo* createSWindowOperatorInfo(SQueryRuntimeEnv* pRuntimeEnv, SOperato pOperator->operatorType = OP_SessionWindow; pOperator->blockingOptr = true; pOperator->status = OP_IN_EXECUTING; - pOperator->upstream = upstream; pOperator->pExpr = pExpr; pOperator->numOfOutput = numOfOutput; pOperator->info = pInfo; @@ -5525,6 +5584,7 @@ SOperatorInfo* createSWindowOperatorInfo(SQueryRuntimeEnv* pRuntimeEnv, SOperato pOperator->exec = doSessionWindowAgg; pOperator->cleanup = destroyBasicOperatorInfo; + appendUpstream(pOperator, upstream); return pOperator; } @@ -5540,7 +5600,6 @@ SOperatorInfo* createMultiTableTimeIntervalOperatorInfo(SQueryRuntimeEnv* pRunti pOperator->operatorType = OP_MultiTableTimeInterval; pOperator->blockingOptr = true; pOperator->status = OP_IN_EXECUTING; - pOperator->upstream = upstream; pOperator->pExpr = pExpr; pOperator->numOfOutput = numOfOutput; pOperator->info = pInfo; @@ -5549,6 +5608,7 @@ SOperatorInfo* createMultiTableTimeIntervalOperatorInfo(SQueryRuntimeEnv* pRunti pOperator->exec = doSTableIntervalAgg; pOperator->cleanup = destroyBasicOperatorInfo; + appendUpstream(pOperator, upstream); return pOperator; } @@ -5565,7 +5625,6 @@ SOperatorInfo* createGroupbyOperatorInfo(SQueryRuntimeEnv* pRuntimeEnv, SOperato pOperator->blockingOptr = true; pOperator->status = OP_IN_EXECUTING; pOperator->operatorType = OP_Groupby; - pOperator->upstream = upstream; pOperator->pExpr = pExpr; pOperator->numOfOutput = numOfOutput; pOperator->info = pInfo; @@ -5573,6 +5632,7 @@ SOperatorInfo* createGroupbyOperatorInfo(SQueryRuntimeEnv* pRuntimeEnv, SOperato pOperator->exec = hashGroupbyAggregate; pOperator->cleanup = destroyGroupbyOperatorInfo; + appendUpstream(pOperator, upstream); return pOperator; } @@ -5602,8 +5662,6 @@ SOperatorInfo* createFillOperatorInfo(SQueryRuntimeEnv* pRuntimeEnv, SOperatorIn pOperator->blockingOptr = false; pOperator->status = OP_IN_EXECUTING; pOperator->operatorType = OP_Fill; - - pOperator->upstream = upstream; pOperator->pExpr = pExpr; pOperator->numOfOutput = numOfOutput; pOperator->info = pInfo; @@ -5612,6 +5670,7 @@ SOperatorInfo* createFillOperatorInfo(SQueryRuntimeEnv* pRuntimeEnv, SOperatorIn pOperator->exec = doFill; pOperator->cleanup = destroySFillOperatorInfo; + appendUpstream(pOperator, upstream); return pOperator; } @@ -5650,11 +5709,12 @@ SOperatorInfo* createSLimitOperatorInfo(SQueryRuntimeEnv* pRuntimeEnv, SOperator pOperator->operatorType = OP_SLimit; pOperator->blockingOptr = false; pOperator->status = OP_IN_EXECUTING; - pOperator->upstream = upstream; pOperator->exec = doSLimit; pOperator->info = pInfo; pOperator->pRuntimeEnv = pRuntimeEnv; pOperator->cleanup = destroySlimitOperatorInfo; + + appendUpstream(pOperator, upstream); return pOperator; } @@ -5821,7 +5881,7 @@ static SSDataBlock* hashDistinct(void* param, bool* newgroup) { pRes->info.rows = 0; SSDataBlock* pBlock = NULL; while(1) { - pBlock = pOperator->upstream->exec(pOperator->upstream, newgroup); + pBlock = pOperator->upstream[0]->exec(pOperator->upstream[0], newgroup); if (pBlock == NULL) { setQueryStatus(pOperator->pRuntimeEnv, QUERY_COMPLETED); pOperator->status = OP_EXEC_DONE; @@ -5882,12 +5942,13 @@ SOperatorInfo* createDistinctOperatorInfo(SQueryRuntimeEnv* pRuntimeEnv, SOperat pOperator->blockingOptr = false; pOperator->status = OP_IN_EXECUTING; pOperator->operatorType = OP_Distinct; - pOperator->upstream = upstream; pOperator->numOfOutput = numOfOutput; pOperator->info = pInfo; pOperator->pRuntimeEnv = pRuntimeEnv; pOperator->exec = hashDistinct; pOperator->cleanup = destroyDistinctOperatorInfo; + + appendUpstream(pOperator, upstream); return pOperator; } @@ -6434,7 +6495,7 @@ static int32_t updateOutputBufForTopBotQuery(SQueriedTableInfo* pTableInfo, SCol return TSDB_CODE_SUCCESS; } -// TODO tag length should be passed from client +// TODO tag length should be passed from client, refactor int32_t createQueryFunc(SQueriedTableInfo* pTableInfo, int32_t numOfOutput, SExprInfo** pExprInfo, SSqlExpr** pExprMsg, SColumnInfo* pTagCols, int32_t queryType, void* pMsg) { *pExprInfo = NULL; @@ -6605,13 +6666,13 @@ int32_t createIndirectQueryFuncExprFromMsg(SQueryTableMsg* pQueryMsg, int32_t nu return TSDB_CODE_SUCCESS; } -SSqlGroupbyExpr *createGroupbyExprFromMsg(SQueryTableMsg *pQueryMsg, SColIndex *pColIndex, int32_t *code) { +SGroupbyExpr *createGroupbyExprFromMsg(SQueryTableMsg *pQueryMsg, SColIndex *pColIndex, int32_t *code) { if (pQueryMsg->numOfGroupCols == 0) { return NULL; } // using group by tag columns - SSqlGroupbyExpr *pGroupbyExpr = (SSqlGroupbyExpr *)calloc(1, sizeof(SSqlGroupbyExpr)); + SGroupbyExpr *pGroupbyExpr = (SGroupbyExpr *)calloc(1, sizeof(SGroupbyExpr)); if (pGroupbyExpr == NULL) { *code = TSDB_CODE_QRY_OUT_OF_MEMORY; return NULL; @@ -6629,8 +6690,7 @@ SSqlGroupbyExpr *createGroupbyExprFromMsg(SQueryTableMsg *pQueryMsg, SColIndex * return pGroupbyExpr; } -static int32_t doCreateFilterInfo(SColumnInfo* pCols, int32_t numOfCols, int32_t numOfFilterCols, - SSingleColumnFilterInfo** pFilterInfo, uint64_t qId) { +int32_t doCreateFilterInfo(SColumnInfo* pCols, int32_t numOfCols, int32_t numOfFilterCols, SSingleColumnFilterInfo** pFilterInfo, uint64_t qId) { *pFilterInfo = calloc(1, sizeof(SSingleColumnFilterInfo) * numOfFilterCols); if (pFilterInfo == NULL) { return TSDB_CODE_QRY_OUT_OF_MEMORY; @@ -6687,7 +6747,7 @@ void* doDestroyFilterInfo(SSingleColumnFilterInfo* pFilterInfo, int32_t numOfFil return NULL; } -static int32_t createFilterInfo(SQueryAttr* pQueryAttr, uint64_t qId) { +int32_t createFilterInfo(SQueryAttr* pQueryAttr, uint64_t qId) { for (int32_t i = 0; i < pQueryAttr->numOfCols; ++i) { if (pQueryAttr->tableCols[i].flist.numOfFilters > 0) { pQueryAttr->numOfFilterCols++; @@ -6772,7 +6832,7 @@ FORCE_INLINE bool checkQIdEqual(void *qHandle, uint64_t qId) { return ((SQInfo *)qHandle)->qId == qId; } -SQInfo* createQInfoImpl(SQueryTableMsg* pQueryMsg, SSqlGroupbyExpr* pGroupbyExpr, SExprInfo* pExprs, +SQInfo* createQInfoImpl(SQueryTableMsg* pQueryMsg, SGroupbyExpr* pGroupbyExpr, SExprInfo* pExprs, SExprInfo* pSecExprs, STableGroupInfo* pTableGroupInfo, SColumnInfo* pTagCols, int32_t vgId, char* sql, uint64_t *qId) { int16_t numOfCols = pQueryMsg->numOfCols; @@ -7075,7 +7135,7 @@ static void doDestroyTableQueryInfo(STableGroupInfo* pTableqinfoGroupInfo) { pTableqinfoGroupInfo->numOfTables = 0; } -static void* destroyQueryFuncExpr(SExprInfo* pExprInfo, int32_t numOfExpr) { +void* destroyQueryFuncExpr(SExprInfo* pExprInfo, int32_t numOfExpr) { if (pExprInfo == NULL) { assert(numOfExpr == 0); return NULL; @@ -7099,6 +7159,20 @@ static void* destroyQueryFuncExpr(SExprInfo* pExprInfo, int32_t numOfExpr) { return NULL; } +void* freeColumnInfo(SColumnInfo* pColumnInfo, int32_t numOfCols) { + if (pColumnInfo != NULL) { + assert(numOfCols >= 0); + + for (int32_t i = 0; i < numOfCols; i++) { + freeColumnFilterInfo(pColumnInfo[i].flist.filterInfo, pColumnInfo[i].flist.numOfFilters); + } + + tfree(pColumnInfo); + } + + return NULL; +} + void freeQInfo(SQInfo *pQInfo) { if (!isValidQInfo(pQInfo)) { return; @@ -7283,13 +7357,7 @@ void freeQueryAttr(SQueryAttr* pQueryAttr) { tfree(pQueryAttr->tagColList); tfree(pQueryAttr->pFilterInfo); - if (pQueryAttr->tableCols != NULL) { - for (int32_t i = 0; i < pQueryAttr->numOfCols; i++) { - SColumnInfo* column = pQueryAttr->tableCols + i; - freeColumnFilterInfo(column->flist.filterInfo, column->flist.numOfFilters); - } - tfree(pQueryAttr->tableCols); - } + pQueryAttr->tableCols = freeColumnInfo(pQueryAttr->tableCols, pQueryAttr->numOfCols); if (pQueryAttr->pGroupbyExpr != NULL) { taosArrayDestroy(pQueryAttr->pGroupbyExpr->columnInfo); diff --git a/src/query/src/qFill.c b/src/query/src/qFill.c index fa572029fc043fc13b9822f1e688696ca9a0a225..7dd73c9fe48be39ee3a7a879348076ac3fbe9f44 100644 --- a/src/query/src/qFill.c +++ b/src/query/src/qFill.c @@ -363,10 +363,6 @@ SFillInfo* taosCreateFillInfo(int32_t order, TSKEY skey, int32_t numOfTags, int3 pFillInfo->rowSize = setTagColumnInfo(pFillInfo, pFillInfo->numOfCols, pFillInfo->alloc); assert(pFillInfo->rowSize > 0); - for(int32_t i = 0; i < pFillInfo->numOfCols; ++i) { - pFillInfo->pData[i] = malloc(pFillInfo->pFillCol[i].col.bytes * pFillInfo->alloc); - } - return pFillInfo; } @@ -392,10 +388,6 @@ void* taosDestroyFillInfo(SFillInfo* pFillInfo) { tfree(pFillInfo->pTags[i].tagVal); } - for(int32_t i = 0; i < pFillInfo->numOfCols; ++i) { - tfree(pFillInfo->pData[i]); - } - tfree(pFillInfo->pTags); tfree(pFillInfo->pData); @@ -417,17 +409,6 @@ void taosFillSetStartInfo(SFillInfo* pFillInfo, int32_t numOfRows, TSKEY endKey) pFillInfo->index = 0; pFillInfo->numOfRows = numOfRows; - - // ensure the space - if (pFillInfo->alloc < numOfRows) { - for(int32_t i = 0; i < pFillInfo->numOfCols; ++i) { - char* tmp = realloc(pFillInfo->pData[i], numOfRows*pFillInfo->pFillCol[i].col.bytes); - assert(tmp != NULL); // todo handle error - - memset(tmp, 0, numOfRows*pFillInfo->pFillCol[i].col.bytes); - pFillInfo->pData[i] = tmp; - } - } } void taosFillSetInputDataBlock(SFillInfo* pFillInfo, const SSDataBlock* pInput) { @@ -435,16 +416,7 @@ void taosFillSetInputDataBlock(SFillInfo* pFillInfo, const SSDataBlock* pInput) SFillColInfo* pCol = &pFillInfo->pFillCol[i]; SColumnInfoData* pColData = taosArrayGet(pInput->pDataBlock, i); -// pFillInfo->pData[i] = pColData->pData; - if (pInput->info.rows > pFillInfo->alloc) { - char* t = realloc(pFillInfo->pData[i], pColData->info.bytes * pInput->info.rows); - assert(t != NULL); - - pFillInfo->pData[i] = t; - pFillInfo->alloc = pInput->info.rows; - } - - memcpy(pFillInfo->pData[i], pColData->pData, pColData->info.bytes * pInput->info.rows); + pFillInfo->pData[i] = pColData->pData; if (TSDB_COL_IS_TAG(pCol->flag)/* || IS_VAR_DATA_TYPE(pCol->col.type)*/) { // copy the tag value to tag value buffer SFillTagColInfo* pTag = &pFillInfo->pTags[pCol->tagIndex]; @@ -454,31 +426,6 @@ void taosFillSetInputDataBlock(SFillInfo* pFillInfo, const SSDataBlock* pInput) } } -void taosFillCopyInputDataFromOneFilePage(SFillInfo* pFillInfo, const tFilePage* pInput) { - assert(pFillInfo->numOfRows == pInput->num); - - for(int32_t i = 0; i < pFillInfo->numOfCols; ++i) { - SFillColInfo* pCol = &pFillInfo->pFillCol[i]; - - const char* data = pInput->data + pCol->col.offset * pInput->num; - if (pInput->num > pFillInfo->alloc) { - char* t = realloc(pFillInfo->pData[i], (size_t)(pCol->col.bytes * pInput->num)); - assert(t != NULL); - - pFillInfo->pData[i] = t; - pFillInfo->alloc = (int32_t)pInput->num; - } - - memcpy(pFillInfo->pData[i], data, (size_t)(pCol->col.bytes * pInput->num)); - - if (TSDB_COL_IS_TAG(pCol->flag)/* || IS_VAR_DATA_TYPE(pCol->col.type)*/) { // copy the tag value to tag value buffer - SFillTagColInfo* pTag = &pFillInfo->pTags[pCol->tagIndex]; - assert (pTag->col.colId == pCol->col.colId); - memcpy(pTag->tagVal, data, pCol->col.bytes); // TODO not memcpy?? - } - } -} - bool taosFillHasMoreResults(SFillInfo* pFillInfo) { int32_t remain = taosNumOfRemainRows(pFillInfo); if (remain > 0) { diff --git a/src/query/src/qPlan.c b/src/query/src/qPlan.c index 0554a887ec68ca3353b5e45c1067ea734ea98cfd..9079d830c9a928f1f27a02fa94b763d1473aa221 100644 --- a/src/query/src/qPlan.c +++ b/src/query/src/qPlan.c @@ -1,44 +1,523 @@ #include "os.h" -#include "tsclient.h" +#include "tschemautil.h" +#include "qPlan.h" +#include "qExecutor.h" #include "qUtil.h" #include "texpr.h" +#include "tscUtil.h" +#include "tsclient.h" + +#define QNODE_TAGSCAN 1 +#define QNODE_TABLESCAN 2 +#define QNODE_PROJECT 3 +#define QNODE_AGGREGATE 4 +#define QNODE_GROUPBY 5 +#define QNODE_LIMIT 6 +#define QNODE_JOIN 7 +#define QNODE_DISTINCT 8 +#define QNODE_SORT 9 +#define QNODE_UNIONALL 10 +#define QNODE_TIMEWINDOW 11 +#define QNODE_SESSIONWINDOW 12 +#define QNODE_FILL 13 + +typedef struct SFillEssInfo { + int32_t fillType; // fill type + int64_t *val; // fill value +} SFillEssInfo; + +typedef struct SJoinCond { + bool tagExists; // denote if tag condition exists or not + SColumn *tagCond[2]; + SColumn *colCond[2]; +} SJoinCond; + +static SQueryNode* createQueryNode(int32_t type, const char* name, SQueryNode** prev, + int32_t numOfPrev, SExprInfo** pExpr, int32_t numOfOutput, SQueryTableInfo* pTableInfo, + void* pExtInfo) { + SQueryNode* pNode = calloc(1, sizeof(SQueryNode)); + + pNode->info.type = type; + pNode->info.name = strdup(name); + + if (pTableInfo->id.uid != 0) { // it is a true table + pNode->tableInfo.id = pTableInfo->id; + pNode->tableInfo.tableName = strdup(pTableInfo->tableName); + } + + pNode->numOfOutput = numOfOutput; + pNode->pExpr = calloc(numOfOutput, sizeof(SExprInfo)); + for(int32_t i = 0; i < numOfOutput; ++i) { + tscExprAssign(&pNode->pExpr[i], pExpr[i]); + } + + pNode->pPrevNodes = taosArrayInit(4, POINTER_BYTES); + for(int32_t i = 0; i < numOfPrev; ++i) { + taosArrayPush(pNode->pPrevNodes, &prev[i]); + } + + switch(type) { + case QNODE_TABLESCAN: { + STimeWindow* window = calloc(1, sizeof(STimeWindow)); + memcpy(window, pExtInfo, sizeof(STimeWindow)); + pNode->pExtInfo = window; + break; + } + + case QNODE_TIMEWINDOW: { + SInterval* pInterval = calloc(1, sizeof(SInterval)); + pNode->pExtInfo = pInterval; + memcpy(pInterval, pExtInfo, sizeof(SInterval)); + break; + } + + case QNODE_GROUPBY: { + SGroupbyExpr* p = (SGroupbyExpr*) pExtInfo; + SGroupbyExpr* pGroupbyExpr = calloc(1, sizeof(SGroupbyExpr)); + + pGroupbyExpr->tableIndex = p->tableIndex; + pGroupbyExpr->orderType = p->orderType; + pGroupbyExpr->orderIndex = p->orderIndex; + pGroupbyExpr->numOfGroupCols = p->numOfGroupCols; + pGroupbyExpr->columnInfo = taosArrayDup(p->columnInfo); + pNode->pExtInfo = pGroupbyExpr; + break; + } + + case QNODE_FILL: { // todo !! + pNode->pExtInfo = pExtInfo; + break; + } + + case QNODE_LIMIT: { + pNode->pExtInfo = calloc(1, sizeof(SLimitVal)); + memcpy(pNode->pExtInfo, pExtInfo, sizeof(SLimitVal)); + break; + } + } + return pNode; +} + +static SQueryNode* doAddTableColumnNode(SQueryInfo* pQueryInfo, STableMetaInfo* pTableMetaInfo, SQueryTableInfo* info, + SArray* pExprs, SArray* tableCols) { + if (pQueryInfo->onlyTagQuery) { + int32_t num = (int32_t) taosArrayGetSize(pExprs); + SQueryNode* pNode = createQueryNode(QNODE_TAGSCAN, "TableTagScan", NULL, 0, pExprs->pData, num, info, NULL); + + if (pQueryInfo->distinctTag) { + pNode = createQueryNode(QNODE_DISTINCT, "Distinct", &pNode, 1, pExprs->pData, num, info, NULL); + } + + return pNode; + } + + STimeWindow* window = &pQueryInfo->window; + SQueryNode* pNode = createQueryNode(QNODE_TABLESCAN, "TableScan", NULL, 0, NULL, 0, + info, window); + if (pQueryInfo->projectionQuery) { + int32_t numOfOutput = (int32_t) taosArrayGetSize(pExprs); + pNode = createQueryNode(QNODE_PROJECT, "Projection", &pNode, 1, pExprs->pData, numOfOutput, info, NULL); + } else { + // table source column projection, generate the projection expr + int32_t numOfCols = (int32_t) taosArrayGetSize(tableCols); + SExprInfo** pExpr = calloc(numOfCols, POINTER_BYTES); + SSchema* pSchema = pTableMetaInfo->pTableMeta->schema; + + for (int32_t i = 0; i < numOfCols; ++i) { + SColumn* pCol = taosArrayGetP(tableCols, i); + + SColumnIndex index = {.tableIndex = 0, .columnIndex = pCol->columnIndex}; + SExprInfo* p = tscExprCreate(pQueryInfo, TSDB_FUNC_PRJ, &index, pCol->info.type, pCol->info.bytes, + pCol->info.colId, 0, TSDB_COL_NORMAL); + strncpy(p->base.aliasName, pSchema[pCol->columnIndex].name, tListLen(p->base.aliasName)); + + pExpr[i] = p; + } + + pNode = createQueryNode(QNODE_PROJECT, "Projection", &pNode, 1, pExpr, numOfCols, info, NULL); + for (int32_t i = 0; i < numOfCols; ++i) { + destroyQueryFuncExpr(pExpr[i], 1); + } + tfree(pExpr); + } + + return pNode; +} + +static SQueryNode* doCreateQueryPlanForOneTableImpl(SQueryInfo* pQueryInfo, SQueryNode* pNode, SQueryTableInfo* info, + SArray* pExprs) { + // check for aggregation + if (pQueryInfo->interval.interval > 0) { + int32_t numOfOutput = (int32_t) taosArrayGetSize(pExprs); + + pNode = createQueryNode(QNODE_TIMEWINDOW, "TimeWindowAgg", &pNode, 1, pExprs->pData, numOfOutput, info, + &pQueryInfo->interval); + } else if (pQueryInfo->groupbyColumn) { + int32_t numOfOutput = (int32_t) taosArrayGetSize(pExprs); + pNode = createQueryNode(QNODE_GROUPBY, "Groupby", &pNode, 1, pExprs->pData, numOfOutput, info, + &pQueryInfo->groupbyExpr); + } else if (pQueryInfo->sessionWindow.gap > 0) { + pNode = createQueryNode(QNODE_SESSIONWINDOW, "SessionWindowAgg", &pNode, 1, NULL, 0, info, NULL); + } else if (pQueryInfo->simpleAgg) { + int32_t numOfOutput = (int32_t) taosArrayGetSize(pExprs); + pNode = createQueryNode(QNODE_AGGREGATE, "Aggregate", &pNode, 1, pExprs->pData, numOfOutput, info, NULL); + } + + if (pQueryInfo->havingFieldNum > 0 || pQueryInfo->arithmeticOnAgg) { + int32_t numOfExpr = (int32_t) taosArrayGetSize(pQueryInfo->exprList1); + pNode = + createQueryNode(QNODE_PROJECT, "Projection", &pNode, 1, pQueryInfo->exprList1->pData, numOfExpr, info, NULL); + } + + if (pQueryInfo->fillType != TSDB_FILL_NONE) { + SFillEssInfo* pInfo = calloc(1, sizeof(SFillEssInfo)); + pInfo->fillType = pQueryInfo->fillType; + pInfo->val = calloc(pNode->numOfOutput, sizeof(int64_t)); + memcpy(pInfo->val, pQueryInfo->fillVal, pNode->numOfOutput); + + pNode = createQueryNode(QNODE_FILL, "Fill", &pNode, 1, NULL, 0, info, pInfo); + } + + + if (pQueryInfo->limit.limit != -1 || pQueryInfo->limit.offset != 0) { + pNode = createQueryNode(QNODE_LIMIT, "Limit", &pNode, 1, NULL, 0, info, &pQueryInfo->limit); + } + + return pNode; +} + +static SQueryNode* doCreateQueryPlanForOneTable(SQueryInfo* pQueryInfo, STableMetaInfo* pTableMetaInfo, SArray* pExprs, + SArray* tableCols) { + char name[TSDB_TABLE_FNAME_LEN] = {0}; + tNameExtractFullName(&pTableMetaInfo->name, name); + + SQueryTableInfo info = {.tableName = strdup(name), .id = pTableMetaInfo->pTableMeta->id,}; + + // handle the only tag query + SQueryNode* pNode = doAddTableColumnNode(pQueryInfo, pTableMetaInfo, &info, pExprs, tableCols); + if (pQueryInfo->onlyTagQuery) { + tfree(info.tableName); + return pNode; + } + + SQueryNode* pNode1 = doCreateQueryPlanForOneTableImpl(pQueryInfo, pNode, &info, pExprs); + tfree(info.tableName); + return pNode1; +} + +SArray* createQueryPlanImpl(SQueryInfo* pQueryInfo) { + SArray* upstream = NULL; + + if (pQueryInfo->pUpstream != NULL && taosArrayGetSize(pQueryInfo->pUpstream) > 0) { // subquery in the from clause + upstream = taosArrayInit(4, POINTER_BYTES); + + size_t size = taosArrayGetSize(pQueryInfo->pUpstream); + for(int32_t i = 0; i < size; ++i) { + SQueryInfo* pq = taosArrayGet(pQueryInfo->pUpstream, i); + SArray* p = createQueryPlanImpl(pq); + taosArrayAddBatch(upstream, p->pData, (int32_t) taosArrayGetSize(p)); + } + } + + if (pQueryInfo->numOfTables > 1) { // it is a join query + // 1. separate the select clause according to table + upstream = taosArrayInit(5, POINTER_BYTES); + + for(int32_t i = 0; i < pQueryInfo->numOfTables; ++i) { + STableMetaInfo* pTableMetaInfo = pQueryInfo->pTableMetaInfo[i]; + uint64_t uid = pTableMetaInfo->pTableMeta->id.uid; + + SArray* exprList = taosArrayInit(4, POINTER_BYTES); + if (tscExprCopy(exprList, pQueryInfo->exprList, uid, true) != 0) { + terrno = TSDB_CODE_TSC_OUT_OF_MEMORY; + exit(-1); + } + + // 2. create the query execution node + char name[TSDB_TABLE_FNAME_LEN] = {0}; + tNameExtractFullName(&pTableMetaInfo->name, name); + SQueryTableInfo info = {.tableName = strdup(name), .id = pTableMetaInfo->pTableMeta->id,}; + + // 3. get the required table column list + SArray* tableColumnList = taosArrayInit(4, sizeof(SColumn)); + tscColumnListCopy(tableColumnList, pQueryInfo->colList, uid); + + // 4. add the projection query node + SQueryNode* pNode = doAddTableColumnNode(pQueryInfo, pTableMetaInfo, &info, exprList, tableColumnList); + taosArrayPush(upstream, &pNode); + } + + // 3. add the join node here + SQueryTableInfo info = {0}; + int32_t num = (int32_t) taosArrayGetSize(pQueryInfo->exprList); + SQueryNode* pNode = createQueryNode(QNODE_JOIN, "Join", upstream->pData, pQueryInfo->numOfTables, + pQueryInfo->exprList->pData, num, &info, NULL); + + // 4. add the aggregation or projection execution node + pNode = doCreateQueryPlanForOneTableImpl(pQueryInfo, pNode, &info, pQueryInfo->exprList); + upstream = taosArrayInit(5, POINTER_BYTES); + taosArrayPush(upstream, &pNode); + } else { // only one table, normal query process + STableMetaInfo* pTableMetaInfo = pQueryInfo->pTableMetaInfo[0]; + SQueryNode* pNode = doCreateQueryPlanForOneTable(pQueryInfo, pTableMetaInfo, pQueryInfo->exprList, pQueryInfo->colList); + upstream = taosArrayInit(5, POINTER_BYTES); + taosArrayPush(upstream, &pNode); + } + + return upstream; +} -#define QNODE_PROJECT 1 -#define QNODE_FILTER 2 -#define QNODE_RELATION 3 -#define QNODE_AGGREGATE 4 -#define QNODE_GROUPBY 5 -#define QNODE_LIMIT 6 -#define QNODE_JOIN 7 -#define QNODE_DIST 8 -#define QNODE_SORT 9 -#define QNODE_UNIONALL 10 -#define QNODE_TIMEWINDOW 11 - -typedef struct SQueryNode { - int32_t type; // the type of logic node - char *name; // the name of logic node - - SSchema *pSchema; // the schema of the input SSDatablock - int32_t numOfCols; // number of input columns - SExprInfo *pExpr; // the query functions or sql aggregations - int32_t numOfOutput; // number of result columns, which is also the number of pExprs - - // previous operator to generated result for current node to process - // in case of join, multiple prev nodes exist. - struct SQueryNode* prevNode; - struct SQueryNode* nextNode; -} SQueryNode; - -// TODO create the query plan SQueryNode* qCreateQueryPlan(SQueryInfo* pQueryInfo) { - return NULL; + SArray* upstream = createQueryPlanImpl(pQueryInfo); + assert(taosArrayGetSize(upstream) == 1); + + SQueryNode* p = taosArrayGetP(upstream, 0); + taosArrayDestroy(upstream); + + return p; } -char* queryPlanToString() { +static void doDestroyQueryNode(SQueryNode* pQueryNode) { + tfree(pQueryNode->pExtInfo); + tfree(pQueryNode->pSchema); + tfree(pQueryNode->info.name); + + tfree(pQueryNode->tableInfo.tableName); + + pQueryNode->pExpr = destroyQueryFuncExpr(pQueryNode->pExpr, pQueryNode->numOfOutput); + + if (pQueryNode->pPrevNodes != NULL) { + int32_t size = (int32_t) taosArrayGetSize(pQueryNode->pPrevNodes); + for(int32_t i = 0; i < size; ++i) { + SQueryNode* p = taosArrayGetP(pQueryNode->pPrevNodes, i); + doDestroyQueryNode(p); + } + + taosArrayDestroy(pQueryNode->pPrevNodes); + } + + tfree(pQueryNode); +} + +void* qDestroyQueryPlan(SQueryNode* pQueryNode) { + if (pQueryNode == NULL) { + return NULL; + } + + doDestroyQueryNode(pQueryNode); return NULL; } +bool hasAliasName(SExprInfo* pExpr) { + assert(pExpr != NULL); + return strncmp(pExpr->base.token, pExpr->base.aliasName, tListLen(pExpr->base.aliasName)) != 0; +} + +static int32_t doPrintPlan(char* buf, SQueryNode* pQueryNode, int32_t level, int32_t totalLen) { + if (level > 0) { + sprintf(buf + totalLen, "%*c", level, ' '); + totalLen += level; + } + + int32_t len1 = sprintf(buf + totalLen, "%s(", pQueryNode->info.name); + int32_t len = len1 + totalLen; + + switch(pQueryNode->info.type) { + case QNODE_TABLESCAN: { + STimeWindow* win = (STimeWindow*)pQueryNode->pExtInfo; + len1 = sprintf(buf + len, "%s #0x%" PRIx64 ") time_range: %" PRId64 " - %" PRId64 "\n", + pQueryNode->tableInfo.tableName, pQueryNode->tableInfo.id.uid, win->skey, win->ekey); + len += len1; + break; + } + + case QNODE_PROJECT: { + len1 = sprintf(buf + len, "cols: "); + len += len1; + + for(int32_t i = 0; i < pQueryNode->numOfOutput; ++i) { + SSqlExpr* p = &pQueryNode->pExpr[i].base; + len1 = sprintf(buf + len, "[%s #%d]", p->aliasName, p->resColId); + len += len1; + + if (i < pQueryNode->numOfOutput - 1) { + len1 = sprintf(buf + len, ", "); + len += len1; + } + } + + len1 = sprintf(buf + len, ")"); + len += len1; + + //todo print filter info + len1 = sprintf(buf + len, " filters:(nil)\n"); + len += len1; + break; + } + + case QNODE_AGGREGATE: { + for(int32_t i = 0; i < pQueryNode->numOfOutput; ++i) { + SSqlExpr* pExpr = &pQueryNode->pExpr[i].base; + if (hasAliasName(&pQueryNode->pExpr[i])) { + len1 = sprintf(buf + len,"[%s #%s]", pExpr->token, pExpr->aliasName); + } else { + len1 = sprintf(buf + len,"[%s]", pExpr->token); + } + + len += len1; + if (i < pQueryNode->numOfOutput - 1) { + len1 = sprintf(buf + len, ", "); + len += len1; + } + } + + len1 = sprintf(buf + len, ")\n"); + len += len1; + break; + } + + case QNODE_TIMEWINDOW: { + for(int32_t i = 0; i < pQueryNode->numOfOutput; ++i) { + SSqlExpr* pExpr = &pQueryNode->pExpr[i].base; + if (hasAliasName(&pQueryNode->pExpr[i])) { + len1 = sprintf(buf + len,"[%s #%s]", pExpr->token, pExpr->aliasName); + } else { + len1 = sprintf(buf + len,"[%s]", pExpr->token); + } + + len += len1; + if (i < pQueryNode->numOfOutput - 1) { + len1 = sprintf(buf + len,", "); + len += len1; + } + } + + len1 = sprintf(buf + len,") "); + len += len1; + + SInterval* pInterval = pQueryNode->pExtInfo; + len1 = sprintf(buf + len, "interval:%" PRId64 "(%c), sliding:%" PRId64 "(%c), offset:%" PRId64 "\n", + pInterval->interval, pInterval->intervalUnit, pInterval->sliding, pInterval->slidingUnit, + pInterval->offset); + len += len1; + + break; + } + + case QNODE_GROUPBY: { // todo hide the invisible column + for(int32_t i = 0; i < pQueryNode->numOfOutput; ++i) { + SSqlExpr* pExpr = &pQueryNode->pExpr[i].base; + + if (hasAliasName(&pQueryNode->pExpr[i])) { + len1 = sprintf(buf + len,"[%s #%s]", pExpr->token, pExpr->aliasName); + } else { + len1 = sprintf(buf + len,"[%s]", pExpr->token); + } + + len += len1; + if (i < pQueryNode->numOfOutput - 1) { + len1 = sprintf(buf + len,", "); + len += len1; + } + } + + SGroupbyExpr* pGroupbyExpr = pQueryNode->pExtInfo; + SColIndex* pIndex = taosArrayGet(pGroupbyExpr->columnInfo, 0); + + len1 = sprintf(buf + len,") groupby_col: [%s #%d]\n", pIndex->name, pIndex->colId); + len += len1; + + break; + } + + case QNODE_FILL: { + SFillEssInfo* pEssInfo = pQueryNode->pExtInfo; + len1 = sprintf(buf + len,"%d", pEssInfo->fillType); + len += len1; + + if (pEssInfo->fillType == TSDB_FILL_SET_VALUE) { + len1 = sprintf(buf + len,", val:"); + len += len1; + + // todo get the correct fill data type + for(int32_t i = 0; i < pQueryNode->numOfOutput; ++i) { + len1 = sprintf(buf + len,"%"PRId64, pEssInfo->val[i]); + len += len1; + + if (i < pQueryNode->numOfOutput - 1) { + len1 = sprintf(buf + len,", "); + len += len1; + } + } + } + + len1 = sprintf(buf + len,")\n"); + len += len1; + break; + } + + case QNODE_LIMIT: { + SLimitVal* pVal = pQueryNode->pExtInfo; + len1 = sprintf(buf + len,"limit: %"PRId64", offset: %"PRId64")\n", pVal->limit, pVal->offset); + len += len1; + break; + } + + case QNODE_DISTINCT: + case QNODE_TAGSCAN: { + len1 = sprintf(buf + len,"cols: "); + len += len1; + + for(int32_t i = 0; i < pQueryNode->numOfOutput; ++i) { + SSqlExpr* p = &pQueryNode->pExpr[i].base; + len1 = sprintf(buf + len,"[%s #%d]", p->aliasName, p->resColId); + len += len1; + + if (i < pQueryNode->numOfOutput - 1) { + len1 = sprintf(buf + len,", "); + len += len1; + } + } + + len1 = sprintf(buf + len,")\n"); + len += len1; + + break; + } + + case QNODE_JOIN: { + // print join condition + len1 = sprintf(buf + len, ")\n"); + len += len1; + break; + } + } + + return len; +} + +int32_t queryPlanToStringImpl(char* buf, SQueryNode* pQueryNode, int32_t level, int32_t totalLen) { + int32_t len = doPrintPlan(buf, pQueryNode, level, totalLen); + + for(int32_t i = 0; i < taosArrayGetSize(pQueryNode->pPrevNodes); ++i) { + SQueryNode* p1 = taosArrayGetP(pQueryNode->pPrevNodes, i); + int32_t len1 = queryPlanToStringImpl(buf, p1, level + 1, len); + len = len1; + } + + return len; +} + +char* queryPlanToString(SQueryNode* pQueryNode) { + assert(pQueryNode); + + char* buf = calloc(1, 4096); + + int32_t len = sprintf(buf, "===== logic plan =====\n"); + queryPlanToStringImpl(buf, pQueryNode, 0, len); + return buf; +} + SQueryNode* queryPlanFromString() { return NULL; } @@ -136,8 +615,13 @@ SArray* createExecOperatorPlan(SQueryAttr* pQueryAttr) { taosArrayPush(plan, &op); } } else { // diff/add/multiply/subtract/division - op = OP_Arithmetic; - taosArrayPush(plan, &op); + if (pQueryAttr->numOfFilterCols > 0 && pQueryAttr->vgId == 0) { // todo refactor + op = OP_Filter; + taosArrayPush(plan, &op); + } else { + op = OP_Arithmetic; + taosArrayPush(plan, &op); + } } if (pQueryAttr->limit.limit > 0 || pQueryAttr->limit.offset > 0) { diff --git a/src/query/src/qSqlParser.c b/src/query/src/qSqlParser.c index 2459439b7b167a3624646168034c6152e2062d47..4a18a1f4a4265b2eb0289a5a16f5d2c53064e0d3 100644 --- a/src/query/src/qSqlParser.c +++ b/src/query/src/qSqlParser.c @@ -229,7 +229,6 @@ tSqlExpr *tSqlExprCreate(tSqlExpr *pLeft, tSqlExpr *pRight, int32_t optrType) { pExpr->flags &= ~(1 << EXPR_FLAG_TS_ERROR); } - switch (optrType) { case TK_PLUS: { pExpr->value.i64 = pLeft->value.i64 + pRight->value.i64; @@ -325,7 +324,6 @@ static FORCE_INLINE int32_t tStrTokenCompare(SStrToken* left, SStrToken* right) return (left->type == right->type && left->n == right->n && strncasecmp(left->z, right->z, left->n) == 0) ? 0 : 1; } - int32_t tSqlExprCompare(tSqlExpr *left, tSqlExpr *right) { if ((left == NULL && right) || (left && right == NULL)) { return 1; @@ -389,8 +387,6 @@ int32_t tSqlExprCompare(tSqlExpr *left, tSqlExpr *right) { return 0; } - - tSqlExpr *tSqlExprClone(tSqlExpr *pSrc) { tSqlExpr *pExpr = calloc(1, sizeof(tSqlExpr)); @@ -536,11 +532,11 @@ SArray *tVariantListInsert(SArray *pList, tVariant *pVar, uint8_t sortOrder, int SRelationInfo *setTableNameList(SRelationInfo* pRelationInfo, SStrToken *pName, SStrToken* pAlias) { if (pRelationInfo == NULL) { pRelationInfo = calloc(1, sizeof(SRelationInfo)); - pRelationInfo->list = taosArrayInit(4, sizeof(STableNamePair)); + pRelationInfo->list = taosArrayInit(4, sizeof(SRelElementPair)); } pRelationInfo->type = SQL_NODE_FROM_TABLELIST; - STableNamePair p = {.name = *pName}; + SRelElementPair p = {.tableName = *pName}; if (pAlias != NULL) { p.aliasName = *pAlias; } else { @@ -551,18 +547,6 @@ SRelationInfo *setTableNameList(SRelationInfo* pRelationInfo, SStrToken *pName, return pRelationInfo; } -SRelationInfo* setSubquery(SRelationInfo* pRelationInfo, SArray* pList) { - if (pRelationInfo == NULL) { - pRelationInfo = calloc(1, sizeof(SRelationInfo)); - pRelationInfo->list = taosArrayInit(4, POINTER_BYTES); - } - - pRelationInfo->type = SQL_NODE_FROM_SUBQUERY; - taosArrayPush(pRelationInfo->list, &pList); - - return pRelationInfo; -} - void* destroyRelationInfo(SRelationInfo* pRelationInfo) { if (pRelationInfo == NULL) { return NULL; @@ -573,7 +557,7 @@ void* destroyRelationInfo(SRelationInfo* pRelationInfo) { } else { size_t size = taosArrayGetSize(pRelationInfo->list); for(int32_t i = 0; i < size; ++i) { - SArray* pa = taosArrayGetP(pRelationInfo->list, 0); + SArray* pa = taosArrayGetP(pRelationInfo->list, i); destroyAllSqlNode(pa); } taosArrayDestroy(pRelationInfo->list); @@ -583,6 +567,24 @@ void* destroyRelationInfo(SRelationInfo* pRelationInfo) { return NULL; } +SRelationInfo* addSubqueryElem(SRelationInfo* pRelationInfo, SArray* pSub, SStrToken* pAlias) { + if (pRelationInfo == NULL) { + pRelationInfo = calloc(1, sizeof(SRelationInfo)); + pRelationInfo->list = taosArrayInit(4, sizeof(SRelElementPair)); + } + + pRelationInfo->type = SQL_NODE_FROM_SUBQUERY; + + SRelElementPair p = {.pSubquery = pSub}; + if (pAlias != NULL) { + p.aliasName = *pAlias; + } else { + TPARSER_SET_NONE_TOKEN(p.aliasName); + } + + taosArrayPush(pRelationInfo->list, &p); + return pRelationInfo; +} void tSetDbName(SStrToken *pCpxName, SStrToken *pDb) { pCpxName->type = pDb->type; @@ -724,9 +726,9 @@ void tSetColumnType(TAOS_FIELD *pField, SStrToken *type) { * extract the select info out of sql string */ SSqlNode *tSetQuerySqlNode(SStrToken *pSelectToken, SArray *pSelNodeList, SRelationInfo *pFrom, tSqlExpr *pWhere, - SArray *pGroupby, SArray *pSortOrder, SIntervalVal *pInterval, - SSessionWindowVal *pSession, SStrToken *pSliding, SArray *pFill, SLimitVal *pLimit, - SLimitVal *psLimit, tSqlExpr *pHaving) { + SArray *pGroupby, SArray *pSortOrder, SIntervalVal *pInterval, SSessionWindowVal *pSession, + SStrToken *pSliding, SArray *pFill, SLimitVal *pLimit, SLimitVal *psLimit, + tSqlExpr *pHaving) { assert(pSelNodeList != NULL); SSqlNode *pSqlNode = calloc(1, sizeof(SSqlNode)); diff --git a/src/query/src/sql.c b/src/query/src/sql.c index 5a5038be796dc6c244f08e7acfe7b5de30d71b28..9a6a0989c7eacb5b7340a3fdb3d0e7a08a96a79a 100644 --- a/src/query/src/sql.c +++ b/src/query/src/sql.c @@ -25,11 +25,11 @@ #include /************ Begin %include sections from the grammar ************************/ -#include -#include #include #include #include +#include +#include #include "qSqlparser.h" #include "tcmdtype.h" #include "ttoken.h" @@ -97,27 +97,27 @@ #endif /************* Begin control #defines *****************************************/ #define YYCODETYPE unsigned short int -#define YYNOCODE 264 +#define YYNOCODE 265 #define YYACTIONTYPE unsigned short int #define ParseTOKENTYPE SStrToken typedef union { int yyinit; ParseTOKENTYPE yy0; - SCreateTableSql* yy14; - int yy20; - SSqlNode* yy116; - tSqlExpr* yy118; - SArray* yy159; - SIntervalVal yy184; - SCreatedTableInfo yy206; - SRelationInfo* yy236; - SSessionWindowVal yy249; - int64_t yy317; - SCreateDbInfo yy322; - SCreateAcctInfo yy351; - TAOS_FIELD yy407; - SLimitVal yy440; - tVariant yy488; + SCreatedTableInfo yy96; + SRelationInfo* yy148; + tSqlExpr* yy178; + SCreateAcctInfo yy187; + SArray* yy285; + TAOS_FIELD yy295; + SSqlNode* yy344; + tVariant yy362; + SIntervalVal yy376; + SLimitVal yy438; + int yy460; + SCreateTableSql* yy470; + SSessionWindowVal yy523; + int64_t yy525; + SCreateDbInfo yy526; } YYMINORTYPE; #ifndef YYSTACKDEPTH #define YYSTACKDEPTH 100 @@ -127,17 +127,17 @@ typedef union { #define ParseARG_FETCH SSqlInfo* pInfo = yypParser->pInfo #define ParseARG_STORE yypParser->pInfo = pInfo #define YYFALLBACK 1 -#define YYNSTATE 317 -#define YYNRULE 270 +#define YYNSTATE 323 +#define YYNRULE 273 #define YYNTOKEN 187 -#define YY_MAX_SHIFT 316 -#define YY_MIN_SHIFTREDUCE 511 -#define YY_MAX_SHIFTREDUCE 780 -#define YY_ERROR_ACTION 781 -#define YY_ACCEPT_ACTION 782 -#define YY_NO_ACTION 783 -#define YY_MIN_REDUCE 784 -#define YY_MAX_REDUCE 1053 +#define YY_MAX_SHIFT 322 +#define YY_MIN_SHIFTREDUCE 518 +#define YY_MAX_SHIFTREDUCE 790 +#define YY_ERROR_ACTION 791 +#define YY_ACCEPT_ACTION 792 +#define YY_NO_ACTION 793 +#define YY_MIN_REDUCE 794 +#define YY_MAX_REDUCE 1066 /************* End control #defines *******************************************/ /* Define the yytestcase() macro to be a no-op if is not already defined @@ -203,261 +203,266 @@ typedef union { ** yy_default[] Default action for each state. ** *********** Begin parsing tables **********************************************/ -#define YY_ACTTAB_COUNT (685) +#define YY_ACTTAB_COUNT (695) static const YYACTIONTYPE yy_action[] = { - /* 0 */ 925, 559, 206, 314, 211, 141, 952, 3, 168, 560, - /* 10 */ 782, 316, 134, 47, 48, 141, 51, 52, 30, 183, - /* 20 */ 217, 41, 183, 50, 264, 55, 53, 57, 54, 1034, - /* 30 */ 931, 214, 1035, 46, 45, 17, 183, 44, 43, 42, - /* 40 */ 47, 48, 223, 51, 52, 213, 1035, 217, 41, 559, - /* 50 */ 50, 264, 55, 53, 57, 54, 943, 560, 181, 208, - /* 60 */ 46, 45, 928, 222, 44, 43, 42, 48, 949, 51, - /* 70 */ 52, 244, 983, 217, 41, 249, 50, 264, 55, 53, - /* 80 */ 57, 54, 984, 638, 259, 85, 46, 45, 280, 931, - /* 90 */ 44, 43, 42, 512, 513, 514, 515, 516, 517, 518, - /* 100 */ 519, 520, 521, 522, 523, 524, 315, 943, 187, 207, - /* 110 */ 70, 290, 289, 47, 48, 30, 51, 52, 300, 919, - /* 120 */ 217, 41, 209, 50, 264, 55, 53, 57, 54, 44, - /* 130 */ 43, 42, 724, 46, 45, 674, 224, 44, 43, 42, - /* 140 */ 47, 49, 24, 51, 52, 228, 141, 217, 41, 559, - /* 150 */ 50, 264, 55, 53, 57, 54, 220, 560, 105, 928, - /* 160 */ 46, 45, 931, 300, 44, 43, 42, 23, 278, 309, - /* 170 */ 308, 277, 276, 275, 307, 274, 306, 305, 304, 273, - /* 180 */ 303, 302, 891, 30, 879, 880, 881, 882, 883, 884, - /* 190 */ 885, 886, 887, 888, 889, 890, 892, 893, 51, 52, - /* 200 */ 830, 1031, 217, 41, 167, 50, 264, 55, 53, 57, - /* 210 */ 54, 261, 18, 78, 230, 46, 45, 287, 286, 44, - /* 220 */ 43, 42, 216, 739, 221, 30, 728, 928, 731, 192, - /* 230 */ 734, 216, 739, 310, 1030, 728, 193, 731, 236, 734, - /* 240 */ 30, 118, 117, 191, 677, 559, 240, 239, 55, 53, - /* 250 */ 57, 54, 25, 560, 202, 203, 46, 45, 263, 931, - /* 260 */ 44, 43, 42, 202, 203, 74, 283, 61, 23, 928, - /* 270 */ 309, 308, 74, 36, 730, 307, 733, 306, 305, 304, - /* 280 */ 36, 303, 302, 899, 927, 662, 897, 898, 659, 62, - /* 290 */ 660, 900, 661, 902, 903, 901, 82, 904, 905, 103, - /* 300 */ 97, 108, 243, 917, 68, 30, 107, 113, 116, 106, - /* 310 */ 199, 5, 33, 157, 141, 110, 231, 232, 156, 92, - /* 320 */ 87, 91, 681, 226, 30, 56, 30, 914, 915, 29, - /* 330 */ 918, 729, 740, 732, 56, 175, 173, 171, 736, 1, - /* 340 */ 155, 740, 170, 121, 120, 119, 284, 736, 229, 928, - /* 350 */ 265, 46, 45, 69, 735, 44, 43, 42, 839, 666, - /* 360 */ 12, 667, 167, 735, 84, 288, 81, 292, 928, 215, - /* 370 */ 928, 313, 312, 126, 132, 130, 129, 80, 705, 706, - /* 380 */ 831, 79, 280, 929, 167, 916, 737, 245, 726, 684, - /* 390 */ 71, 31, 227, 994, 663, 282, 690, 247, 696, 697, - /* 400 */ 136, 760, 60, 20, 741, 19, 64, 648, 19, 241, - /* 410 */ 267, 31, 650, 6, 31, 269, 60, 1029, 649, 83, - /* 420 */ 28, 200, 60, 270, 727, 201, 65, 96, 95, 185, - /* 430 */ 14, 13, 993, 102, 101, 67, 218, 637, 16, 15, - /* 440 */ 664, 186, 665, 738, 115, 114, 743, 188, 182, 189, - /* 450 */ 190, 196, 197, 195, 180, 194, 184, 133, 1045, 990, - /* 460 */ 930, 989, 219, 291, 39, 951, 959, 944, 961, 135, - /* 470 */ 139, 976, 248, 975, 926, 131, 152, 151, 924, 153, - /* 480 */ 250, 154, 689, 210, 252, 150, 257, 145, 142, 842, - /* 490 */ 941, 143, 272, 144, 262, 37, 146, 66, 58, 178, - /* 500 */ 63, 260, 34, 258, 256, 281, 838, 147, 1050, 254, - /* 510 */ 93, 1049, 1047, 158, 285, 1044, 99, 148, 1043, 1041, - /* 520 */ 159, 860, 251, 35, 32, 38, 149, 179, 827, 109, - /* 530 */ 825, 111, 112, 823, 822, 233, 169, 820, 819, 818, - /* 540 */ 817, 816, 815, 172, 174, 40, 812, 810, 808, 806, - /* 550 */ 176, 803, 177, 301, 246, 72, 75, 104, 253, 977, - /* 560 */ 293, 294, 295, 296, 297, 204, 225, 298, 271, 299, - /* 570 */ 311, 780, 205, 198, 234, 88, 89, 235, 779, 237, - /* 580 */ 238, 778, 766, 765, 242, 247, 821, 814, 162, 266, - /* 590 */ 122, 861, 160, 165, 161, 164, 163, 166, 123, 124, - /* 600 */ 813, 805, 895, 125, 804, 2, 8, 73, 4, 669, - /* 610 */ 76, 691, 137, 212, 694, 86, 138, 77, 907, 255, - /* 620 */ 9, 698, 140, 26, 742, 7, 27, 11, 10, 21, - /* 630 */ 84, 744, 22, 268, 601, 597, 595, 594, 593, 590, - /* 640 */ 563, 279, 94, 90, 31, 59, 640, 639, 636, 585, - /* 650 */ 583, 98, 575, 581, 577, 579, 573, 571, 604, 603, - /* 660 */ 602, 600, 599, 100, 598, 596, 592, 591, 60, 561, - /* 670 */ 528, 784, 526, 783, 783, 783, 783, 783, 783, 127, - /* 680 */ 783, 783, 783, 783, 128, + /* 0 */ 962, 566, 209, 320, 70, 18, 215, 953, 186, 567, + /* 10 */ 792, 322, 184, 48, 49, 144, 52, 53, 218, 1048, + /* 20 */ 221, 42, 212, 51, 270, 56, 54, 58, 55, 927, + /* 30 */ 645, 186, 941, 47, 46, 186, 926, 45, 44, 43, + /* 40 */ 48, 49, 1047, 52, 53, 217, 1048, 221, 42, 566, + /* 50 */ 51, 270, 56, 54, 58, 55, 953, 567, 296, 295, + /* 60 */ 47, 46, 959, 144, 45, 44, 43, 49, 31, 52, + /* 70 */ 53, 248, 137, 221, 42, 83, 51, 270, 56, 54, + /* 80 */ 58, 55, 286, 997, 88, 265, 47, 46, 72, 306, + /* 90 */ 45, 44, 43, 519, 520, 521, 522, 523, 524, 525, + /* 100 */ 526, 527, 528, 529, 530, 531, 321, 233, 286, 210, + /* 110 */ 71, 566, 937, 48, 49, 31, 52, 53, 929, 567, + /* 120 */ 221, 42, 566, 51, 270, 56, 54, 58, 55, 267, + /* 130 */ 567, 81, 734, 47, 46, 255, 254, 45, 44, 43, + /* 140 */ 48, 50, 939, 52, 53, 232, 190, 221, 42, 240, + /* 150 */ 51, 270, 56, 54, 58, 55, 211, 244, 243, 938, + /* 160 */ 47, 46, 1, 158, 45, 44, 43, 24, 284, 315, + /* 170 */ 314, 283, 282, 281, 313, 280, 312, 311, 310, 279, + /* 180 */ 309, 308, 901, 31, 889, 890, 891, 892, 893, 894, + /* 190 */ 895, 896, 897, 898, 899, 900, 902, 903, 52, 53, + /* 200 */ 715, 716, 221, 42, 230, 51, 270, 56, 54, 58, + /* 210 */ 55, 935, 19, 1044, 234, 47, 46, 293, 292, 45, + /* 220 */ 44, 43, 220, 749, 224, 25, 738, 938, 741, 195, + /* 230 */ 744, 220, 749, 3, 171, 738, 196, 741, 740, 744, + /* 240 */ 743, 121, 120, 194, 45, 44, 43, 1043, 56, 54, + /* 250 */ 58, 55, 681, 227, 205, 206, 47, 46, 269, 8, + /* 260 */ 45, 44, 43, 205, 206, 1042, 77, 739, 24, 742, + /* 270 */ 315, 314, 77, 231, 37, 313, 288, 312, 311, 310, + /* 280 */ 37, 309, 308, 909, 108, 669, 907, 908, 666, 306, + /* 290 */ 667, 910, 668, 912, 913, 911, 85, 914, 915, 106, + /* 300 */ 100, 111, 247, 203, 69, 31, 110, 116, 119, 109, + /* 310 */ 202, 5, 34, 160, 226, 113, 235, 236, 159, 95, + /* 320 */ 90, 94, 228, 31, 31, 57, 684, 924, 925, 30, + /* 330 */ 928, 31, 750, 31, 57, 178, 176, 174, 746, 144, + /* 340 */ 941, 750, 173, 124, 123, 122, 225, 746, 941, 938, + /* 350 */ 747, 47, 46, 204, 745, 45, 44, 43, 319, 318, + /* 360 */ 129, 316, 144, 745, 289, 290, 271, 938, 938, 135, + /* 370 */ 133, 132, 294, 953, 298, 938, 13, 938, 188, 748, + /* 380 */ 87, 840, 84, 736, 74, 170, 849, 941, 213, 841, + /* 390 */ 170, 249, 251, 170, 670, 32, 700, 75, 219, 706, + /* 400 */ 139, 189, 707, 61, 688, 251, 21, 996, 770, 62, + /* 410 */ 751, 191, 20, 655, 20, 65, 273, 32, 657, 737, + /* 420 */ 32, 275, 61, 1058, 656, 86, 29, 185, 61, 276, + /* 430 */ 82, 63, 940, 99, 98, 66, 15, 14, 192, 105, + /* 440 */ 104, 68, 6, 644, 118, 117, 193, 17, 16, 673, + /* 450 */ 199, 674, 200, 671, 198, 672, 183, 197, 187, 1007, + /* 460 */ 245, 1006, 753, 222, 40, 1003, 1002, 223, 297, 136, + /* 470 */ 989, 961, 972, 969, 988, 970, 954, 252, 974, 138, + /* 480 */ 936, 142, 154, 134, 256, 214, 905, 155, 699, 150, + /* 490 */ 934, 258, 145, 156, 260, 157, 148, 146, 951, 147, + /* 500 */ 59, 852, 67, 278, 263, 38, 181, 64, 35, 268, + /* 510 */ 266, 287, 848, 264, 1063, 96, 1062, 1060, 161, 291, + /* 520 */ 1057, 102, 1056, 1054, 162, 870, 36, 33, 149, 262, + /* 530 */ 39, 182, 837, 112, 835, 114, 115, 833, 832, 237, + /* 540 */ 172, 830, 257, 829, 828, 827, 826, 825, 175, 177, + /* 550 */ 822, 820, 818, 816, 179, 813, 180, 41, 250, 73, + /* 560 */ 307, 78, 259, 107, 990, 299, 300, 301, 302, 303, + /* 570 */ 304, 207, 305, 229, 317, 277, 790, 238, 239, 789, + /* 580 */ 208, 201, 91, 92, 241, 242, 788, 776, 775, 246, + /* 590 */ 831, 251, 272, 125, 126, 824, 165, 164, 871, 168, + /* 600 */ 163, 166, 167, 169, 127, 823, 2, 128, 815, 814, + /* 610 */ 9, 26, 4, 676, 76, 701, 216, 151, 152, 153, + /* 620 */ 253, 79, 917, 140, 10, 704, 141, 80, 261, 754, + /* 630 */ 708, 143, 11, 89, 752, 27, 7, 28, 12, 22, + /* 640 */ 274, 23, 608, 604, 87, 602, 601, 600, 597, 570, + /* 650 */ 285, 93, 97, 32, 647, 60, 646, 643, 592, 590, + /* 660 */ 101, 103, 582, 588, 584, 586, 580, 578, 611, 610, + /* 670 */ 609, 607, 606, 605, 603, 599, 598, 61, 568, 535, + /* 680 */ 533, 794, 793, 793, 793, 793, 793, 793, 793, 793, + /* 690 */ 793, 793, 793, 130, 131, }; static const YYCODETYPE yy_lookahead[] = { - /* 0 */ 191, 1, 190, 191, 210, 191, 191, 194, 195, 9, - /* 10 */ 188, 189, 191, 13, 14, 191, 16, 17, 191, 252, - /* 20 */ 20, 21, 252, 23, 24, 25, 26, 27, 28, 262, - /* 30 */ 236, 261, 262, 33, 34, 252, 252, 37, 38, 39, - /* 40 */ 13, 14, 233, 16, 17, 261, 262, 20, 21, 1, - /* 50 */ 23, 24, 25, 26, 27, 28, 234, 9, 252, 232, - /* 60 */ 33, 34, 235, 210, 37, 38, 39, 14, 253, 16, - /* 70 */ 17, 249, 258, 20, 21, 254, 23, 24, 25, 26, - /* 80 */ 27, 28, 258, 5, 260, 197, 33, 34, 79, 236, + /* 0 */ 191, 1, 190, 191, 197, 252, 210, 234, 252, 9, + /* 10 */ 188, 189, 252, 13, 14, 191, 16, 17, 262, 263, + /* 20 */ 20, 21, 249, 23, 24, 25, 26, 27, 28, 0, + /* 30 */ 5, 252, 236, 33, 34, 252, 229, 37, 38, 39, + /* 40 */ 13, 14, 263, 16, 17, 262, 263, 20, 21, 1, + /* 50 */ 23, 24, 25, 26, 27, 28, 234, 9, 33, 34, + /* 60 */ 33, 34, 253, 191, 37, 38, 39, 14, 191, 16, + /* 70 */ 17, 249, 191, 20, 21, 237, 23, 24, 25, 26, + /* 80 */ 27, 28, 79, 259, 197, 261, 33, 34, 250, 81, /* 90 */ 37, 38, 39, 45, 46, 47, 48, 49, 50, 51, - /* 100 */ 52, 53, 54, 55, 56, 57, 58, 234, 252, 61, - /* 110 */ 110, 33, 34, 13, 14, 191, 16, 17, 81, 231, - /* 120 */ 20, 21, 249, 23, 24, 25, 26, 27, 28, 37, - /* 130 */ 38, 39, 105, 33, 34, 109, 210, 37, 38, 39, - /* 140 */ 13, 14, 116, 16, 17, 68, 191, 20, 21, 1, - /* 150 */ 23, 24, 25, 26, 27, 28, 232, 9, 76, 235, - /* 160 */ 33, 34, 236, 81, 37, 38, 39, 88, 89, 90, + /* 100 */ 52, 53, 54, 55, 56, 57, 58, 191, 79, 61, + /* 110 */ 110, 1, 235, 13, 14, 191, 16, 17, 231, 9, + /* 120 */ 20, 21, 1, 23, 24, 25, 26, 27, 28, 257, + /* 130 */ 9, 259, 105, 33, 34, 254, 255, 37, 38, 39, + /* 140 */ 13, 14, 226, 16, 17, 68, 252, 20, 21, 135, + /* 150 */ 23, 24, 25, 26, 27, 28, 232, 143, 144, 235, + /* 160 */ 33, 34, 198, 199, 37, 38, 39, 88, 89, 90, /* 170 */ 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, /* 180 */ 101, 102, 209, 191, 211, 212, 213, 214, 215, 216, /* 190 */ 217, 218, 219, 220, 221, 222, 223, 224, 16, 17, - /* 200 */ 196, 252, 20, 21, 200, 23, 24, 25, 26, 27, - /* 210 */ 28, 256, 44, 258, 137, 33, 34, 140, 141, 37, - /* 220 */ 38, 39, 1, 2, 232, 191, 5, 235, 7, 61, - /* 230 */ 9, 1, 2, 210, 252, 5, 68, 7, 135, 9, - /* 240 */ 191, 73, 74, 75, 37, 1, 143, 144, 25, 26, - /* 250 */ 27, 28, 104, 9, 33, 34, 33, 34, 37, 236, - /* 260 */ 37, 38, 39, 33, 34, 104, 232, 109, 88, 235, - /* 270 */ 90, 91, 104, 112, 5, 95, 7, 97, 98, 99, - /* 280 */ 112, 101, 102, 209, 235, 2, 212, 213, 5, 131, + /* 200 */ 124, 125, 20, 21, 68, 23, 24, 25, 26, 27, + /* 210 */ 28, 191, 44, 252, 137, 33, 34, 140, 141, 37, + /* 220 */ 38, 39, 1, 2, 232, 104, 5, 235, 7, 61, + /* 230 */ 9, 1, 2, 194, 195, 5, 68, 7, 5, 9, + /* 240 */ 7, 73, 74, 75, 37, 38, 39, 252, 25, 26, + /* 250 */ 27, 28, 109, 233, 33, 34, 33, 34, 37, 116, + /* 260 */ 37, 38, 39, 33, 34, 252, 104, 5, 88, 7, + /* 270 */ 90, 91, 104, 137, 112, 95, 140, 97, 98, 99, + /* 280 */ 112, 101, 102, 209, 76, 2, 212, 213, 5, 81, /* 290 */ 7, 217, 9, 219, 220, 221, 197, 223, 224, 62, - /* 300 */ 63, 64, 134, 0, 136, 191, 69, 70, 71, 72, - /* 310 */ 142, 62, 63, 64, 191, 78, 33, 34, 69, 70, - /* 320 */ 71, 72, 115, 68, 191, 104, 191, 228, 229, 230, - /* 330 */ 231, 5, 111, 7, 104, 62, 63, 64, 117, 198, - /* 340 */ 199, 111, 69, 70, 71, 72, 232, 117, 191, 235, - /* 350 */ 15, 33, 34, 197, 133, 37, 38, 39, 196, 5, - /* 360 */ 104, 7, 200, 133, 108, 232, 110, 232, 235, 60, - /* 370 */ 235, 65, 66, 67, 62, 63, 64, 237, 124, 125, - /* 380 */ 196, 258, 79, 226, 200, 229, 117, 105, 1, 105, - /* 390 */ 250, 109, 137, 227, 111, 140, 105, 113, 105, 105, - /* 400 */ 109, 105, 109, 109, 105, 109, 109, 105, 109, 191, - /* 410 */ 105, 109, 105, 104, 109, 105, 109, 252, 105, 109, - /* 420 */ 104, 252, 109, 107, 37, 252, 129, 138, 139, 252, - /* 430 */ 138, 139, 227, 138, 139, 104, 227, 106, 138, 139, - /* 440 */ 5, 252, 7, 117, 76, 77, 111, 252, 252, 252, - /* 450 */ 252, 252, 252, 252, 252, 252, 252, 191, 236, 227, - /* 460 */ 236, 227, 227, 227, 251, 191, 191, 234, 191, 191, - /* 470 */ 191, 259, 234, 259, 234, 60, 191, 238, 191, 191, - /* 480 */ 255, 191, 117, 255, 255, 239, 255, 244, 247, 191, - /* 490 */ 248, 246, 191, 245, 122, 191, 243, 128, 127, 191, - /* 500 */ 130, 126, 191, 121, 120, 191, 191, 242, 191, 119, - /* 510 */ 191, 191, 191, 191, 191, 191, 191, 241, 191, 191, - /* 520 */ 191, 191, 118, 191, 191, 191, 240, 191, 191, 191, + /* 300 */ 63, 64, 134, 252, 136, 191, 69, 70, 71, 72, + /* 310 */ 142, 62, 63, 64, 210, 78, 33, 34, 69, 70, + /* 320 */ 71, 72, 210, 191, 191, 104, 37, 228, 229, 230, + /* 330 */ 231, 191, 111, 191, 104, 62, 63, 64, 117, 191, + /* 340 */ 236, 111, 69, 70, 71, 72, 232, 117, 236, 235, + /* 350 */ 117, 33, 34, 252, 133, 37, 38, 39, 65, 66, + /* 360 */ 67, 210, 191, 133, 232, 232, 15, 235, 235, 62, + /* 370 */ 63, 64, 232, 234, 232, 235, 104, 235, 252, 117, + /* 380 */ 108, 196, 110, 1, 105, 200, 196, 236, 249, 196, + /* 390 */ 200, 105, 113, 200, 111, 109, 105, 105, 60, 105, + /* 400 */ 109, 252, 105, 109, 115, 113, 109, 259, 105, 109, + /* 410 */ 105, 252, 109, 105, 109, 109, 105, 109, 105, 37, + /* 420 */ 109, 105, 109, 236, 105, 109, 104, 252, 109, 107, + /* 430 */ 259, 131, 236, 138, 139, 129, 138, 139, 252, 138, + /* 440 */ 139, 104, 104, 106, 76, 77, 252, 138, 139, 5, + /* 450 */ 252, 7, 252, 5, 252, 7, 252, 252, 252, 227, + /* 460 */ 191, 227, 111, 227, 251, 227, 227, 227, 227, 191, + /* 470 */ 260, 191, 191, 191, 260, 191, 234, 234, 191, 191, + /* 480 */ 234, 191, 238, 60, 256, 256, 225, 191, 117, 242, + /* 490 */ 191, 256, 247, 191, 119, 191, 244, 246, 248, 245, + /* 500 */ 127, 191, 128, 191, 256, 191, 191, 130, 191, 122, + /* 510 */ 126, 191, 191, 121, 191, 191, 191, 191, 191, 191, + /* 520 */ 191, 191, 191, 191, 191, 191, 191, 191, 243, 120, /* 530 */ 191, 191, 191, 191, 191, 191, 191, 191, 191, 191, - /* 540 */ 191, 191, 191, 191, 191, 132, 191, 191, 191, 191, - /* 550 */ 191, 191, 191, 103, 192, 192, 192, 87, 192, 192, - /* 560 */ 86, 50, 83, 85, 54, 192, 192, 84, 192, 82, - /* 570 */ 79, 5, 192, 192, 145, 197, 197, 5, 5, 145, - /* 580 */ 5, 5, 90, 89, 135, 113, 192, 192, 202, 107, - /* 590 */ 193, 208, 207, 204, 206, 203, 205, 201, 193, 193, - /* 600 */ 192, 192, 225, 193, 192, 198, 104, 114, 194, 105, - /* 610 */ 109, 105, 104, 1, 105, 76, 109, 104, 225, 104, - /* 620 */ 123, 105, 104, 109, 105, 104, 109, 104, 123, 104, - /* 630 */ 108, 111, 104, 107, 9, 5, 5, 5, 5, 5, - /* 640 */ 80, 15, 139, 76, 109, 16, 5, 5, 105, 5, - /* 650 */ 5, 139, 5, 5, 5, 5, 5, 5, 5, 5, - /* 660 */ 5, 5, 5, 139, 5, 5, 5, 5, 109, 80, - /* 670 */ 60, 0, 59, 263, 263, 263, 263, 263, 263, 21, - /* 680 */ 263, 263, 263, 263, 21, 263, 263, 263, 263, 263, - /* 690 */ 263, 263, 263, 263, 263, 263, 263, 263, 263, 263, - /* 700 */ 263, 263, 263, 263, 263, 263, 263, 263, 263, 263, - /* 710 */ 263, 263, 263, 263, 263, 263, 263, 263, 263, 263, - /* 720 */ 263, 263, 263, 263, 263, 263, 263, 263, 263, 263, - /* 730 */ 263, 263, 263, 263, 263, 263, 263, 263, 263, 263, - /* 740 */ 263, 263, 263, 263, 263, 263, 263, 263, 263, 263, - /* 750 */ 263, 263, 263, 263, 263, 263, 263, 263, 263, 263, - /* 760 */ 263, 263, 263, 263, 263, 263, 263, 263, 263, 263, - /* 770 */ 263, 263, 263, 263, 263, 263, 263, 263, 263, 263, - /* 780 */ 263, 263, 263, 263, 263, 263, 263, 263, 263, 263, - /* 790 */ 263, 263, 263, 263, 263, 263, 263, 263, 263, 263, - /* 800 */ 263, 263, 263, 263, 263, 263, 263, 263, 263, 263, - /* 810 */ 263, 263, 263, 263, 263, 263, 263, 263, 263, 263, - /* 820 */ 263, 263, 263, 263, 263, 263, 263, 263, 263, 263, - /* 830 */ 263, 263, 263, 263, 263, 263, 263, 263, 263, 263, - /* 840 */ 263, 263, 263, 263, 263, 263, 263, 263, 263, 263, - /* 850 */ 263, 263, 263, 263, 263, 263, 263, 263, 263, 263, - /* 860 */ 263, 263, 263, 263, 263, 263, 263, 263, 263, 263, - /* 870 */ 263, 263, + /* 540 */ 191, 191, 118, 191, 191, 191, 191, 191, 191, 191, + /* 550 */ 191, 191, 191, 191, 191, 191, 191, 132, 192, 192, + /* 560 */ 103, 192, 192, 87, 192, 86, 50, 83, 85, 54, + /* 570 */ 84, 192, 82, 192, 79, 192, 5, 145, 5, 5, + /* 580 */ 192, 192, 197, 197, 145, 5, 5, 90, 89, 135, + /* 590 */ 192, 113, 107, 193, 193, 192, 202, 206, 208, 204, + /* 600 */ 207, 205, 203, 201, 193, 192, 198, 193, 192, 192, + /* 610 */ 104, 104, 194, 105, 114, 105, 1, 241, 240, 239, + /* 620 */ 109, 109, 225, 104, 123, 105, 109, 104, 104, 111, + /* 630 */ 105, 104, 123, 76, 105, 109, 104, 109, 104, 104, + /* 640 */ 107, 104, 9, 5, 108, 5, 5, 5, 5, 80, + /* 650 */ 15, 76, 139, 109, 5, 16, 5, 105, 5, 5, + /* 660 */ 139, 139, 5, 5, 5, 5, 5, 5, 5, 5, + /* 670 */ 5, 5, 5, 5, 5, 5, 5, 109, 80, 60, + /* 680 */ 59, 0, 264, 264, 264, 264, 264, 264, 264, 264, + /* 690 */ 264, 264, 264, 21, 21, 264, 264, 264, 264, 264, + /* 700 */ 264, 264, 264, 264, 264, 264, 264, 264, 264, 264, + /* 710 */ 264, 264, 264, 264, 264, 264, 264, 264, 264, 264, + /* 720 */ 264, 264, 264, 264, 264, 264, 264, 264, 264, 264, + /* 730 */ 264, 264, 264, 264, 264, 264, 264, 264, 264, 264, + /* 740 */ 264, 264, 264, 264, 264, 264, 264, 264, 264, 264, + /* 750 */ 264, 264, 264, 264, 264, 264, 264, 264, 264, 264, + /* 760 */ 264, 264, 264, 264, 264, 264, 264, 264, 264, 264, + /* 770 */ 264, 264, 264, 264, 264, 264, 264, 264, 264, 264, + /* 780 */ 264, 264, 264, 264, 264, 264, 264, 264, 264, 264, + /* 790 */ 264, 264, 264, 264, 264, 264, 264, 264, 264, 264, + /* 800 */ 264, 264, 264, 264, 264, 264, 264, 264, 264, 264, + /* 810 */ 264, 264, 264, 264, 264, 264, 264, 264, 264, 264, + /* 820 */ 264, 264, 264, 264, 264, 264, 264, 264, 264, 264, + /* 830 */ 264, 264, 264, 264, 264, 264, 264, 264, 264, 264, + /* 840 */ 264, 264, 264, 264, 264, 264, 264, 264, 264, 264, + /* 850 */ 264, 264, 264, 264, 264, 264, 264, 264, 264, 264, + /* 860 */ 264, 264, 264, 264, 264, 264, 264, 264, 264, 264, + /* 870 */ 264, 264, 264, 264, 264, 264, 264, 264, 264, 264, + /* 880 */ 264, 264, }; -#define YY_SHIFT_COUNT (316) +#define YY_SHIFT_COUNT (322) #define YY_SHIFT_MIN (0) -#define YY_SHIFT_MAX (671) +#define YY_SHIFT_MAX (681) static const unsigned short int yy_shift_ofst[] = { - /* 0 */ 168, 79, 79, 180, 180, 9, 221, 230, 244, 244, - /* 10 */ 244, 244, 244, 244, 244, 244, 244, 0, 48, 230, - /* 20 */ 283, 283, 283, 283, 148, 161, 244, 244, 244, 303, - /* 30 */ 244, 244, 82, 9, 37, 37, 685, 685, 685, 230, + /* 0 */ 168, 79, 79, 180, 180, 3, 221, 230, 121, 110, + /* 10 */ 110, 110, 110, 110, 110, 110, 110, 110, 0, 48, + /* 20 */ 230, 283, 283, 283, 283, 162, 162, 110, 110, 110, + /* 30 */ 29, 110, 110, 208, 3, 8, 8, 695, 695, 695, /* 40 */ 230, 230, 230, 230, 230, 230, 230, 230, 230, 230, - /* 50 */ 230, 230, 230, 230, 230, 230, 230, 230, 230, 283, - /* 60 */ 283, 78, 78, 78, 78, 78, 78, 78, 244, 244, - /* 70 */ 244, 207, 244, 161, 161, 244, 244, 244, 254, 254, - /* 80 */ 26, 161, 244, 244, 244, 244, 244, 244, 244, 244, - /* 90 */ 244, 244, 244, 244, 244, 244, 244, 244, 244, 244, - /* 100 */ 244, 244, 244, 244, 244, 244, 244, 244, 244, 244, - /* 110 */ 244, 244, 244, 244, 244, 244, 244, 244, 244, 244, - /* 120 */ 244, 244, 244, 244, 244, 244, 244, 244, 244, 244, - /* 130 */ 244, 244, 244, 415, 415, 415, 365, 365, 365, 415, - /* 140 */ 365, 415, 369, 370, 371, 372, 375, 382, 384, 390, - /* 150 */ 404, 413, 415, 415, 415, 450, 9, 9, 415, 415, - /* 160 */ 470, 474, 511, 479, 478, 510, 483, 487, 450, 415, - /* 170 */ 491, 491, 415, 491, 415, 491, 415, 415, 685, 685, - /* 180 */ 27, 100, 127, 100, 100, 53, 182, 223, 223, 223, - /* 190 */ 223, 237, 249, 273, 318, 318, 318, 318, 77, 103, - /* 200 */ 92, 92, 269, 326, 256, 255, 306, 312, 282, 284, - /* 210 */ 291, 293, 294, 296, 299, 387, 309, 335, 158, 297, - /* 220 */ 302, 305, 307, 310, 313, 316, 289, 292, 295, 331, - /* 230 */ 300, 354, 435, 368, 566, 429, 572, 573, 434, 575, - /* 240 */ 576, 492, 494, 449, 472, 482, 502, 493, 504, 501, - /* 250 */ 506, 508, 509, 507, 513, 612, 515, 516, 518, 514, - /* 260 */ 497, 517, 505, 519, 521, 520, 523, 482, 525, 526, - /* 270 */ 528, 522, 539, 625, 630, 631, 632, 633, 634, 560, - /* 280 */ 626, 567, 503, 535, 535, 629, 512, 524, 535, 641, - /* 290 */ 642, 543, 535, 644, 645, 647, 648, 649, 650, 651, - /* 300 */ 652, 653, 654, 655, 656, 657, 659, 660, 661, 662, - /* 310 */ 559, 589, 658, 663, 610, 613, 671, + /* 50 */ 230, 230, 230, 230, 230, 230, 230, 230, 230, 230, + /* 60 */ 283, 283, 25, 25, 25, 25, 25, 25, 25, 110, + /* 70 */ 110, 110, 289, 110, 110, 110, 162, 162, 110, 110, + /* 80 */ 110, 76, 76, 143, 162, 110, 110, 110, 110, 110, + /* 90 */ 110, 110, 110, 110, 110, 110, 110, 110, 110, 110, + /* 100 */ 110, 110, 110, 110, 110, 110, 110, 110, 110, 110, + /* 110 */ 110, 110, 110, 110, 110, 110, 110, 110, 110, 110, + /* 120 */ 110, 110, 110, 110, 110, 110, 110, 110, 110, 110, + /* 130 */ 110, 110, 110, 110, 110, 110, 423, 423, 423, 371, + /* 140 */ 371, 371, 423, 371, 423, 374, 377, 373, 387, 384, + /* 150 */ 392, 409, 375, 424, 425, 423, 423, 423, 457, 3, + /* 160 */ 3, 423, 423, 476, 479, 516, 484, 483, 515, 486, + /* 170 */ 490, 457, 423, 495, 495, 423, 495, 423, 495, 423, + /* 180 */ 423, 695, 695, 27, 100, 127, 100, 100, 53, 182, + /* 190 */ 223, 223, 223, 223, 237, 249, 273, 318, 318, 318, + /* 200 */ 318, 77, 14, 207, 207, 233, 262, 272, 136, 293, + /* 210 */ 307, 286, 279, 292, 291, 294, 297, 303, 305, 382, + /* 220 */ 338, 351, 300, 306, 308, 311, 313, 316, 319, 322, + /* 230 */ 295, 298, 301, 337, 309, 444, 448, 368, 571, 432, + /* 240 */ 573, 574, 439, 580, 581, 497, 499, 454, 478, 485, + /* 250 */ 506, 500, 508, 507, 511, 512, 510, 519, 520, 517, + /* 260 */ 523, 615, 524, 525, 527, 526, 501, 528, 509, 529, + /* 270 */ 532, 518, 534, 485, 535, 533, 537, 536, 557, 633, + /* 280 */ 638, 640, 641, 642, 643, 569, 635, 575, 513, 544, + /* 290 */ 544, 639, 521, 522, 544, 649, 651, 552, 544, 653, + /* 300 */ 654, 657, 658, 659, 660, 661, 662, 663, 664, 665, + /* 310 */ 666, 667, 668, 669, 670, 671, 568, 598, 672, 673, + /* 320 */ 619, 621, 681, }; -#define YY_REDUCE_COUNT (179) -#define YY_REDUCE_MIN (-233) -#define YY_REDUCE_MAX (414) +#define YY_REDUCE_COUNT (182) +#define YY_REDUCE_MIN (-247) +#define YY_REDUCE_MAX (418) static const short yy_reduce_ofst[] = { - /* 0 */ -178, -27, -27, 74, 74, 99, -230, -216, -173, -176, - /* 10 */ -45, -76, -8, 34, 114, 133, 135, -185, -188, -233, - /* 20 */ -206, -147, -74, 23, -179, -127, -186, 123, -191, -112, - /* 30 */ 157, 49, 4, 156, 162, 184, 140, 141, -187, -217, - /* 40 */ -194, -144, -51, -18, 165, 169, 173, 177, 189, 195, - /* 50 */ 196, 197, 198, 199, 200, 201, 202, 203, 204, 222, - /* 60 */ 224, 166, 205, 209, 232, 234, 235, 236, 218, 266, - /* 70 */ 274, 213, 275, 233, 238, 277, 278, 279, 212, 214, - /* 80 */ 239, 240, 285, 287, 288, 290, 298, 301, 304, 308, - /* 90 */ 311, 314, 315, 317, 319, 320, 321, 322, 323, 324, - /* 100 */ 325, 327, 328, 329, 330, 332, 333, 334, 336, 337, - /* 110 */ 338, 339, 340, 341, 342, 343, 344, 345, 346, 347, - /* 120 */ 348, 349, 350, 351, 352, 353, 355, 356, 357, 358, - /* 130 */ 359, 360, 361, 362, 363, 364, 225, 228, 229, 366, - /* 140 */ 231, 367, 242, 241, 245, 248, 243, 253, 265, 276, - /* 150 */ 286, 246, 373, 374, 376, 377, 378, 379, 380, 381, - /* 160 */ 383, 385, 388, 386, 391, 392, 389, 396, 393, 394, - /* 170 */ 397, 405, 395, 406, 408, 410, 409, 412, 407, 414, + /* 0 */ -178, -27, -27, 74, 74, 99, -244, -217, -119, -76, + /* 10 */ -176, -128, -8, 114, 132, 133, 140, 142, -191, -188, + /* 20 */ -221, -204, 104, 112, 151, -227, 139, 148, 171, 20, + /* 30 */ -113, -84, -123, 185, -193, 190, 193, -162, -36, 39, + /* 40 */ -247, -240, -106, -39, -5, 13, 51, 101, 126, 149, + /* 50 */ 159, 175, 186, 194, 198, 200, 202, 204, 205, 206, + /* 60 */ 187, 196, 232, 234, 236, 238, 239, 240, 241, 269, + /* 70 */ 278, 280, 213, 281, 282, 284, 242, 243, 287, 288, + /* 80 */ 290, 210, 214, 244, 246, 296, 299, 302, 304, 310, + /* 90 */ 312, 314, 315, 317, 320, 321, 323, 324, 325, 326, + /* 100 */ 327, 328, 329, 330, 331, 332, 333, 334, 335, 336, + /* 110 */ 339, 340, 341, 342, 343, 344, 345, 346, 347, 348, + /* 120 */ 349, 350, 352, 353, 354, 355, 356, 357, 358, 359, + /* 130 */ 360, 361, 362, 363, 364, 365, 366, 367, 369, 228, + /* 140 */ 229, 235, 370, 248, 372, 250, 245, 251, 254, 252, + /* 150 */ 285, 247, 376, 378, 380, 379, 381, 383, 261, 385, + /* 160 */ 386, 388, 389, 390, 393, 391, 394, 396, 399, 395, + /* 170 */ 402, 397, 398, 400, 401, 403, 411, 413, 414, 416, + /* 180 */ 417, 408, 418, }; static const YYACTIONTYPE yy_default[] = { - /* 0 */ 781, 894, 840, 906, 828, 837, 1037, 1037, 781, 781, - /* 10 */ 781, 781, 781, 781, 781, 781, 781, 953, 800, 1037, - /* 20 */ 781, 781, 781, 781, 781, 781, 781, 781, 781, 837, - /* 30 */ 781, 781, 843, 837, 843, 843, 948, 878, 896, 781, - /* 40 */ 781, 781, 781, 781, 781, 781, 781, 781, 781, 781, - /* 50 */ 781, 781, 781, 781, 781, 781, 781, 781, 781, 781, - /* 60 */ 781, 781, 781, 781, 781, 781, 781, 781, 781, 781, - /* 70 */ 781, 955, 958, 781, 781, 960, 781, 781, 980, 980, - /* 80 */ 946, 781, 781, 781, 781, 781, 781, 781, 781, 781, - /* 90 */ 781, 781, 781, 781, 781, 781, 781, 781, 781, 781, - /* 100 */ 781, 781, 781, 781, 781, 781, 781, 781, 781, 826, - /* 110 */ 781, 824, 781, 781, 781, 781, 781, 781, 781, 781, - /* 120 */ 781, 781, 781, 781, 781, 781, 811, 781, 781, 781, - /* 130 */ 781, 781, 781, 802, 802, 802, 781, 781, 781, 802, - /* 140 */ 781, 802, 987, 991, 985, 973, 981, 972, 968, 966, - /* 150 */ 965, 995, 802, 802, 802, 841, 837, 837, 802, 802, - /* 160 */ 859, 857, 855, 847, 853, 849, 851, 845, 829, 802, - /* 170 */ 835, 835, 802, 835, 802, 835, 802, 802, 878, 896, - /* 180 */ 781, 996, 781, 1036, 986, 1026, 1025, 1032, 1024, 1023, - /* 190 */ 1022, 781, 781, 781, 1018, 1019, 1021, 1020, 781, 781, - /* 200 */ 1028, 1027, 781, 781, 781, 781, 781, 781, 781, 781, - /* 210 */ 781, 781, 781, 781, 781, 781, 998, 781, 992, 988, - /* 220 */ 781, 781, 781, 781, 781, 781, 781, 781, 781, 908, - /* 230 */ 781, 781, 781, 781, 781, 781, 781, 781, 781, 781, - /* 240 */ 781, 781, 781, 781, 945, 781, 781, 781, 781, 956, - /* 250 */ 781, 781, 781, 781, 781, 781, 781, 781, 781, 982, - /* 260 */ 781, 974, 781, 781, 781, 781, 781, 920, 781, 781, - /* 270 */ 781, 781, 781, 781, 781, 781, 781, 781, 781, 781, - /* 280 */ 781, 781, 781, 1048, 1046, 781, 781, 781, 1042, 781, - /* 290 */ 781, 781, 1040, 781, 781, 781, 781, 781, 781, 781, - /* 300 */ 781, 781, 781, 781, 781, 781, 781, 781, 781, 781, - /* 310 */ 862, 781, 809, 807, 781, 798, 781, + /* 0 */ 791, 904, 850, 916, 838, 847, 1050, 1050, 791, 791, + /* 10 */ 791, 791, 791, 791, 791, 791, 791, 791, 963, 810, + /* 20 */ 1050, 791, 791, 791, 791, 791, 791, 791, 791, 791, + /* 30 */ 847, 791, 791, 853, 847, 853, 853, 958, 888, 906, + /* 40 */ 791, 791, 791, 791, 791, 791, 791, 791, 791, 791, + /* 50 */ 791, 791, 791, 791, 791, 791, 791, 791, 791, 791, + /* 60 */ 791, 791, 791, 791, 791, 791, 791, 791, 791, 791, + /* 70 */ 791, 791, 965, 971, 968, 791, 791, 791, 973, 791, + /* 80 */ 791, 993, 993, 956, 791, 791, 791, 791, 791, 791, + /* 90 */ 791, 791, 791, 791, 791, 791, 791, 791, 791, 791, + /* 100 */ 791, 791, 791, 791, 791, 791, 791, 791, 791, 791, + /* 110 */ 791, 791, 836, 791, 834, 791, 791, 791, 791, 791, + /* 120 */ 791, 791, 791, 791, 791, 791, 791, 791, 791, 821, + /* 130 */ 791, 791, 791, 791, 791, 791, 812, 812, 812, 791, + /* 140 */ 791, 791, 812, 791, 812, 1000, 1004, 998, 986, 994, + /* 150 */ 985, 981, 979, 978, 1008, 812, 812, 812, 851, 847, + /* 160 */ 847, 812, 812, 869, 867, 865, 857, 863, 859, 861, + /* 170 */ 855, 839, 812, 845, 845, 812, 845, 812, 845, 812, + /* 180 */ 812, 888, 906, 791, 1009, 791, 1049, 999, 1039, 1038, + /* 190 */ 1045, 1037, 1036, 1035, 791, 791, 791, 1031, 1032, 1034, + /* 200 */ 1033, 791, 791, 1041, 1040, 791, 791, 791, 791, 791, + /* 210 */ 791, 791, 791, 791, 791, 791, 791, 791, 791, 791, + /* 220 */ 1011, 791, 1005, 1001, 791, 791, 791, 791, 791, 791, + /* 230 */ 791, 791, 791, 918, 791, 791, 791, 791, 791, 791, + /* 240 */ 791, 791, 791, 791, 791, 791, 791, 791, 955, 791, + /* 250 */ 791, 791, 791, 791, 967, 966, 791, 791, 791, 791, + /* 260 */ 791, 791, 791, 791, 791, 995, 791, 987, 791, 791, + /* 270 */ 791, 791, 791, 930, 791, 791, 791, 791, 791, 791, + /* 280 */ 791, 791, 791, 791, 791, 791, 791, 791, 791, 1061, + /* 290 */ 1059, 791, 791, 791, 1055, 791, 791, 791, 1053, 791, + /* 300 */ 791, 791, 791, 791, 791, 791, 791, 791, 791, 791, + /* 310 */ 791, 791, 791, 791, 791, 791, 872, 791, 819, 817, + /* 320 */ 791, 808, 791, }; /********** End of lemon-generated parsing tables *****************************/ @@ -1005,14 +1010,15 @@ static const char *const yyTokenName[] = { /* 252 */ "expr", /* 253 */ "as", /* 254 */ "tablelist", - /* 255 */ "tmvar", - /* 256 */ "sortlist", - /* 257 */ "sortitem", - /* 258 */ "item", - /* 259 */ "sortorder", - /* 260 */ "grouplist", - /* 261 */ "exprlist", - /* 262 */ "expritem", + /* 255 */ "sub", + /* 256 */ "tmvar", + /* 257 */ "sortlist", + /* 258 */ "sortitem", + /* 259 */ "item", + /* 260 */ "sortorder", + /* 261 */ "grouplist", + /* 262 */ "exprlist", + /* 263 */ "expritem", }; #endif /* defined(YYCOVERAGE) || !defined(NDEBUG) */ @@ -1193,103 +1199,106 @@ static const char *const yyRuleName[] = { /* 170 */ "distinct ::= DISTINCT", /* 171 */ "distinct ::=", /* 172 */ "from ::= FROM tablelist", - /* 173 */ "from ::= FROM LP union RP", - /* 174 */ "tablelist ::= ids cpxName", - /* 175 */ "tablelist ::= ids cpxName ids", - /* 176 */ "tablelist ::= tablelist COMMA ids cpxName", - /* 177 */ "tablelist ::= tablelist COMMA ids cpxName ids", - /* 178 */ "tmvar ::= VARIABLE", - /* 179 */ "interval_opt ::= INTERVAL LP tmvar RP", - /* 180 */ "interval_opt ::= INTERVAL LP tmvar COMMA tmvar RP", - /* 181 */ "interval_opt ::=", - /* 182 */ "session_option ::=", - /* 183 */ "session_option ::= SESSION LP ids cpxName COMMA tmvar RP", - /* 184 */ "fill_opt ::=", - /* 185 */ "fill_opt ::= FILL LP ID COMMA tagitemlist RP", - /* 186 */ "fill_opt ::= FILL LP ID RP", - /* 187 */ "sliding_opt ::= SLIDING LP tmvar RP", - /* 188 */ "sliding_opt ::=", - /* 189 */ "orderby_opt ::=", - /* 190 */ "orderby_opt ::= ORDER BY sortlist", - /* 191 */ "sortlist ::= sortlist COMMA item sortorder", - /* 192 */ "sortlist ::= item sortorder", - /* 193 */ "item ::= ids cpxName", - /* 194 */ "sortorder ::= ASC", - /* 195 */ "sortorder ::= DESC", - /* 196 */ "sortorder ::=", - /* 197 */ "groupby_opt ::=", - /* 198 */ "groupby_opt ::= GROUP BY grouplist", - /* 199 */ "grouplist ::= grouplist COMMA item", - /* 200 */ "grouplist ::= item", - /* 201 */ "having_opt ::=", - /* 202 */ "having_opt ::= HAVING expr", - /* 203 */ "limit_opt ::=", - /* 204 */ "limit_opt ::= LIMIT signed", - /* 205 */ "limit_opt ::= LIMIT signed OFFSET signed", - /* 206 */ "limit_opt ::= LIMIT signed COMMA signed", - /* 207 */ "slimit_opt ::=", - /* 208 */ "slimit_opt ::= SLIMIT signed", - /* 209 */ "slimit_opt ::= SLIMIT signed SOFFSET signed", - /* 210 */ "slimit_opt ::= SLIMIT signed COMMA signed", - /* 211 */ "where_opt ::=", - /* 212 */ "where_opt ::= WHERE expr", - /* 213 */ "expr ::= LP expr RP", - /* 214 */ "expr ::= ID", - /* 215 */ "expr ::= ID DOT ID", - /* 216 */ "expr ::= ID DOT STAR", - /* 217 */ "expr ::= INTEGER", - /* 218 */ "expr ::= MINUS INTEGER", - /* 219 */ "expr ::= PLUS INTEGER", - /* 220 */ "expr ::= FLOAT", - /* 221 */ "expr ::= MINUS FLOAT", - /* 222 */ "expr ::= PLUS FLOAT", - /* 223 */ "expr ::= STRING", - /* 224 */ "expr ::= NOW", - /* 225 */ "expr ::= VARIABLE", - /* 226 */ "expr ::= PLUS VARIABLE", - /* 227 */ "expr ::= MINUS VARIABLE", - /* 228 */ "expr ::= BOOL", - /* 229 */ "expr ::= NULL", - /* 230 */ "expr ::= ID LP exprlist RP", - /* 231 */ "expr ::= ID LP STAR RP", - /* 232 */ "expr ::= expr IS NULL", - /* 233 */ "expr ::= expr IS NOT NULL", - /* 234 */ "expr ::= expr LT expr", - /* 235 */ "expr ::= expr GT expr", - /* 236 */ "expr ::= expr LE expr", - /* 237 */ "expr ::= expr GE expr", - /* 238 */ "expr ::= expr NE expr", - /* 239 */ "expr ::= expr EQ expr", - /* 240 */ "expr ::= expr BETWEEN expr AND expr", - /* 241 */ "expr ::= expr AND expr", - /* 242 */ "expr ::= expr OR expr", - /* 243 */ "expr ::= expr PLUS expr", - /* 244 */ "expr ::= expr MINUS expr", - /* 245 */ "expr ::= expr STAR expr", - /* 246 */ "expr ::= expr SLASH expr", - /* 247 */ "expr ::= expr REM expr", - /* 248 */ "expr ::= expr LIKE expr", - /* 249 */ "expr ::= expr IN LP exprlist RP", - /* 250 */ "exprlist ::= exprlist COMMA expritem", - /* 251 */ "exprlist ::= expritem", - /* 252 */ "expritem ::= expr", - /* 253 */ "expritem ::=", - /* 254 */ "cmd ::= RESET QUERY CACHE", - /* 255 */ "cmd ::= SYNCDB ids REPLICA", - /* 256 */ "cmd ::= ALTER TABLE ids cpxName ADD COLUMN columnlist", - /* 257 */ "cmd ::= ALTER TABLE ids cpxName DROP COLUMN ids", - /* 258 */ "cmd ::= ALTER TABLE ids cpxName ADD TAG columnlist", - /* 259 */ "cmd ::= ALTER TABLE ids cpxName DROP TAG ids", - /* 260 */ "cmd ::= ALTER TABLE ids cpxName CHANGE TAG ids ids", - /* 261 */ "cmd ::= ALTER TABLE ids cpxName SET TAG ids EQ tagitem", - /* 262 */ "cmd ::= ALTER STABLE ids cpxName ADD COLUMN columnlist", - /* 263 */ "cmd ::= ALTER STABLE ids cpxName DROP COLUMN ids", - /* 264 */ "cmd ::= ALTER STABLE ids cpxName ADD TAG columnlist", - /* 265 */ "cmd ::= ALTER STABLE ids cpxName DROP TAG ids", - /* 266 */ "cmd ::= ALTER STABLE ids cpxName CHANGE TAG ids ids", - /* 267 */ "cmd ::= KILL CONNECTION INTEGER", - /* 268 */ "cmd ::= KILL STREAM INTEGER COLON INTEGER", - /* 269 */ "cmd ::= KILL QUERY INTEGER COLON INTEGER", + /* 173 */ "from ::= FROM sub", + /* 174 */ "sub ::= LP union RP", + /* 175 */ "sub ::= LP union RP ids", + /* 176 */ "sub ::= sub COMMA LP union RP ids", + /* 177 */ "tablelist ::= ids cpxName", + /* 178 */ "tablelist ::= ids cpxName ids", + /* 179 */ "tablelist ::= tablelist COMMA ids cpxName", + /* 180 */ "tablelist ::= tablelist COMMA ids cpxName ids", + /* 181 */ "tmvar ::= VARIABLE", + /* 182 */ "interval_opt ::= INTERVAL LP tmvar RP", + /* 183 */ "interval_opt ::= INTERVAL LP tmvar COMMA tmvar RP", + /* 184 */ "interval_opt ::=", + /* 185 */ "session_option ::=", + /* 186 */ "session_option ::= SESSION LP ids cpxName COMMA tmvar RP", + /* 187 */ "fill_opt ::=", + /* 188 */ "fill_opt ::= FILL LP ID COMMA tagitemlist RP", + /* 189 */ "fill_opt ::= FILL LP ID RP", + /* 190 */ "sliding_opt ::= SLIDING LP tmvar RP", + /* 191 */ "sliding_opt ::=", + /* 192 */ "orderby_opt ::=", + /* 193 */ "orderby_opt ::= ORDER BY sortlist", + /* 194 */ "sortlist ::= sortlist COMMA item sortorder", + /* 195 */ "sortlist ::= item sortorder", + /* 196 */ "item ::= ids cpxName", + /* 197 */ "sortorder ::= ASC", + /* 198 */ "sortorder ::= DESC", + /* 199 */ "sortorder ::=", + /* 200 */ "groupby_opt ::=", + /* 201 */ "groupby_opt ::= GROUP BY grouplist", + /* 202 */ "grouplist ::= grouplist COMMA item", + /* 203 */ "grouplist ::= item", + /* 204 */ "having_opt ::=", + /* 205 */ "having_opt ::= HAVING expr", + /* 206 */ "limit_opt ::=", + /* 207 */ "limit_opt ::= LIMIT signed", + /* 208 */ "limit_opt ::= LIMIT signed OFFSET signed", + /* 209 */ "limit_opt ::= LIMIT signed COMMA signed", + /* 210 */ "slimit_opt ::=", + /* 211 */ "slimit_opt ::= SLIMIT signed", + /* 212 */ "slimit_opt ::= SLIMIT signed SOFFSET signed", + /* 213 */ "slimit_opt ::= SLIMIT signed COMMA signed", + /* 214 */ "where_opt ::=", + /* 215 */ "where_opt ::= WHERE expr", + /* 216 */ "expr ::= LP expr RP", + /* 217 */ "expr ::= ID", + /* 218 */ "expr ::= ID DOT ID", + /* 219 */ "expr ::= ID DOT STAR", + /* 220 */ "expr ::= INTEGER", + /* 221 */ "expr ::= MINUS INTEGER", + /* 222 */ "expr ::= PLUS INTEGER", + /* 223 */ "expr ::= FLOAT", + /* 224 */ "expr ::= MINUS FLOAT", + /* 225 */ "expr ::= PLUS FLOAT", + /* 226 */ "expr ::= STRING", + /* 227 */ "expr ::= NOW", + /* 228 */ "expr ::= VARIABLE", + /* 229 */ "expr ::= PLUS VARIABLE", + /* 230 */ "expr ::= MINUS VARIABLE", + /* 231 */ "expr ::= BOOL", + /* 232 */ "expr ::= NULL", + /* 233 */ "expr ::= ID LP exprlist RP", + /* 234 */ "expr ::= ID LP STAR RP", + /* 235 */ "expr ::= expr IS NULL", + /* 236 */ "expr ::= expr IS NOT NULL", + /* 237 */ "expr ::= expr LT expr", + /* 238 */ "expr ::= expr GT expr", + /* 239 */ "expr ::= expr LE expr", + /* 240 */ "expr ::= expr GE expr", + /* 241 */ "expr ::= expr NE expr", + /* 242 */ "expr ::= expr EQ expr", + /* 243 */ "expr ::= expr BETWEEN expr AND expr", + /* 244 */ "expr ::= expr AND expr", + /* 245 */ "expr ::= expr OR expr", + /* 246 */ "expr ::= expr PLUS expr", + /* 247 */ "expr ::= expr MINUS expr", + /* 248 */ "expr ::= expr STAR expr", + /* 249 */ "expr ::= expr SLASH expr", + /* 250 */ "expr ::= expr REM expr", + /* 251 */ "expr ::= expr LIKE expr", + /* 252 */ "expr ::= expr IN LP exprlist RP", + /* 253 */ "exprlist ::= exprlist COMMA expritem", + /* 254 */ "exprlist ::= expritem", + /* 255 */ "expritem ::= expr", + /* 256 */ "expritem ::=", + /* 257 */ "cmd ::= RESET QUERY CACHE", + /* 258 */ "cmd ::= SYNCDB ids REPLICA", + /* 259 */ "cmd ::= ALTER TABLE ids cpxName ADD COLUMN columnlist", + /* 260 */ "cmd ::= ALTER TABLE ids cpxName DROP COLUMN ids", + /* 261 */ "cmd ::= ALTER TABLE ids cpxName ADD TAG columnlist", + /* 262 */ "cmd ::= ALTER TABLE ids cpxName DROP TAG ids", + /* 263 */ "cmd ::= ALTER TABLE ids cpxName CHANGE TAG ids ids", + /* 264 */ "cmd ::= ALTER TABLE ids cpxName SET TAG ids EQ tagitem", + /* 265 */ "cmd ::= ALTER STABLE ids cpxName ADD COLUMN columnlist", + /* 266 */ "cmd ::= ALTER STABLE ids cpxName DROP COLUMN ids", + /* 267 */ "cmd ::= ALTER STABLE ids cpxName ADD TAG columnlist", + /* 268 */ "cmd ::= ALTER STABLE ids cpxName DROP TAG ids", + /* 269 */ "cmd ::= ALTER STABLE ids cpxName CHANGE TAG ids ids", + /* 270 */ "cmd ::= KILL CONNECTION INTEGER", + /* 271 */ "cmd ::= KILL STREAM INTEGER COLON INTEGER", + /* 272 */ "cmd ::= KILL QUERY INTEGER COLON INTEGER", }; #endif /* NDEBUG */ @@ -1417,51 +1426,52 @@ static void yy_destructor( case 242: /* fill_opt */ case 244: /* groupby_opt */ case 245: /* orderby_opt */ - case 256: /* sortlist */ - case 260: /* grouplist */ + case 257: /* sortlist */ + case 261: /* grouplist */ { -taosArrayDestroy((yypminor->yy159)); +taosArrayDestroy((yypminor->yy285)); } break; case 230: /* create_table_list */ { -destroyCreateTableSql((yypminor->yy14)); +destroyCreateTableSql((yypminor->yy470)); } break; case 234: /* select */ { -destroySqlNode((yypminor->yy116)); +destroySqlNode((yypminor->yy344)); } break; case 237: /* selcollist */ case 250: /* sclp */ - case 261: /* exprlist */ + case 262: /* exprlist */ { -tSqlExprListDestroy((yypminor->yy159)); +tSqlExprListDestroy((yypminor->yy285)); } break; case 238: /* from */ case 254: /* tablelist */ + case 255: /* sub */ { -destroyRelationInfo((yypminor->yy236)); +destroyRelationInfo((yypminor->yy148)); } break; case 239: /* where_opt */ case 246: /* having_opt */ case 252: /* expr */ - case 262: /* expritem */ + case 263: /* expritem */ { -tSqlExprDestroy((yypminor->yy118)); +tSqlExprDestroy((yypminor->yy178)); } break; case 249: /* union */ { -destroyAllSqlNode((yypminor->yy159)); +destroyAllSqlNode((yypminor->yy285)); } break; - case 257: /* sortitem */ + case 258: /* sortitem */ { -tVariantDestroy(&(yypminor->yy488)); +tVariantDestroy(&(yypminor->yy362)); } break; /********* End destructor definitions *****************************************/ @@ -1928,103 +1938,106 @@ static const struct { { 251, -1 }, /* (170) distinct ::= DISTINCT */ { 251, 0 }, /* (171) distinct ::= */ { 238, -2 }, /* (172) from ::= FROM tablelist */ - { 238, -4 }, /* (173) from ::= FROM LP union RP */ - { 254, -2 }, /* (174) tablelist ::= ids cpxName */ - { 254, -3 }, /* (175) tablelist ::= ids cpxName ids */ - { 254, -4 }, /* (176) tablelist ::= tablelist COMMA ids cpxName */ - { 254, -5 }, /* (177) tablelist ::= tablelist COMMA ids cpxName ids */ - { 255, -1 }, /* (178) tmvar ::= VARIABLE */ - { 240, -4 }, /* (179) interval_opt ::= INTERVAL LP tmvar RP */ - { 240, -6 }, /* (180) interval_opt ::= INTERVAL LP tmvar COMMA tmvar RP */ - { 240, 0 }, /* (181) interval_opt ::= */ - { 241, 0 }, /* (182) session_option ::= */ - { 241, -7 }, /* (183) session_option ::= SESSION LP ids cpxName COMMA tmvar RP */ - { 242, 0 }, /* (184) fill_opt ::= */ - { 242, -6 }, /* (185) fill_opt ::= FILL LP ID COMMA tagitemlist RP */ - { 242, -4 }, /* (186) fill_opt ::= FILL LP ID RP */ - { 243, -4 }, /* (187) sliding_opt ::= SLIDING LP tmvar RP */ - { 243, 0 }, /* (188) sliding_opt ::= */ - { 245, 0 }, /* (189) orderby_opt ::= */ - { 245, -3 }, /* (190) orderby_opt ::= ORDER BY sortlist */ - { 256, -4 }, /* (191) sortlist ::= sortlist COMMA item sortorder */ - { 256, -2 }, /* (192) sortlist ::= item sortorder */ - { 258, -2 }, /* (193) item ::= ids cpxName */ - { 259, -1 }, /* (194) sortorder ::= ASC */ - { 259, -1 }, /* (195) sortorder ::= DESC */ - { 259, 0 }, /* (196) sortorder ::= */ - { 244, 0 }, /* (197) groupby_opt ::= */ - { 244, -3 }, /* (198) groupby_opt ::= GROUP BY grouplist */ - { 260, -3 }, /* (199) grouplist ::= grouplist COMMA item */ - { 260, -1 }, /* (200) grouplist ::= item */ - { 246, 0 }, /* (201) having_opt ::= */ - { 246, -2 }, /* (202) having_opt ::= HAVING expr */ - { 248, 0 }, /* (203) limit_opt ::= */ - { 248, -2 }, /* (204) limit_opt ::= LIMIT signed */ - { 248, -4 }, /* (205) limit_opt ::= LIMIT signed OFFSET signed */ - { 248, -4 }, /* (206) limit_opt ::= LIMIT signed COMMA signed */ - { 247, 0 }, /* (207) slimit_opt ::= */ - { 247, -2 }, /* (208) slimit_opt ::= SLIMIT signed */ - { 247, -4 }, /* (209) slimit_opt ::= SLIMIT signed SOFFSET signed */ - { 247, -4 }, /* (210) slimit_opt ::= SLIMIT signed COMMA signed */ - { 239, 0 }, /* (211) where_opt ::= */ - { 239, -2 }, /* (212) where_opt ::= WHERE expr */ - { 252, -3 }, /* (213) expr ::= LP expr RP */ - { 252, -1 }, /* (214) expr ::= ID */ - { 252, -3 }, /* (215) expr ::= ID DOT ID */ - { 252, -3 }, /* (216) expr ::= ID DOT STAR */ - { 252, -1 }, /* (217) expr ::= INTEGER */ - { 252, -2 }, /* (218) expr ::= MINUS INTEGER */ - { 252, -2 }, /* (219) expr ::= PLUS INTEGER */ - { 252, -1 }, /* (220) expr ::= FLOAT */ - { 252, -2 }, /* (221) expr ::= MINUS FLOAT */ - { 252, -2 }, /* (222) expr ::= PLUS FLOAT */ - { 252, -1 }, /* (223) expr ::= STRING */ - { 252, -1 }, /* (224) expr ::= NOW */ - { 252, -1 }, /* (225) expr ::= VARIABLE */ - { 252, -2 }, /* (226) expr ::= PLUS VARIABLE */ - { 252, -2 }, /* (227) expr ::= MINUS VARIABLE */ - { 252, -1 }, /* (228) expr ::= BOOL */ - { 252, -1 }, /* (229) expr ::= NULL */ - { 252, -4 }, /* (230) expr ::= ID LP exprlist RP */ - { 252, -4 }, /* (231) expr ::= ID LP STAR RP */ - { 252, -3 }, /* (232) expr ::= expr IS NULL */ - { 252, -4 }, /* (233) expr ::= expr IS NOT NULL */ - { 252, -3 }, /* (234) expr ::= expr LT expr */ - { 252, -3 }, /* (235) expr ::= expr GT expr */ - { 252, -3 }, /* (236) expr ::= expr LE expr */ - { 252, -3 }, /* (237) expr ::= expr GE expr */ - { 252, -3 }, /* (238) expr ::= expr NE expr */ - { 252, -3 }, /* (239) expr ::= expr EQ expr */ - { 252, -5 }, /* (240) expr ::= expr BETWEEN expr AND expr */ - { 252, -3 }, /* (241) expr ::= expr AND expr */ - { 252, -3 }, /* (242) expr ::= expr OR expr */ - { 252, -3 }, /* (243) expr ::= expr PLUS expr */ - { 252, -3 }, /* (244) expr ::= expr MINUS expr */ - { 252, -3 }, /* (245) expr ::= expr STAR expr */ - { 252, -3 }, /* (246) expr ::= expr SLASH expr */ - { 252, -3 }, /* (247) expr ::= expr REM expr */ - { 252, -3 }, /* (248) expr ::= expr LIKE expr */ - { 252, -5 }, /* (249) expr ::= expr IN LP exprlist RP */ - { 261, -3 }, /* (250) exprlist ::= exprlist COMMA expritem */ - { 261, -1 }, /* (251) exprlist ::= expritem */ - { 262, -1 }, /* (252) expritem ::= expr */ - { 262, 0 }, /* (253) expritem ::= */ - { 189, -3 }, /* (254) cmd ::= RESET QUERY CACHE */ - { 189, -3 }, /* (255) cmd ::= SYNCDB ids REPLICA */ - { 189, -7 }, /* (256) cmd ::= ALTER TABLE ids cpxName ADD COLUMN columnlist */ - { 189, -7 }, /* (257) cmd ::= ALTER TABLE ids cpxName DROP COLUMN ids */ - { 189, -7 }, /* (258) cmd ::= ALTER TABLE ids cpxName ADD TAG columnlist */ - { 189, -7 }, /* (259) cmd ::= ALTER TABLE ids cpxName DROP TAG ids */ - { 189, -8 }, /* (260) cmd ::= ALTER TABLE ids cpxName CHANGE TAG ids ids */ - { 189, -9 }, /* (261) cmd ::= ALTER TABLE ids cpxName SET TAG ids EQ tagitem */ - { 189, -7 }, /* (262) cmd ::= ALTER STABLE ids cpxName ADD COLUMN columnlist */ - { 189, -7 }, /* (263) cmd ::= ALTER STABLE ids cpxName DROP COLUMN ids */ - { 189, -7 }, /* (264) cmd ::= ALTER STABLE ids cpxName ADD TAG columnlist */ - { 189, -7 }, /* (265) cmd ::= ALTER STABLE ids cpxName DROP TAG ids */ - { 189, -8 }, /* (266) cmd ::= ALTER STABLE ids cpxName CHANGE TAG ids ids */ - { 189, -3 }, /* (267) cmd ::= KILL CONNECTION INTEGER */ - { 189, -5 }, /* (268) cmd ::= KILL STREAM INTEGER COLON INTEGER */ - { 189, -5 }, /* (269) cmd ::= KILL QUERY INTEGER COLON INTEGER */ + { 238, -2 }, /* (173) from ::= FROM sub */ + { 255, -3 }, /* (174) sub ::= LP union RP */ + { 255, -4 }, /* (175) sub ::= LP union RP ids */ + { 255, -6 }, /* (176) sub ::= sub COMMA LP union RP ids */ + { 254, -2 }, /* (177) tablelist ::= ids cpxName */ + { 254, -3 }, /* (178) tablelist ::= ids cpxName ids */ + { 254, -4 }, /* (179) tablelist ::= tablelist COMMA ids cpxName */ + { 254, -5 }, /* (180) tablelist ::= tablelist COMMA ids cpxName ids */ + { 256, -1 }, /* (181) tmvar ::= VARIABLE */ + { 240, -4 }, /* (182) interval_opt ::= INTERVAL LP tmvar RP */ + { 240, -6 }, /* (183) interval_opt ::= INTERVAL LP tmvar COMMA tmvar RP */ + { 240, 0 }, /* (184) interval_opt ::= */ + { 241, 0 }, /* (185) session_option ::= */ + { 241, -7 }, /* (186) session_option ::= SESSION LP ids cpxName COMMA tmvar RP */ + { 242, 0 }, /* (187) fill_opt ::= */ + { 242, -6 }, /* (188) fill_opt ::= FILL LP ID COMMA tagitemlist RP */ + { 242, -4 }, /* (189) fill_opt ::= FILL LP ID RP */ + { 243, -4 }, /* (190) sliding_opt ::= SLIDING LP tmvar RP */ + { 243, 0 }, /* (191) sliding_opt ::= */ + { 245, 0 }, /* (192) orderby_opt ::= */ + { 245, -3 }, /* (193) orderby_opt ::= ORDER BY sortlist */ + { 257, -4 }, /* (194) sortlist ::= sortlist COMMA item sortorder */ + { 257, -2 }, /* (195) sortlist ::= item sortorder */ + { 259, -2 }, /* (196) item ::= ids cpxName */ + { 260, -1 }, /* (197) sortorder ::= ASC */ + { 260, -1 }, /* (198) sortorder ::= DESC */ + { 260, 0 }, /* (199) sortorder ::= */ + { 244, 0 }, /* (200) groupby_opt ::= */ + { 244, -3 }, /* (201) groupby_opt ::= GROUP BY grouplist */ + { 261, -3 }, /* (202) grouplist ::= grouplist COMMA item */ + { 261, -1 }, /* (203) grouplist ::= item */ + { 246, 0 }, /* (204) having_opt ::= */ + { 246, -2 }, /* (205) having_opt ::= HAVING expr */ + { 248, 0 }, /* (206) limit_opt ::= */ + { 248, -2 }, /* (207) limit_opt ::= LIMIT signed */ + { 248, -4 }, /* (208) limit_opt ::= LIMIT signed OFFSET signed */ + { 248, -4 }, /* (209) limit_opt ::= LIMIT signed COMMA signed */ + { 247, 0 }, /* (210) slimit_opt ::= */ + { 247, -2 }, /* (211) slimit_opt ::= SLIMIT signed */ + { 247, -4 }, /* (212) slimit_opt ::= SLIMIT signed SOFFSET signed */ + { 247, -4 }, /* (213) slimit_opt ::= SLIMIT signed COMMA signed */ + { 239, 0 }, /* (214) where_opt ::= */ + { 239, -2 }, /* (215) where_opt ::= WHERE expr */ + { 252, -3 }, /* (216) expr ::= LP expr RP */ + { 252, -1 }, /* (217) expr ::= ID */ + { 252, -3 }, /* (218) expr ::= ID DOT ID */ + { 252, -3 }, /* (219) expr ::= ID DOT STAR */ + { 252, -1 }, /* (220) expr ::= INTEGER */ + { 252, -2 }, /* (221) expr ::= MINUS INTEGER */ + { 252, -2 }, /* (222) expr ::= PLUS INTEGER */ + { 252, -1 }, /* (223) expr ::= FLOAT */ + { 252, -2 }, /* (224) expr ::= MINUS FLOAT */ + { 252, -2 }, /* (225) expr ::= PLUS FLOAT */ + { 252, -1 }, /* (226) expr ::= STRING */ + { 252, -1 }, /* (227) expr ::= NOW */ + { 252, -1 }, /* (228) expr ::= VARIABLE */ + { 252, -2 }, /* (229) expr ::= PLUS VARIABLE */ + { 252, -2 }, /* (230) expr ::= MINUS VARIABLE */ + { 252, -1 }, /* (231) expr ::= BOOL */ + { 252, -1 }, /* (232) expr ::= NULL */ + { 252, -4 }, /* (233) expr ::= ID LP exprlist RP */ + { 252, -4 }, /* (234) expr ::= ID LP STAR RP */ + { 252, -3 }, /* (235) expr ::= expr IS NULL */ + { 252, -4 }, /* (236) expr ::= expr IS NOT NULL */ + { 252, -3 }, /* (237) expr ::= expr LT expr */ + { 252, -3 }, /* (238) expr ::= expr GT expr */ + { 252, -3 }, /* (239) expr ::= expr LE expr */ + { 252, -3 }, /* (240) expr ::= expr GE expr */ + { 252, -3 }, /* (241) expr ::= expr NE expr */ + { 252, -3 }, /* (242) expr ::= expr EQ expr */ + { 252, -5 }, /* (243) expr ::= expr BETWEEN expr AND expr */ + { 252, -3 }, /* (244) expr ::= expr AND expr */ + { 252, -3 }, /* (245) expr ::= expr OR expr */ + { 252, -3 }, /* (246) expr ::= expr PLUS expr */ + { 252, -3 }, /* (247) expr ::= expr MINUS expr */ + { 252, -3 }, /* (248) expr ::= expr STAR expr */ + { 252, -3 }, /* (249) expr ::= expr SLASH expr */ + { 252, -3 }, /* (250) expr ::= expr REM expr */ + { 252, -3 }, /* (251) expr ::= expr LIKE expr */ + { 252, -5 }, /* (252) expr ::= expr IN LP exprlist RP */ + { 262, -3 }, /* (253) exprlist ::= exprlist COMMA expritem */ + { 262, -1 }, /* (254) exprlist ::= expritem */ + { 263, -1 }, /* (255) expritem ::= expr */ + { 263, 0 }, /* (256) expritem ::= */ + { 189, -3 }, /* (257) cmd ::= RESET QUERY CACHE */ + { 189, -3 }, /* (258) cmd ::= SYNCDB ids REPLICA */ + { 189, -7 }, /* (259) cmd ::= ALTER TABLE ids cpxName ADD COLUMN columnlist */ + { 189, -7 }, /* (260) cmd ::= ALTER TABLE ids cpxName DROP COLUMN ids */ + { 189, -7 }, /* (261) cmd ::= ALTER TABLE ids cpxName ADD TAG columnlist */ + { 189, -7 }, /* (262) cmd ::= ALTER TABLE ids cpxName DROP TAG ids */ + { 189, -8 }, /* (263) cmd ::= ALTER TABLE ids cpxName CHANGE TAG ids ids */ + { 189, -9 }, /* (264) cmd ::= ALTER TABLE ids cpxName SET TAG ids EQ tagitem */ + { 189, -7 }, /* (265) cmd ::= ALTER STABLE ids cpxName ADD COLUMN columnlist */ + { 189, -7 }, /* (266) cmd ::= ALTER STABLE ids cpxName DROP COLUMN ids */ + { 189, -7 }, /* (267) cmd ::= ALTER STABLE ids cpxName ADD TAG columnlist */ + { 189, -7 }, /* (268) cmd ::= ALTER STABLE ids cpxName DROP TAG ids */ + { 189, -8 }, /* (269) cmd ::= ALTER STABLE ids cpxName CHANGE TAG ids ids */ + { 189, -3 }, /* (270) cmd ::= KILL CONNECTION INTEGER */ + { 189, -5 }, /* (271) cmd ::= KILL STREAM INTEGER COLON INTEGER */ + { 189, -5 }, /* (272) cmd ::= KILL QUERY INTEGER COLON INTEGER */ }; static void yy_accept(yyParser*); /* Forward Declaration */ @@ -2277,13 +2290,13 @@ static void yy_reduce( break; case 44: /* cmd ::= ALTER DATABASE ids alter_db_optr */ case 45: /* cmd ::= ALTER TOPIC ids alter_topic_optr */ yytestcase(yyruleno==45); -{ SStrToken t = {0}; setCreateDbInfo(pInfo, TSDB_SQL_ALTER_DB, &yymsp[-1].minor.yy0, &yymsp[0].minor.yy322, &t);} +{ SStrToken t = {0}; setCreateDbInfo(pInfo, TSDB_SQL_ALTER_DB, &yymsp[-1].minor.yy0, &yymsp[0].minor.yy526, &t);} break; case 46: /* cmd ::= ALTER ACCOUNT ids acct_optr */ -{ setCreateAcctSql(pInfo, TSDB_SQL_ALTER_ACCT, &yymsp[-1].minor.yy0, NULL, &yymsp[0].minor.yy351);} +{ setCreateAcctSql(pInfo, TSDB_SQL_ALTER_ACCT, &yymsp[-1].minor.yy0, NULL, &yymsp[0].minor.yy187);} break; case 47: /* cmd ::= ALTER ACCOUNT ids PASS ids acct_optr */ -{ setCreateAcctSql(pInfo, TSDB_SQL_ALTER_ACCT, &yymsp[-3].minor.yy0, &yymsp[-1].minor.yy0, &yymsp[0].minor.yy351);} +{ setCreateAcctSql(pInfo, TSDB_SQL_ALTER_ACCT, &yymsp[-3].minor.yy0, &yymsp[-1].minor.yy0, &yymsp[0].minor.yy187);} break; case 48: /* ids ::= ID */ case 49: /* ids ::= STRING */ yytestcase(yyruleno==49); @@ -2305,11 +2318,11 @@ static void yy_reduce( { setDCLSqlElems(pInfo, TSDB_SQL_CREATE_DNODE, 1, &yymsp[0].minor.yy0);} break; case 55: /* cmd ::= CREATE ACCOUNT ids PASS ids acct_optr */ -{ setCreateAcctSql(pInfo, TSDB_SQL_CREATE_ACCT, &yymsp[-3].minor.yy0, &yymsp[-1].minor.yy0, &yymsp[0].minor.yy351);} +{ setCreateAcctSql(pInfo, TSDB_SQL_CREATE_ACCT, &yymsp[-3].minor.yy0, &yymsp[-1].minor.yy0, &yymsp[0].minor.yy187);} break; case 56: /* cmd ::= CREATE DATABASE ifnotexists ids db_optr */ case 57: /* cmd ::= CREATE TOPIC ifnotexists ids topic_optr */ yytestcase(yyruleno==57); -{ setCreateDbInfo(pInfo, TSDB_SQL_CREATE_DB, &yymsp[-1].minor.yy0, &yymsp[0].minor.yy322, &yymsp[-2].minor.yy0);} +{ setCreateDbInfo(pInfo, TSDB_SQL_CREATE_DB, &yymsp[-1].minor.yy0, &yymsp[0].minor.yy526, &yymsp[-2].minor.yy0);} break; case 58: /* cmd ::= CREATE USER ids PASS ids */ { setCreateUserSql(pInfo, &yymsp[-2].minor.yy0, &yymsp[0].minor.yy0);} @@ -2338,20 +2351,20 @@ static void yy_reduce( break; case 77: /* acct_optr ::= pps tseries storage streams qtime dbs users conns state */ { - yylhsminor.yy351.maxUsers = (yymsp[-2].minor.yy0.n>0)?atoi(yymsp[-2].minor.yy0.z):-1; - yylhsminor.yy351.maxDbs = (yymsp[-3].minor.yy0.n>0)?atoi(yymsp[-3].minor.yy0.z):-1; - yylhsminor.yy351.maxTimeSeries = (yymsp[-7].minor.yy0.n>0)?atoi(yymsp[-7].minor.yy0.z):-1; - yylhsminor.yy351.maxStreams = (yymsp[-5].minor.yy0.n>0)?atoi(yymsp[-5].minor.yy0.z):-1; - yylhsminor.yy351.maxPointsPerSecond = (yymsp[-8].minor.yy0.n>0)?atoi(yymsp[-8].minor.yy0.z):-1; - yylhsminor.yy351.maxStorage = (yymsp[-6].minor.yy0.n>0)?strtoll(yymsp[-6].minor.yy0.z, NULL, 10):-1; - yylhsminor.yy351.maxQueryTime = (yymsp[-4].minor.yy0.n>0)?strtoll(yymsp[-4].minor.yy0.z, NULL, 10):-1; - yylhsminor.yy351.maxConnections = (yymsp[-1].minor.yy0.n>0)?atoi(yymsp[-1].minor.yy0.z):-1; - yylhsminor.yy351.stat = yymsp[0].minor.yy0; -} - yymsp[-8].minor.yy351 = yylhsminor.yy351; + yylhsminor.yy187.maxUsers = (yymsp[-2].minor.yy0.n>0)?atoi(yymsp[-2].minor.yy0.z):-1; + yylhsminor.yy187.maxDbs = (yymsp[-3].minor.yy0.n>0)?atoi(yymsp[-3].minor.yy0.z):-1; + yylhsminor.yy187.maxTimeSeries = (yymsp[-7].minor.yy0.n>0)?atoi(yymsp[-7].minor.yy0.z):-1; + yylhsminor.yy187.maxStreams = (yymsp[-5].minor.yy0.n>0)?atoi(yymsp[-5].minor.yy0.z):-1; + yylhsminor.yy187.maxPointsPerSecond = (yymsp[-8].minor.yy0.n>0)?atoi(yymsp[-8].minor.yy0.z):-1; + yylhsminor.yy187.maxStorage = (yymsp[-6].minor.yy0.n>0)?strtoll(yymsp[-6].minor.yy0.z, NULL, 10):-1; + yylhsminor.yy187.maxQueryTime = (yymsp[-4].minor.yy0.n>0)?strtoll(yymsp[-4].minor.yy0.z, NULL, 10):-1; + yylhsminor.yy187.maxConnections = (yymsp[-1].minor.yy0.n>0)?atoi(yymsp[-1].minor.yy0.z):-1; + yylhsminor.yy187.stat = yymsp[0].minor.yy0; +} + yymsp[-8].minor.yy187 = yylhsminor.yy187; break; case 78: /* keep ::= KEEP tagitemlist */ -{ yymsp[-1].minor.yy159 = yymsp[0].minor.yy159; } +{ yymsp[-1].minor.yy285 = yymsp[0].minor.yy285; } break; case 79: /* cache ::= CACHE INTEGER */ case 80: /* replica ::= REPLICA INTEGER */ yytestcase(yyruleno==80); @@ -2371,234 +2384,234 @@ static void yy_reduce( { yymsp[-1].minor.yy0 = yymsp[0].minor.yy0; } break; case 94: /* db_optr ::= */ -{setDefaultCreateDbOption(&yymsp[1].minor.yy322); yymsp[1].minor.yy322.dbType = TSDB_DB_TYPE_DEFAULT;} +{setDefaultCreateDbOption(&yymsp[1].minor.yy526); yymsp[1].minor.yy526.dbType = TSDB_DB_TYPE_DEFAULT;} break; case 95: /* db_optr ::= db_optr cache */ -{ yylhsminor.yy322 = yymsp[-1].minor.yy322; yylhsminor.yy322.cacheBlockSize = strtol(yymsp[0].minor.yy0.z, NULL, 10); } - yymsp[-1].minor.yy322 = yylhsminor.yy322; +{ yylhsminor.yy526 = yymsp[-1].minor.yy526; yylhsminor.yy526.cacheBlockSize = strtol(yymsp[0].minor.yy0.z, NULL, 10); } + yymsp[-1].minor.yy526 = yylhsminor.yy526; break; case 96: /* db_optr ::= db_optr replica */ case 113: /* alter_db_optr ::= alter_db_optr replica */ yytestcase(yyruleno==113); -{ yylhsminor.yy322 = yymsp[-1].minor.yy322; yylhsminor.yy322.replica = strtol(yymsp[0].minor.yy0.z, NULL, 10); } - yymsp[-1].minor.yy322 = yylhsminor.yy322; +{ yylhsminor.yy526 = yymsp[-1].minor.yy526; yylhsminor.yy526.replica = strtol(yymsp[0].minor.yy0.z, NULL, 10); } + yymsp[-1].minor.yy526 = yylhsminor.yy526; break; case 97: /* db_optr ::= db_optr quorum */ case 114: /* alter_db_optr ::= alter_db_optr quorum */ yytestcase(yyruleno==114); -{ yylhsminor.yy322 = yymsp[-1].minor.yy322; yylhsminor.yy322.quorum = strtol(yymsp[0].minor.yy0.z, NULL, 10); } - yymsp[-1].minor.yy322 = yylhsminor.yy322; +{ yylhsminor.yy526 = yymsp[-1].minor.yy526; yylhsminor.yy526.quorum = strtol(yymsp[0].minor.yy0.z, NULL, 10); } + yymsp[-1].minor.yy526 = yylhsminor.yy526; break; case 98: /* db_optr ::= db_optr days */ -{ yylhsminor.yy322 = yymsp[-1].minor.yy322; yylhsminor.yy322.daysPerFile = strtol(yymsp[0].minor.yy0.z, NULL, 10); } - yymsp[-1].minor.yy322 = yylhsminor.yy322; +{ yylhsminor.yy526 = yymsp[-1].minor.yy526; yylhsminor.yy526.daysPerFile = strtol(yymsp[0].minor.yy0.z, NULL, 10); } + yymsp[-1].minor.yy526 = yylhsminor.yy526; break; case 99: /* db_optr ::= db_optr minrows */ -{ yylhsminor.yy322 = yymsp[-1].minor.yy322; yylhsminor.yy322.minRowsPerBlock = strtod(yymsp[0].minor.yy0.z, NULL); } - yymsp[-1].minor.yy322 = yylhsminor.yy322; +{ yylhsminor.yy526 = yymsp[-1].minor.yy526; yylhsminor.yy526.minRowsPerBlock = strtod(yymsp[0].minor.yy0.z, NULL); } + yymsp[-1].minor.yy526 = yylhsminor.yy526; break; case 100: /* db_optr ::= db_optr maxrows */ -{ yylhsminor.yy322 = yymsp[-1].minor.yy322; yylhsminor.yy322.maxRowsPerBlock = strtod(yymsp[0].minor.yy0.z, NULL); } - yymsp[-1].minor.yy322 = yylhsminor.yy322; +{ yylhsminor.yy526 = yymsp[-1].minor.yy526; yylhsminor.yy526.maxRowsPerBlock = strtod(yymsp[0].minor.yy0.z, NULL); } + yymsp[-1].minor.yy526 = yylhsminor.yy526; break; case 101: /* db_optr ::= db_optr blocks */ case 116: /* alter_db_optr ::= alter_db_optr blocks */ yytestcase(yyruleno==116); -{ yylhsminor.yy322 = yymsp[-1].minor.yy322; yylhsminor.yy322.numOfBlocks = strtol(yymsp[0].minor.yy0.z, NULL, 10); } - yymsp[-1].minor.yy322 = yylhsminor.yy322; +{ yylhsminor.yy526 = yymsp[-1].minor.yy526; yylhsminor.yy526.numOfBlocks = strtol(yymsp[0].minor.yy0.z, NULL, 10); } + yymsp[-1].minor.yy526 = yylhsminor.yy526; break; case 102: /* db_optr ::= db_optr ctime */ -{ yylhsminor.yy322 = yymsp[-1].minor.yy322; yylhsminor.yy322.commitTime = strtol(yymsp[0].minor.yy0.z, NULL, 10); } - yymsp[-1].minor.yy322 = yylhsminor.yy322; +{ yylhsminor.yy526 = yymsp[-1].minor.yy526; yylhsminor.yy526.commitTime = strtol(yymsp[0].minor.yy0.z, NULL, 10); } + yymsp[-1].minor.yy526 = yylhsminor.yy526; break; case 103: /* db_optr ::= db_optr wal */ case 118: /* alter_db_optr ::= alter_db_optr wal */ yytestcase(yyruleno==118); -{ yylhsminor.yy322 = yymsp[-1].minor.yy322; yylhsminor.yy322.walLevel = strtol(yymsp[0].minor.yy0.z, NULL, 10); } - yymsp[-1].minor.yy322 = yylhsminor.yy322; +{ yylhsminor.yy526 = yymsp[-1].minor.yy526; yylhsminor.yy526.walLevel = strtol(yymsp[0].minor.yy0.z, NULL, 10); } + yymsp[-1].minor.yy526 = yylhsminor.yy526; break; case 104: /* db_optr ::= db_optr fsync */ case 119: /* alter_db_optr ::= alter_db_optr fsync */ yytestcase(yyruleno==119); -{ yylhsminor.yy322 = yymsp[-1].minor.yy322; yylhsminor.yy322.fsyncPeriod = strtol(yymsp[0].minor.yy0.z, NULL, 10); } - yymsp[-1].minor.yy322 = yylhsminor.yy322; +{ yylhsminor.yy526 = yymsp[-1].minor.yy526; yylhsminor.yy526.fsyncPeriod = strtol(yymsp[0].minor.yy0.z, NULL, 10); } + yymsp[-1].minor.yy526 = yylhsminor.yy526; break; case 105: /* db_optr ::= db_optr comp */ case 117: /* alter_db_optr ::= alter_db_optr comp */ yytestcase(yyruleno==117); -{ yylhsminor.yy322 = yymsp[-1].minor.yy322; yylhsminor.yy322.compressionLevel = strtol(yymsp[0].minor.yy0.z, NULL, 10); } - yymsp[-1].minor.yy322 = yylhsminor.yy322; +{ yylhsminor.yy526 = yymsp[-1].minor.yy526; yylhsminor.yy526.compressionLevel = strtol(yymsp[0].minor.yy0.z, NULL, 10); } + yymsp[-1].minor.yy526 = yylhsminor.yy526; break; case 106: /* db_optr ::= db_optr prec */ -{ yylhsminor.yy322 = yymsp[-1].minor.yy322; yylhsminor.yy322.precision = yymsp[0].minor.yy0; } - yymsp[-1].minor.yy322 = yylhsminor.yy322; +{ yylhsminor.yy526 = yymsp[-1].minor.yy526; yylhsminor.yy526.precision = yymsp[0].minor.yy0; } + yymsp[-1].minor.yy526 = yylhsminor.yy526; break; case 107: /* db_optr ::= db_optr keep */ case 115: /* alter_db_optr ::= alter_db_optr keep */ yytestcase(yyruleno==115); -{ yylhsminor.yy322 = yymsp[-1].minor.yy322; yylhsminor.yy322.keep = yymsp[0].minor.yy159; } - yymsp[-1].minor.yy322 = yylhsminor.yy322; +{ yylhsminor.yy526 = yymsp[-1].minor.yy526; yylhsminor.yy526.keep = yymsp[0].minor.yy285; } + yymsp[-1].minor.yy526 = yylhsminor.yy526; break; case 108: /* db_optr ::= db_optr update */ case 120: /* alter_db_optr ::= alter_db_optr update */ yytestcase(yyruleno==120); -{ yylhsminor.yy322 = yymsp[-1].minor.yy322; yylhsminor.yy322.update = strtol(yymsp[0].minor.yy0.z, NULL, 10); } - yymsp[-1].minor.yy322 = yylhsminor.yy322; +{ yylhsminor.yy526 = yymsp[-1].minor.yy526; yylhsminor.yy526.update = strtol(yymsp[0].minor.yy0.z, NULL, 10); } + yymsp[-1].minor.yy526 = yylhsminor.yy526; break; case 109: /* db_optr ::= db_optr cachelast */ case 121: /* alter_db_optr ::= alter_db_optr cachelast */ yytestcase(yyruleno==121); -{ yylhsminor.yy322 = yymsp[-1].minor.yy322; yylhsminor.yy322.cachelast = strtol(yymsp[0].minor.yy0.z, NULL, 10); } - yymsp[-1].minor.yy322 = yylhsminor.yy322; +{ yylhsminor.yy526 = yymsp[-1].minor.yy526; yylhsminor.yy526.cachelast = strtol(yymsp[0].minor.yy0.z, NULL, 10); } + yymsp[-1].minor.yy526 = yylhsminor.yy526; break; case 110: /* topic_optr ::= db_optr */ case 122: /* alter_topic_optr ::= alter_db_optr */ yytestcase(yyruleno==122); -{ yylhsminor.yy322 = yymsp[0].minor.yy322; yylhsminor.yy322.dbType = TSDB_DB_TYPE_TOPIC; } - yymsp[0].minor.yy322 = yylhsminor.yy322; +{ yylhsminor.yy526 = yymsp[0].minor.yy526; yylhsminor.yy526.dbType = TSDB_DB_TYPE_TOPIC; } + yymsp[0].minor.yy526 = yylhsminor.yy526; break; case 111: /* topic_optr ::= topic_optr partitions */ case 123: /* alter_topic_optr ::= alter_topic_optr partitions */ yytestcase(yyruleno==123); -{ yylhsminor.yy322 = yymsp[-1].minor.yy322; yylhsminor.yy322.partitions = strtol(yymsp[0].minor.yy0.z, NULL, 10); } - yymsp[-1].minor.yy322 = yylhsminor.yy322; +{ yylhsminor.yy526 = yymsp[-1].minor.yy526; yylhsminor.yy526.partitions = strtol(yymsp[0].minor.yy0.z, NULL, 10); } + yymsp[-1].minor.yy526 = yylhsminor.yy526; break; case 112: /* alter_db_optr ::= */ -{ setDefaultCreateDbOption(&yymsp[1].minor.yy322); yymsp[1].minor.yy322.dbType = TSDB_DB_TYPE_DEFAULT;} +{ setDefaultCreateDbOption(&yymsp[1].minor.yy526); yymsp[1].minor.yy526.dbType = TSDB_DB_TYPE_DEFAULT;} break; case 124: /* typename ::= ids */ { yymsp[0].minor.yy0.type = 0; - tSetColumnType (&yylhsminor.yy407, &yymsp[0].minor.yy0); + tSetColumnType (&yylhsminor.yy295, &yymsp[0].minor.yy0); } - yymsp[0].minor.yy407 = yylhsminor.yy407; + yymsp[0].minor.yy295 = yylhsminor.yy295; break; case 125: /* typename ::= ids LP signed RP */ { - if (yymsp[-1].minor.yy317 <= 0) { + if (yymsp[-1].minor.yy525 <= 0) { yymsp[-3].minor.yy0.type = 0; - tSetColumnType(&yylhsminor.yy407, &yymsp[-3].minor.yy0); + tSetColumnType(&yylhsminor.yy295, &yymsp[-3].minor.yy0); } else { - yymsp[-3].minor.yy0.type = -yymsp[-1].minor.yy317; // negative value of name length - tSetColumnType(&yylhsminor.yy407, &yymsp[-3].minor.yy0); + yymsp[-3].minor.yy0.type = -yymsp[-1].minor.yy525; // negative value of name length + tSetColumnType(&yylhsminor.yy295, &yymsp[-3].minor.yy0); } } - yymsp[-3].minor.yy407 = yylhsminor.yy407; + yymsp[-3].minor.yy295 = yylhsminor.yy295; break; case 126: /* typename ::= ids UNSIGNED */ { yymsp[-1].minor.yy0.type = 0; yymsp[-1].minor.yy0.n = ((yymsp[0].minor.yy0.z + yymsp[0].minor.yy0.n) - yymsp[-1].minor.yy0.z); - tSetColumnType (&yylhsminor.yy407, &yymsp[-1].minor.yy0); + tSetColumnType (&yylhsminor.yy295, &yymsp[-1].minor.yy0); } - yymsp[-1].minor.yy407 = yylhsminor.yy407; + yymsp[-1].minor.yy295 = yylhsminor.yy295; break; case 127: /* signed ::= INTEGER */ -{ yylhsminor.yy317 = strtol(yymsp[0].minor.yy0.z, NULL, 10); } - yymsp[0].minor.yy317 = yylhsminor.yy317; +{ yylhsminor.yy525 = strtol(yymsp[0].minor.yy0.z, NULL, 10); } + yymsp[0].minor.yy525 = yylhsminor.yy525; break; case 128: /* signed ::= PLUS INTEGER */ -{ yymsp[-1].minor.yy317 = strtol(yymsp[0].minor.yy0.z, NULL, 10); } +{ yymsp[-1].minor.yy525 = strtol(yymsp[0].minor.yy0.z, NULL, 10); } break; case 129: /* signed ::= MINUS INTEGER */ -{ yymsp[-1].minor.yy317 = -strtol(yymsp[0].minor.yy0.z, NULL, 10);} +{ yymsp[-1].minor.yy525 = -strtol(yymsp[0].minor.yy0.z, NULL, 10);} break; case 133: /* cmd ::= CREATE TABLE create_table_list */ -{ pInfo->type = TSDB_SQL_CREATE_TABLE; pInfo->pCreateTableInfo = yymsp[0].minor.yy14;} +{ pInfo->type = TSDB_SQL_CREATE_TABLE; pInfo->pCreateTableInfo = yymsp[0].minor.yy470;} break; case 134: /* create_table_list ::= create_from_stable */ { SCreateTableSql* pCreateTable = calloc(1, sizeof(SCreateTableSql)); pCreateTable->childTableInfo = taosArrayInit(4, sizeof(SCreatedTableInfo)); - taosArrayPush(pCreateTable->childTableInfo, &yymsp[0].minor.yy206); + taosArrayPush(pCreateTable->childTableInfo, &yymsp[0].minor.yy96); pCreateTable->type = TSQL_CREATE_TABLE_FROM_STABLE; - yylhsminor.yy14 = pCreateTable; + yylhsminor.yy470 = pCreateTable; } - yymsp[0].minor.yy14 = yylhsminor.yy14; + yymsp[0].minor.yy470 = yylhsminor.yy470; break; case 135: /* create_table_list ::= create_table_list create_from_stable */ { - taosArrayPush(yymsp[-1].minor.yy14->childTableInfo, &yymsp[0].minor.yy206); - yylhsminor.yy14 = yymsp[-1].minor.yy14; + taosArrayPush(yymsp[-1].minor.yy470->childTableInfo, &yymsp[0].minor.yy96); + yylhsminor.yy470 = yymsp[-1].minor.yy470; } - yymsp[-1].minor.yy14 = yylhsminor.yy14; + yymsp[-1].minor.yy470 = yylhsminor.yy470; break; case 136: /* create_table_args ::= ifnotexists ids cpxName LP columnlist RP */ { - yylhsminor.yy14 = tSetCreateTableInfo(yymsp[-1].minor.yy159, NULL, NULL, TSQL_CREATE_TABLE); - setSqlInfo(pInfo, yylhsminor.yy14, NULL, TSDB_SQL_CREATE_TABLE); + yylhsminor.yy470 = tSetCreateTableInfo(yymsp[-1].minor.yy285, NULL, NULL, TSQL_CREATE_TABLE); + setSqlInfo(pInfo, yylhsminor.yy470, NULL, TSDB_SQL_CREATE_TABLE); yymsp[-4].minor.yy0.n += yymsp[-3].minor.yy0.n; setCreatedTableName(pInfo, &yymsp[-4].minor.yy0, &yymsp[-5].minor.yy0); } - yymsp[-5].minor.yy14 = yylhsminor.yy14; + yymsp[-5].minor.yy470 = yylhsminor.yy470; break; case 137: /* create_stable_args ::= ifnotexists ids cpxName LP columnlist RP TAGS LP columnlist RP */ { - yylhsminor.yy14 = tSetCreateTableInfo(yymsp[-5].minor.yy159, yymsp[-1].minor.yy159, NULL, TSQL_CREATE_STABLE); - setSqlInfo(pInfo, yylhsminor.yy14, NULL, TSDB_SQL_CREATE_TABLE); + yylhsminor.yy470 = tSetCreateTableInfo(yymsp[-5].minor.yy285, yymsp[-1].minor.yy285, NULL, TSQL_CREATE_STABLE); + setSqlInfo(pInfo, yylhsminor.yy470, NULL, TSDB_SQL_CREATE_TABLE); yymsp[-8].minor.yy0.n += yymsp[-7].minor.yy0.n; setCreatedTableName(pInfo, &yymsp[-8].minor.yy0, &yymsp[-9].minor.yy0); } - yymsp[-9].minor.yy14 = yylhsminor.yy14; + yymsp[-9].minor.yy470 = yylhsminor.yy470; break; case 138: /* create_from_stable ::= ifnotexists ids cpxName USING ids cpxName TAGS LP tagitemlist RP */ { yymsp[-5].minor.yy0.n += yymsp[-4].minor.yy0.n; yymsp[-8].minor.yy0.n += yymsp[-7].minor.yy0.n; - yylhsminor.yy206 = createNewChildTableInfo(&yymsp[-5].minor.yy0, NULL, yymsp[-1].minor.yy159, &yymsp[-8].minor.yy0, &yymsp[-9].minor.yy0); + yylhsminor.yy96 = createNewChildTableInfo(&yymsp[-5].minor.yy0, NULL, yymsp[-1].minor.yy285, &yymsp[-8].minor.yy0, &yymsp[-9].minor.yy0); } - yymsp[-9].minor.yy206 = yylhsminor.yy206; + yymsp[-9].minor.yy96 = yylhsminor.yy96; break; case 139: /* create_from_stable ::= ifnotexists ids cpxName USING ids cpxName LP tagNamelist RP TAGS LP tagitemlist RP */ { yymsp[-8].minor.yy0.n += yymsp[-7].minor.yy0.n; yymsp[-11].minor.yy0.n += yymsp[-10].minor.yy0.n; - yylhsminor.yy206 = createNewChildTableInfo(&yymsp[-8].minor.yy0, yymsp[-5].minor.yy159, yymsp[-1].minor.yy159, &yymsp[-11].minor.yy0, &yymsp[-12].minor.yy0); + yylhsminor.yy96 = createNewChildTableInfo(&yymsp[-8].minor.yy0, yymsp[-5].minor.yy285, yymsp[-1].minor.yy285, &yymsp[-11].minor.yy0, &yymsp[-12].minor.yy0); } - yymsp[-12].minor.yy206 = yylhsminor.yy206; + yymsp[-12].minor.yy96 = yylhsminor.yy96; break; case 140: /* tagNamelist ::= tagNamelist COMMA ids */ -{taosArrayPush(yymsp[-2].minor.yy159, &yymsp[0].minor.yy0); yylhsminor.yy159 = yymsp[-2].minor.yy159; } - yymsp[-2].minor.yy159 = yylhsminor.yy159; +{taosArrayPush(yymsp[-2].minor.yy285, &yymsp[0].minor.yy0); yylhsminor.yy285 = yymsp[-2].minor.yy285; } + yymsp[-2].minor.yy285 = yylhsminor.yy285; break; case 141: /* tagNamelist ::= ids */ -{yylhsminor.yy159 = taosArrayInit(4, sizeof(SStrToken)); taosArrayPush(yylhsminor.yy159, &yymsp[0].minor.yy0);} - yymsp[0].minor.yy159 = yylhsminor.yy159; +{yylhsminor.yy285 = taosArrayInit(4, sizeof(SStrToken)); taosArrayPush(yylhsminor.yy285, &yymsp[0].minor.yy0);} + yymsp[0].minor.yy285 = yylhsminor.yy285; break; case 142: /* create_table_args ::= ifnotexists ids cpxName AS select */ { - yylhsminor.yy14 = tSetCreateTableInfo(NULL, NULL, yymsp[0].minor.yy116, TSQL_CREATE_STREAM); - setSqlInfo(pInfo, yylhsminor.yy14, NULL, TSDB_SQL_CREATE_TABLE); + yylhsminor.yy470 = tSetCreateTableInfo(NULL, NULL, yymsp[0].minor.yy344, TSQL_CREATE_STREAM); + setSqlInfo(pInfo, yylhsminor.yy470, NULL, TSDB_SQL_CREATE_TABLE); yymsp[-3].minor.yy0.n += yymsp[-2].minor.yy0.n; setCreatedTableName(pInfo, &yymsp[-3].minor.yy0, &yymsp[-4].minor.yy0); } - yymsp[-4].minor.yy14 = yylhsminor.yy14; + yymsp[-4].minor.yy470 = yylhsminor.yy470; break; case 143: /* columnlist ::= columnlist COMMA column */ -{taosArrayPush(yymsp[-2].minor.yy159, &yymsp[0].minor.yy407); yylhsminor.yy159 = yymsp[-2].minor.yy159; } - yymsp[-2].minor.yy159 = yylhsminor.yy159; +{taosArrayPush(yymsp[-2].minor.yy285, &yymsp[0].minor.yy295); yylhsminor.yy285 = yymsp[-2].minor.yy285; } + yymsp[-2].minor.yy285 = yylhsminor.yy285; break; case 144: /* columnlist ::= column */ -{yylhsminor.yy159 = taosArrayInit(4, sizeof(TAOS_FIELD)); taosArrayPush(yylhsminor.yy159, &yymsp[0].minor.yy407);} - yymsp[0].minor.yy159 = yylhsminor.yy159; +{yylhsminor.yy285 = taosArrayInit(4, sizeof(TAOS_FIELD)); taosArrayPush(yylhsminor.yy285, &yymsp[0].minor.yy295);} + yymsp[0].minor.yy285 = yylhsminor.yy285; break; case 145: /* column ::= ids typename */ { - tSetColumnInfo(&yylhsminor.yy407, &yymsp[-1].minor.yy0, &yymsp[0].minor.yy407); + tSetColumnInfo(&yylhsminor.yy295, &yymsp[-1].minor.yy0, &yymsp[0].minor.yy295); } - yymsp[-1].minor.yy407 = yylhsminor.yy407; + yymsp[-1].minor.yy295 = yylhsminor.yy295; break; case 146: /* tagitemlist ::= tagitemlist COMMA tagitem */ -{ yylhsminor.yy159 = tVariantListAppend(yymsp[-2].minor.yy159, &yymsp[0].minor.yy488, -1); } - yymsp[-2].minor.yy159 = yylhsminor.yy159; +{ yylhsminor.yy285 = tVariantListAppend(yymsp[-2].minor.yy285, &yymsp[0].minor.yy362, -1); } + yymsp[-2].minor.yy285 = yylhsminor.yy285; break; case 147: /* tagitemlist ::= tagitem */ -{ yylhsminor.yy159 = tVariantListAppend(NULL, &yymsp[0].minor.yy488, -1); } - yymsp[0].minor.yy159 = yylhsminor.yy159; +{ yylhsminor.yy285 = tVariantListAppend(NULL, &yymsp[0].minor.yy362, -1); } + yymsp[0].minor.yy285 = yylhsminor.yy285; break; case 148: /* tagitem ::= INTEGER */ case 149: /* tagitem ::= FLOAT */ yytestcase(yyruleno==149); case 150: /* tagitem ::= STRING */ yytestcase(yyruleno==150); case 151: /* tagitem ::= BOOL */ yytestcase(yyruleno==151); -{ toTSDBType(yymsp[0].minor.yy0.type); tVariantCreate(&yylhsminor.yy488, &yymsp[0].minor.yy0); } - yymsp[0].minor.yy488 = yylhsminor.yy488; +{ toTSDBType(yymsp[0].minor.yy0.type); tVariantCreate(&yylhsminor.yy362, &yymsp[0].minor.yy0); } + yymsp[0].minor.yy362 = yylhsminor.yy362; break; case 152: /* tagitem ::= NULL */ -{ yymsp[0].minor.yy0.type = 0; tVariantCreate(&yylhsminor.yy488, &yymsp[0].minor.yy0); } - yymsp[0].minor.yy488 = yylhsminor.yy488; +{ yymsp[0].minor.yy0.type = 0; tVariantCreate(&yylhsminor.yy362, &yymsp[0].minor.yy0); } + yymsp[0].minor.yy362 = yylhsminor.yy362; break; case 153: /* tagitem ::= MINUS INTEGER */ case 154: /* tagitem ::= MINUS FLOAT */ yytestcase(yyruleno==154); @@ -2608,56 +2621,56 @@ static void yy_reduce( yymsp[-1].minor.yy0.n += yymsp[0].minor.yy0.n; yymsp[-1].minor.yy0.type = yymsp[0].minor.yy0.type; toTSDBType(yymsp[-1].minor.yy0.type); - tVariantCreate(&yylhsminor.yy488, &yymsp[-1].minor.yy0); + tVariantCreate(&yylhsminor.yy362, &yymsp[-1].minor.yy0); } - yymsp[-1].minor.yy488 = yylhsminor.yy488; + yymsp[-1].minor.yy362 = yylhsminor.yy362; break; case 157: /* select ::= SELECT selcollist from where_opt interval_opt session_option fill_opt sliding_opt groupby_opt orderby_opt having_opt slimit_opt limit_opt */ { - yylhsminor.yy116 = tSetQuerySqlNode(&yymsp[-12].minor.yy0, yymsp[-11].minor.yy159, yymsp[-10].minor.yy236, yymsp[-9].minor.yy118, yymsp[-4].minor.yy159, yymsp[-3].minor.yy159, &yymsp[-8].minor.yy184, &yymsp[-7].minor.yy249, &yymsp[-5].minor.yy0, yymsp[-6].minor.yy159, &yymsp[0].minor.yy440, &yymsp[-1].minor.yy440, yymsp[-2].minor.yy118); + yylhsminor.yy344 = tSetQuerySqlNode(&yymsp[-12].minor.yy0, yymsp[-11].minor.yy285, yymsp[-10].minor.yy148, yymsp[-9].minor.yy178, yymsp[-4].minor.yy285, yymsp[-3].minor.yy285, &yymsp[-8].minor.yy376, &yymsp[-7].minor.yy523, &yymsp[-5].minor.yy0, yymsp[-6].minor.yy285, &yymsp[0].minor.yy438, &yymsp[-1].minor.yy438, yymsp[-2].minor.yy178); } - yymsp[-12].minor.yy116 = yylhsminor.yy116; + yymsp[-12].minor.yy344 = yylhsminor.yy344; break; case 158: /* select ::= LP select RP */ -{yymsp[-2].minor.yy116 = yymsp[-1].minor.yy116;} +{yymsp[-2].minor.yy344 = yymsp[-1].minor.yy344;} break; case 159: /* union ::= select */ -{ yylhsminor.yy159 = setSubclause(NULL, yymsp[0].minor.yy116); } - yymsp[0].minor.yy159 = yylhsminor.yy159; +{ yylhsminor.yy285 = setSubclause(NULL, yymsp[0].minor.yy344); } + yymsp[0].minor.yy285 = yylhsminor.yy285; break; case 160: /* union ::= union UNION ALL select */ -{ yylhsminor.yy159 = appendSelectClause(yymsp[-3].minor.yy159, yymsp[0].minor.yy116); } - yymsp[-3].minor.yy159 = yylhsminor.yy159; +{ yylhsminor.yy285 = appendSelectClause(yymsp[-3].minor.yy285, yymsp[0].minor.yy344); } + yymsp[-3].minor.yy285 = yylhsminor.yy285; break; case 161: /* cmd ::= union */ -{ setSqlInfo(pInfo, yymsp[0].minor.yy159, NULL, TSDB_SQL_SELECT); } +{ setSqlInfo(pInfo, yymsp[0].minor.yy285, NULL, TSDB_SQL_SELECT); } break; case 162: /* select ::= SELECT selcollist */ { - yylhsminor.yy116 = tSetQuerySqlNode(&yymsp[-1].minor.yy0, yymsp[0].minor.yy159, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL); + yylhsminor.yy344 = tSetQuerySqlNode(&yymsp[-1].minor.yy0, yymsp[0].minor.yy285, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL); } - yymsp[-1].minor.yy116 = yylhsminor.yy116; + yymsp[-1].minor.yy344 = yylhsminor.yy344; break; case 163: /* sclp ::= selcollist COMMA */ -{yylhsminor.yy159 = yymsp[-1].minor.yy159;} - yymsp[-1].minor.yy159 = yylhsminor.yy159; +{yylhsminor.yy285 = yymsp[-1].minor.yy285;} + yymsp[-1].minor.yy285 = yylhsminor.yy285; break; case 164: /* sclp ::= */ - case 189: /* orderby_opt ::= */ yytestcase(yyruleno==189); -{yymsp[1].minor.yy159 = 0;} + case 192: /* orderby_opt ::= */ yytestcase(yyruleno==192); +{yymsp[1].minor.yy285 = 0;} break; case 165: /* selcollist ::= sclp distinct expr as */ { - yylhsminor.yy159 = tSqlExprListAppend(yymsp[-3].minor.yy159, yymsp[-1].minor.yy118, yymsp[-2].minor.yy0.n? &yymsp[-2].minor.yy0:0, yymsp[0].minor.yy0.n?&yymsp[0].minor.yy0:0); + yylhsminor.yy285 = tSqlExprListAppend(yymsp[-3].minor.yy285, yymsp[-1].minor.yy178, yymsp[-2].minor.yy0.n? &yymsp[-2].minor.yy0:0, yymsp[0].minor.yy0.n?&yymsp[0].minor.yy0:0); } - yymsp[-3].minor.yy159 = yylhsminor.yy159; + yymsp[-3].minor.yy285 = yylhsminor.yy285; break; case 166: /* selcollist ::= sclp STAR */ { tSqlExpr *pNode = tSqlExprCreateIdValue(NULL, TK_ALL); - yylhsminor.yy159 = tSqlExprListAppend(yymsp[-1].minor.yy159, pNode, 0, 0); + yylhsminor.yy285 = tSqlExprListAppend(yymsp[-1].minor.yy285, pNode, 0, 0); } - yymsp[-1].minor.yy159 = yylhsminor.yy159; + yymsp[-1].minor.yy285 = yylhsminor.yy285; break; case 167: /* as ::= AS ids */ { yymsp[-1].minor.yy0 = yymsp[0].minor.yy0; } @@ -2674,332 +2687,340 @@ static void yy_reduce( yymsp[0].minor.yy0 = yylhsminor.yy0; break; case 172: /* from ::= FROM tablelist */ -{yymsp[-1].minor.yy236 = yymsp[0].minor.yy236;} + case 173: /* from ::= FROM sub */ yytestcase(yyruleno==173); +{yymsp[-1].minor.yy148 = yymsp[0].minor.yy148;} + break; + case 174: /* sub ::= LP union RP */ +{yymsp[-2].minor.yy148 = addSubqueryElem(NULL, yymsp[-1].minor.yy285, NULL);} + break; + case 175: /* sub ::= LP union RP ids */ +{yymsp[-3].minor.yy148 = addSubqueryElem(NULL, yymsp[-2].minor.yy285, &yymsp[0].minor.yy0);} break; - case 173: /* from ::= FROM LP union RP */ -{yymsp[-3].minor.yy236 = setSubquery(NULL, yymsp[-1].minor.yy159);} + case 176: /* sub ::= sub COMMA LP union RP ids */ +{yylhsminor.yy148 = addSubqueryElem(yymsp[-5].minor.yy148, yymsp[-2].minor.yy285, &yymsp[0].minor.yy0);} + yymsp[-5].minor.yy148 = yylhsminor.yy148; break; - case 174: /* tablelist ::= ids cpxName */ + case 177: /* tablelist ::= ids cpxName */ { yymsp[-1].minor.yy0.n += yymsp[0].minor.yy0.n; - yylhsminor.yy236 = setTableNameList(NULL, &yymsp[-1].minor.yy0, NULL); + yylhsminor.yy148 = setTableNameList(NULL, &yymsp[-1].minor.yy0, NULL); } - yymsp[-1].minor.yy236 = yylhsminor.yy236; + yymsp[-1].minor.yy148 = yylhsminor.yy148; break; - case 175: /* tablelist ::= ids cpxName ids */ + case 178: /* tablelist ::= ids cpxName ids */ { yymsp[-2].minor.yy0.n += yymsp[-1].minor.yy0.n; - yylhsminor.yy236 = setTableNameList(NULL, &yymsp[-2].minor.yy0, &yymsp[0].minor.yy0); + yylhsminor.yy148 = setTableNameList(NULL, &yymsp[-2].minor.yy0, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy236 = yylhsminor.yy236; + yymsp[-2].minor.yy148 = yylhsminor.yy148; break; - case 176: /* tablelist ::= tablelist COMMA ids cpxName */ + case 179: /* tablelist ::= tablelist COMMA ids cpxName */ { yymsp[-1].minor.yy0.n += yymsp[0].minor.yy0.n; - yylhsminor.yy236 = setTableNameList(yymsp[-3].minor.yy236, &yymsp[-1].minor.yy0, NULL); + yylhsminor.yy148 = setTableNameList(yymsp[-3].minor.yy148, &yymsp[-1].minor.yy0, NULL); } - yymsp[-3].minor.yy236 = yylhsminor.yy236; + yymsp[-3].minor.yy148 = yylhsminor.yy148; break; - case 177: /* tablelist ::= tablelist COMMA ids cpxName ids */ + case 180: /* tablelist ::= tablelist COMMA ids cpxName ids */ { yymsp[-2].minor.yy0.n += yymsp[-1].minor.yy0.n; - yylhsminor.yy236 = setTableNameList(yymsp[-4].minor.yy236, &yymsp[-2].minor.yy0, &yymsp[0].minor.yy0); + yylhsminor.yy148 = setTableNameList(yymsp[-4].minor.yy148, &yymsp[-2].minor.yy0, &yymsp[0].minor.yy0); } - yymsp[-4].minor.yy236 = yylhsminor.yy236; + yymsp[-4].minor.yy148 = yylhsminor.yy148; break; - case 178: /* tmvar ::= VARIABLE */ + case 181: /* tmvar ::= VARIABLE */ {yylhsminor.yy0 = yymsp[0].minor.yy0;} yymsp[0].minor.yy0 = yylhsminor.yy0; break; - case 179: /* interval_opt ::= INTERVAL LP tmvar RP */ -{yymsp[-3].minor.yy184.interval = yymsp[-1].minor.yy0; yymsp[-3].minor.yy184.offset.n = 0;} + case 182: /* interval_opt ::= INTERVAL LP tmvar RP */ +{yymsp[-3].minor.yy376.interval = yymsp[-1].minor.yy0; yymsp[-3].minor.yy376.offset.n = 0;} break; - case 180: /* interval_opt ::= INTERVAL LP tmvar COMMA tmvar RP */ -{yymsp[-5].minor.yy184.interval = yymsp[-3].minor.yy0; yymsp[-5].minor.yy184.offset = yymsp[-1].minor.yy0;} + case 183: /* interval_opt ::= INTERVAL LP tmvar COMMA tmvar RP */ +{yymsp[-5].minor.yy376.interval = yymsp[-3].minor.yy0; yymsp[-5].minor.yy376.offset = yymsp[-1].minor.yy0;} break; - case 181: /* interval_opt ::= */ -{memset(&yymsp[1].minor.yy184, 0, sizeof(yymsp[1].minor.yy184));} + case 184: /* interval_opt ::= */ +{memset(&yymsp[1].minor.yy376, 0, sizeof(yymsp[1].minor.yy376));} break; - case 182: /* session_option ::= */ -{yymsp[1].minor.yy249.col.n = 0; yymsp[1].minor.yy249.gap.n = 0;} + case 185: /* session_option ::= */ +{yymsp[1].minor.yy523.col.n = 0; yymsp[1].minor.yy523.gap.n = 0;} break; - case 183: /* session_option ::= SESSION LP ids cpxName COMMA tmvar RP */ + case 186: /* session_option ::= SESSION LP ids cpxName COMMA tmvar RP */ { yymsp[-4].minor.yy0.n += yymsp[-3].minor.yy0.n; - yymsp[-6].minor.yy249.col = yymsp[-4].minor.yy0; - yymsp[-6].minor.yy249.gap = yymsp[-1].minor.yy0; + yymsp[-6].minor.yy523.col = yymsp[-4].minor.yy0; + yymsp[-6].minor.yy523.gap = yymsp[-1].minor.yy0; } break; - case 184: /* fill_opt ::= */ -{ yymsp[1].minor.yy159 = 0; } + case 187: /* fill_opt ::= */ +{ yymsp[1].minor.yy285 = 0; } break; - case 185: /* fill_opt ::= FILL LP ID COMMA tagitemlist RP */ + case 188: /* fill_opt ::= FILL LP ID COMMA tagitemlist RP */ { tVariant A = {0}; toTSDBType(yymsp[-3].minor.yy0.type); tVariantCreate(&A, &yymsp[-3].minor.yy0); - tVariantListInsert(yymsp[-1].minor.yy159, &A, -1, 0); - yymsp[-5].minor.yy159 = yymsp[-1].minor.yy159; + tVariantListInsert(yymsp[-1].minor.yy285, &A, -1, 0); + yymsp[-5].minor.yy285 = yymsp[-1].minor.yy285; } break; - case 186: /* fill_opt ::= FILL LP ID RP */ + case 189: /* fill_opt ::= FILL LP ID RP */ { toTSDBType(yymsp[-1].minor.yy0.type); - yymsp[-3].minor.yy159 = tVariantListAppendToken(NULL, &yymsp[-1].minor.yy0, -1); + yymsp[-3].minor.yy285 = tVariantListAppendToken(NULL, &yymsp[-1].minor.yy0, -1); } break; - case 187: /* sliding_opt ::= SLIDING LP tmvar RP */ + case 190: /* sliding_opt ::= SLIDING LP tmvar RP */ {yymsp[-3].minor.yy0 = yymsp[-1].minor.yy0; } break; - case 188: /* sliding_opt ::= */ + case 191: /* sliding_opt ::= */ {yymsp[1].minor.yy0.n = 0; yymsp[1].minor.yy0.z = NULL; yymsp[1].minor.yy0.type = 0; } break; - case 190: /* orderby_opt ::= ORDER BY sortlist */ -{yymsp[-2].minor.yy159 = yymsp[0].minor.yy159;} + case 193: /* orderby_opt ::= ORDER BY sortlist */ +{yymsp[-2].minor.yy285 = yymsp[0].minor.yy285;} break; - case 191: /* sortlist ::= sortlist COMMA item sortorder */ + case 194: /* sortlist ::= sortlist COMMA item sortorder */ { - yylhsminor.yy159 = tVariantListAppend(yymsp[-3].minor.yy159, &yymsp[-1].minor.yy488, yymsp[0].minor.yy20); + yylhsminor.yy285 = tVariantListAppend(yymsp[-3].minor.yy285, &yymsp[-1].minor.yy362, yymsp[0].minor.yy460); } - yymsp[-3].minor.yy159 = yylhsminor.yy159; + yymsp[-3].minor.yy285 = yylhsminor.yy285; break; - case 192: /* sortlist ::= item sortorder */ + case 195: /* sortlist ::= item sortorder */ { - yylhsminor.yy159 = tVariantListAppend(NULL, &yymsp[-1].minor.yy488, yymsp[0].minor.yy20); + yylhsminor.yy285 = tVariantListAppend(NULL, &yymsp[-1].minor.yy362, yymsp[0].minor.yy460); } - yymsp[-1].minor.yy159 = yylhsminor.yy159; + yymsp[-1].minor.yy285 = yylhsminor.yy285; break; - case 193: /* item ::= ids cpxName */ + case 196: /* item ::= ids cpxName */ { toTSDBType(yymsp[-1].minor.yy0.type); yymsp[-1].minor.yy0.n += yymsp[0].minor.yy0.n; - tVariantCreate(&yylhsminor.yy488, &yymsp[-1].minor.yy0); + tVariantCreate(&yylhsminor.yy362, &yymsp[-1].minor.yy0); } - yymsp[-1].minor.yy488 = yylhsminor.yy488; + yymsp[-1].minor.yy362 = yylhsminor.yy362; break; - case 194: /* sortorder ::= ASC */ -{ yymsp[0].minor.yy20 = TSDB_ORDER_ASC; } + case 197: /* sortorder ::= ASC */ +{ yymsp[0].minor.yy460 = TSDB_ORDER_ASC; } break; - case 195: /* sortorder ::= DESC */ -{ yymsp[0].minor.yy20 = TSDB_ORDER_DESC;} + case 198: /* sortorder ::= DESC */ +{ yymsp[0].minor.yy460 = TSDB_ORDER_DESC;} break; - case 196: /* sortorder ::= */ -{ yymsp[1].minor.yy20 = TSDB_ORDER_ASC; } + case 199: /* sortorder ::= */ +{ yymsp[1].minor.yy460 = TSDB_ORDER_ASC; } break; - case 197: /* groupby_opt ::= */ -{ yymsp[1].minor.yy159 = 0;} + case 200: /* groupby_opt ::= */ +{ yymsp[1].minor.yy285 = 0;} break; - case 198: /* groupby_opt ::= GROUP BY grouplist */ -{ yymsp[-2].minor.yy159 = yymsp[0].minor.yy159;} + case 201: /* groupby_opt ::= GROUP BY grouplist */ +{ yymsp[-2].minor.yy285 = yymsp[0].minor.yy285;} break; - case 199: /* grouplist ::= grouplist COMMA item */ + case 202: /* grouplist ::= grouplist COMMA item */ { - yylhsminor.yy159 = tVariantListAppend(yymsp[-2].minor.yy159, &yymsp[0].minor.yy488, -1); + yylhsminor.yy285 = tVariantListAppend(yymsp[-2].minor.yy285, &yymsp[0].minor.yy362, -1); } - yymsp[-2].minor.yy159 = yylhsminor.yy159; + yymsp[-2].minor.yy285 = yylhsminor.yy285; break; - case 200: /* grouplist ::= item */ + case 203: /* grouplist ::= item */ { - yylhsminor.yy159 = tVariantListAppend(NULL, &yymsp[0].minor.yy488, -1); + yylhsminor.yy285 = tVariantListAppend(NULL, &yymsp[0].minor.yy362, -1); } - yymsp[0].minor.yy159 = yylhsminor.yy159; + yymsp[0].minor.yy285 = yylhsminor.yy285; break; - case 201: /* having_opt ::= */ - case 211: /* where_opt ::= */ yytestcase(yyruleno==211); - case 253: /* expritem ::= */ yytestcase(yyruleno==253); -{yymsp[1].minor.yy118 = 0;} + case 204: /* having_opt ::= */ + case 214: /* where_opt ::= */ yytestcase(yyruleno==214); + case 256: /* expritem ::= */ yytestcase(yyruleno==256); +{yymsp[1].minor.yy178 = 0;} break; - case 202: /* having_opt ::= HAVING expr */ - case 212: /* where_opt ::= WHERE expr */ yytestcase(yyruleno==212); -{yymsp[-1].minor.yy118 = yymsp[0].minor.yy118;} + case 205: /* having_opt ::= HAVING expr */ + case 215: /* where_opt ::= WHERE expr */ yytestcase(yyruleno==215); +{yymsp[-1].minor.yy178 = yymsp[0].minor.yy178;} break; - case 203: /* limit_opt ::= */ - case 207: /* slimit_opt ::= */ yytestcase(yyruleno==207); -{yymsp[1].minor.yy440.limit = -1; yymsp[1].minor.yy440.offset = 0;} + case 206: /* limit_opt ::= */ + case 210: /* slimit_opt ::= */ yytestcase(yyruleno==210); +{yymsp[1].minor.yy438.limit = -1; yymsp[1].minor.yy438.offset = 0;} break; - case 204: /* limit_opt ::= LIMIT signed */ - case 208: /* slimit_opt ::= SLIMIT signed */ yytestcase(yyruleno==208); -{yymsp[-1].minor.yy440.limit = yymsp[0].minor.yy317; yymsp[-1].minor.yy440.offset = 0;} + case 207: /* limit_opt ::= LIMIT signed */ + case 211: /* slimit_opt ::= SLIMIT signed */ yytestcase(yyruleno==211); +{yymsp[-1].minor.yy438.limit = yymsp[0].minor.yy525; yymsp[-1].minor.yy438.offset = 0;} break; - case 205: /* limit_opt ::= LIMIT signed OFFSET signed */ -{ yymsp[-3].minor.yy440.limit = yymsp[-2].minor.yy317; yymsp[-3].minor.yy440.offset = yymsp[0].minor.yy317;} + case 208: /* limit_opt ::= LIMIT signed OFFSET signed */ +{ yymsp[-3].minor.yy438.limit = yymsp[-2].minor.yy525; yymsp[-3].minor.yy438.offset = yymsp[0].minor.yy525;} break; - case 206: /* limit_opt ::= LIMIT signed COMMA signed */ -{ yymsp[-3].minor.yy440.limit = yymsp[0].minor.yy317; yymsp[-3].minor.yy440.offset = yymsp[-2].minor.yy317;} + case 209: /* limit_opt ::= LIMIT signed COMMA signed */ +{ yymsp[-3].minor.yy438.limit = yymsp[0].minor.yy525; yymsp[-3].minor.yy438.offset = yymsp[-2].minor.yy525;} break; - case 209: /* slimit_opt ::= SLIMIT signed SOFFSET signed */ -{yymsp[-3].minor.yy440.limit = yymsp[-2].minor.yy317; yymsp[-3].minor.yy440.offset = yymsp[0].minor.yy317;} + case 212: /* slimit_opt ::= SLIMIT signed SOFFSET signed */ +{yymsp[-3].minor.yy438.limit = yymsp[-2].minor.yy525; yymsp[-3].minor.yy438.offset = yymsp[0].minor.yy525;} break; - case 210: /* slimit_opt ::= SLIMIT signed COMMA signed */ -{yymsp[-3].minor.yy440.limit = yymsp[0].minor.yy317; yymsp[-3].minor.yy440.offset = yymsp[-2].minor.yy317;} + case 213: /* slimit_opt ::= SLIMIT signed COMMA signed */ +{yymsp[-3].minor.yy438.limit = yymsp[0].minor.yy525; yymsp[-3].minor.yy438.offset = yymsp[-2].minor.yy525;} break; - case 213: /* expr ::= LP expr RP */ -{yylhsminor.yy118 = yymsp[-1].minor.yy118; yylhsminor.yy118->token.z = yymsp[-2].minor.yy0.z; yylhsminor.yy118->token.n = (yymsp[0].minor.yy0.z - yymsp[-2].minor.yy0.z + 1);} - yymsp[-2].minor.yy118 = yylhsminor.yy118; + case 216: /* expr ::= LP expr RP */ +{yylhsminor.yy178 = yymsp[-1].minor.yy178; yylhsminor.yy178->token.z = yymsp[-2].minor.yy0.z; yylhsminor.yy178->token.n = (yymsp[0].minor.yy0.z - yymsp[-2].minor.yy0.z + 1);} + yymsp[-2].minor.yy178 = yylhsminor.yy178; break; - case 214: /* expr ::= ID */ -{ yylhsminor.yy118 = tSqlExprCreateIdValue(&yymsp[0].minor.yy0, TK_ID);} - yymsp[0].minor.yy118 = yylhsminor.yy118; + case 217: /* expr ::= ID */ +{ yylhsminor.yy178 = tSqlExprCreateIdValue(&yymsp[0].minor.yy0, TK_ID);} + yymsp[0].minor.yy178 = yylhsminor.yy178; break; - case 215: /* expr ::= ID DOT ID */ -{ yymsp[-2].minor.yy0.n += (1+yymsp[0].minor.yy0.n); yylhsminor.yy118 = tSqlExprCreateIdValue(&yymsp[-2].minor.yy0, TK_ID);} - yymsp[-2].minor.yy118 = yylhsminor.yy118; + case 218: /* expr ::= ID DOT ID */ +{ yymsp[-2].minor.yy0.n += (1+yymsp[0].minor.yy0.n); yylhsminor.yy178 = tSqlExprCreateIdValue(&yymsp[-2].minor.yy0, TK_ID);} + yymsp[-2].minor.yy178 = yylhsminor.yy178; break; - case 216: /* expr ::= ID DOT STAR */ -{ yymsp[-2].minor.yy0.n += (1+yymsp[0].minor.yy0.n); yylhsminor.yy118 = tSqlExprCreateIdValue(&yymsp[-2].minor.yy0, TK_ALL);} - yymsp[-2].minor.yy118 = yylhsminor.yy118; + case 219: /* expr ::= ID DOT STAR */ +{ yymsp[-2].minor.yy0.n += (1+yymsp[0].minor.yy0.n); yylhsminor.yy178 = tSqlExprCreateIdValue(&yymsp[-2].minor.yy0, TK_ALL);} + yymsp[-2].minor.yy178 = yylhsminor.yy178; break; - case 217: /* expr ::= INTEGER */ -{ yylhsminor.yy118 = tSqlExprCreateIdValue(&yymsp[0].minor.yy0, TK_INTEGER);} - yymsp[0].minor.yy118 = yylhsminor.yy118; + case 220: /* expr ::= INTEGER */ +{ yylhsminor.yy178 = tSqlExprCreateIdValue(&yymsp[0].minor.yy0, TK_INTEGER);} + yymsp[0].minor.yy178 = yylhsminor.yy178; break; - case 218: /* expr ::= MINUS INTEGER */ - case 219: /* expr ::= PLUS INTEGER */ yytestcase(yyruleno==219); -{ yymsp[-1].minor.yy0.n += yymsp[0].minor.yy0.n; yymsp[-1].minor.yy0.type = TK_INTEGER; yylhsminor.yy118 = tSqlExprCreateIdValue(&yymsp[-1].minor.yy0, TK_INTEGER);} - yymsp[-1].minor.yy118 = yylhsminor.yy118; + case 221: /* expr ::= MINUS INTEGER */ + case 222: /* expr ::= PLUS INTEGER */ yytestcase(yyruleno==222); +{ yymsp[-1].minor.yy0.n += yymsp[0].minor.yy0.n; yymsp[-1].minor.yy0.type = TK_INTEGER; yylhsminor.yy178 = tSqlExprCreateIdValue(&yymsp[-1].minor.yy0, TK_INTEGER);} + yymsp[-1].minor.yy178 = yylhsminor.yy178; break; - case 220: /* expr ::= FLOAT */ -{ yylhsminor.yy118 = tSqlExprCreateIdValue(&yymsp[0].minor.yy0, TK_FLOAT);} - yymsp[0].minor.yy118 = yylhsminor.yy118; + case 223: /* expr ::= FLOAT */ +{ yylhsminor.yy178 = tSqlExprCreateIdValue(&yymsp[0].minor.yy0, TK_FLOAT);} + yymsp[0].minor.yy178 = yylhsminor.yy178; break; - case 221: /* expr ::= MINUS FLOAT */ - case 222: /* expr ::= PLUS FLOAT */ yytestcase(yyruleno==222); -{ yymsp[-1].minor.yy0.n += yymsp[0].minor.yy0.n; yymsp[-1].minor.yy0.type = TK_FLOAT; yylhsminor.yy118 = tSqlExprCreateIdValue(&yymsp[-1].minor.yy0, TK_FLOAT);} - yymsp[-1].minor.yy118 = yylhsminor.yy118; + case 224: /* expr ::= MINUS FLOAT */ + case 225: /* expr ::= PLUS FLOAT */ yytestcase(yyruleno==225); +{ yymsp[-1].minor.yy0.n += yymsp[0].minor.yy0.n; yymsp[-1].minor.yy0.type = TK_FLOAT; yylhsminor.yy178 = tSqlExprCreateIdValue(&yymsp[-1].minor.yy0, TK_FLOAT);} + yymsp[-1].minor.yy178 = yylhsminor.yy178; break; - case 223: /* expr ::= STRING */ -{ yylhsminor.yy118 = tSqlExprCreateIdValue(&yymsp[0].minor.yy0, TK_STRING);} - yymsp[0].minor.yy118 = yylhsminor.yy118; + case 226: /* expr ::= STRING */ +{ yylhsminor.yy178 = tSqlExprCreateIdValue(&yymsp[0].minor.yy0, TK_STRING);} + yymsp[0].minor.yy178 = yylhsminor.yy178; break; - case 224: /* expr ::= NOW */ -{ yylhsminor.yy118 = tSqlExprCreateIdValue(&yymsp[0].minor.yy0, TK_NOW); } - yymsp[0].minor.yy118 = yylhsminor.yy118; + case 227: /* expr ::= NOW */ +{ yylhsminor.yy178 = tSqlExprCreateIdValue(&yymsp[0].minor.yy0, TK_NOW); } + yymsp[0].minor.yy178 = yylhsminor.yy178; break; - case 225: /* expr ::= VARIABLE */ -{ yylhsminor.yy118 = tSqlExprCreateIdValue(&yymsp[0].minor.yy0, TK_VARIABLE);} - yymsp[0].minor.yy118 = yylhsminor.yy118; + case 228: /* expr ::= VARIABLE */ +{ yylhsminor.yy178 = tSqlExprCreateIdValue(&yymsp[0].minor.yy0, TK_VARIABLE);} + yymsp[0].minor.yy178 = yylhsminor.yy178; break; - case 226: /* expr ::= PLUS VARIABLE */ - case 227: /* expr ::= MINUS VARIABLE */ yytestcase(yyruleno==227); -{ yymsp[-1].minor.yy0.n += yymsp[0].minor.yy0.n; yymsp[-1].minor.yy0.type = TK_VARIABLE; yylhsminor.yy118 = tSqlExprCreateIdValue(&yymsp[-1].minor.yy0, TK_VARIABLE);} - yymsp[-1].minor.yy118 = yylhsminor.yy118; + case 229: /* expr ::= PLUS VARIABLE */ + case 230: /* expr ::= MINUS VARIABLE */ yytestcase(yyruleno==230); +{ yymsp[-1].minor.yy0.n += yymsp[0].minor.yy0.n; yymsp[-1].minor.yy0.type = TK_VARIABLE; yylhsminor.yy178 = tSqlExprCreateIdValue(&yymsp[-1].minor.yy0, TK_VARIABLE);} + yymsp[-1].minor.yy178 = yylhsminor.yy178; break; - case 228: /* expr ::= BOOL */ -{ yylhsminor.yy118 = tSqlExprCreateIdValue(&yymsp[0].minor.yy0, TK_BOOL);} - yymsp[0].minor.yy118 = yylhsminor.yy118; + case 231: /* expr ::= BOOL */ +{ yylhsminor.yy178 = tSqlExprCreateIdValue(&yymsp[0].minor.yy0, TK_BOOL);} + yymsp[0].minor.yy178 = yylhsminor.yy178; break; - case 229: /* expr ::= NULL */ -{ yylhsminor.yy118 = tSqlExprCreateIdValue(&yymsp[0].minor.yy0, TK_NULL);} - yymsp[0].minor.yy118 = yylhsminor.yy118; + case 232: /* expr ::= NULL */ +{ yylhsminor.yy178 = tSqlExprCreateIdValue(&yymsp[0].minor.yy0, TK_NULL);} + yymsp[0].minor.yy178 = yylhsminor.yy178; break; - case 230: /* expr ::= ID LP exprlist RP */ -{ yylhsminor.yy118 = tSqlExprCreateFunction(yymsp[-1].minor.yy159, &yymsp[-3].minor.yy0, &yymsp[0].minor.yy0, yymsp[-3].minor.yy0.type); } - yymsp[-3].minor.yy118 = yylhsminor.yy118; + case 233: /* expr ::= ID LP exprlist RP */ +{ yylhsminor.yy178 = tSqlExprCreateFunction(yymsp[-1].minor.yy285, &yymsp[-3].minor.yy0, &yymsp[0].minor.yy0, yymsp[-3].minor.yy0.type); } + yymsp[-3].minor.yy178 = yylhsminor.yy178; break; - case 231: /* expr ::= ID LP STAR RP */ -{ yylhsminor.yy118 = tSqlExprCreateFunction(NULL, &yymsp[-3].minor.yy0, &yymsp[0].minor.yy0, yymsp[-3].minor.yy0.type); } - yymsp[-3].minor.yy118 = yylhsminor.yy118; + case 234: /* expr ::= ID LP STAR RP */ +{ yylhsminor.yy178 = tSqlExprCreateFunction(NULL, &yymsp[-3].minor.yy0, &yymsp[0].minor.yy0, yymsp[-3].minor.yy0.type); } + yymsp[-3].minor.yy178 = yylhsminor.yy178; break; - case 232: /* expr ::= expr IS NULL */ -{yylhsminor.yy118 = tSqlExprCreate(yymsp[-2].minor.yy118, NULL, TK_ISNULL);} - yymsp[-2].minor.yy118 = yylhsminor.yy118; + case 235: /* expr ::= expr IS NULL */ +{yylhsminor.yy178 = tSqlExprCreate(yymsp[-2].minor.yy178, NULL, TK_ISNULL);} + yymsp[-2].minor.yy178 = yylhsminor.yy178; break; - case 233: /* expr ::= expr IS NOT NULL */ -{yylhsminor.yy118 = tSqlExprCreate(yymsp[-3].minor.yy118, NULL, TK_NOTNULL);} - yymsp[-3].minor.yy118 = yylhsminor.yy118; + case 236: /* expr ::= expr IS NOT NULL */ +{yylhsminor.yy178 = tSqlExprCreate(yymsp[-3].minor.yy178, NULL, TK_NOTNULL);} + yymsp[-3].minor.yy178 = yylhsminor.yy178; break; - case 234: /* expr ::= expr LT expr */ -{yylhsminor.yy118 = tSqlExprCreate(yymsp[-2].minor.yy118, yymsp[0].minor.yy118, TK_LT);} - yymsp[-2].minor.yy118 = yylhsminor.yy118; + case 237: /* expr ::= expr LT expr */ +{yylhsminor.yy178 = tSqlExprCreate(yymsp[-2].minor.yy178, yymsp[0].minor.yy178, TK_LT);} + yymsp[-2].minor.yy178 = yylhsminor.yy178; break; - case 235: /* expr ::= expr GT expr */ -{yylhsminor.yy118 = tSqlExprCreate(yymsp[-2].minor.yy118, yymsp[0].minor.yy118, TK_GT);} - yymsp[-2].minor.yy118 = yylhsminor.yy118; + case 238: /* expr ::= expr GT expr */ +{yylhsminor.yy178 = tSqlExprCreate(yymsp[-2].minor.yy178, yymsp[0].minor.yy178, TK_GT);} + yymsp[-2].minor.yy178 = yylhsminor.yy178; break; - case 236: /* expr ::= expr LE expr */ -{yylhsminor.yy118 = tSqlExprCreate(yymsp[-2].minor.yy118, yymsp[0].minor.yy118, TK_LE);} - yymsp[-2].minor.yy118 = yylhsminor.yy118; + case 239: /* expr ::= expr LE expr */ +{yylhsminor.yy178 = tSqlExprCreate(yymsp[-2].minor.yy178, yymsp[0].minor.yy178, TK_LE);} + yymsp[-2].minor.yy178 = yylhsminor.yy178; break; - case 237: /* expr ::= expr GE expr */ -{yylhsminor.yy118 = tSqlExprCreate(yymsp[-2].minor.yy118, yymsp[0].minor.yy118, TK_GE);} - yymsp[-2].minor.yy118 = yylhsminor.yy118; + case 240: /* expr ::= expr GE expr */ +{yylhsminor.yy178 = tSqlExprCreate(yymsp[-2].minor.yy178, yymsp[0].minor.yy178, TK_GE);} + yymsp[-2].minor.yy178 = yylhsminor.yy178; break; - case 238: /* expr ::= expr NE expr */ -{yylhsminor.yy118 = tSqlExprCreate(yymsp[-2].minor.yy118, yymsp[0].minor.yy118, TK_NE);} - yymsp[-2].minor.yy118 = yylhsminor.yy118; + case 241: /* expr ::= expr NE expr */ +{yylhsminor.yy178 = tSqlExprCreate(yymsp[-2].minor.yy178, yymsp[0].minor.yy178, TK_NE);} + yymsp[-2].minor.yy178 = yylhsminor.yy178; break; - case 239: /* expr ::= expr EQ expr */ -{yylhsminor.yy118 = tSqlExprCreate(yymsp[-2].minor.yy118, yymsp[0].minor.yy118, TK_EQ);} - yymsp[-2].minor.yy118 = yylhsminor.yy118; + case 242: /* expr ::= expr EQ expr */ +{yylhsminor.yy178 = tSqlExprCreate(yymsp[-2].minor.yy178, yymsp[0].minor.yy178, TK_EQ);} + yymsp[-2].minor.yy178 = yylhsminor.yy178; break; - case 240: /* expr ::= expr BETWEEN expr AND expr */ -{ tSqlExpr* X2 = tSqlExprClone(yymsp[-4].minor.yy118); yylhsminor.yy118 = tSqlExprCreate(tSqlExprCreate(yymsp[-4].minor.yy118, yymsp[-2].minor.yy118, TK_GE), tSqlExprCreate(X2, yymsp[0].minor.yy118, TK_LE), TK_AND);} - yymsp[-4].minor.yy118 = yylhsminor.yy118; + case 243: /* expr ::= expr BETWEEN expr AND expr */ +{ tSqlExpr* X2 = tSqlExprClone(yymsp[-4].minor.yy178); yylhsminor.yy178 = tSqlExprCreate(tSqlExprCreate(yymsp[-4].minor.yy178, yymsp[-2].minor.yy178, TK_GE), tSqlExprCreate(X2, yymsp[0].minor.yy178, TK_LE), TK_AND);} + yymsp[-4].minor.yy178 = yylhsminor.yy178; break; - case 241: /* expr ::= expr AND expr */ -{yylhsminor.yy118 = tSqlExprCreate(yymsp[-2].minor.yy118, yymsp[0].minor.yy118, TK_AND);} - yymsp[-2].minor.yy118 = yylhsminor.yy118; + case 244: /* expr ::= expr AND expr */ +{yylhsminor.yy178 = tSqlExprCreate(yymsp[-2].minor.yy178, yymsp[0].minor.yy178, TK_AND);} + yymsp[-2].minor.yy178 = yylhsminor.yy178; break; - case 242: /* expr ::= expr OR expr */ -{yylhsminor.yy118 = tSqlExprCreate(yymsp[-2].minor.yy118, yymsp[0].minor.yy118, TK_OR); } - yymsp[-2].minor.yy118 = yylhsminor.yy118; + case 245: /* expr ::= expr OR expr */ +{yylhsminor.yy178 = tSqlExprCreate(yymsp[-2].minor.yy178, yymsp[0].minor.yy178, TK_OR); } + yymsp[-2].minor.yy178 = yylhsminor.yy178; break; - case 243: /* expr ::= expr PLUS expr */ -{yylhsminor.yy118 = tSqlExprCreate(yymsp[-2].minor.yy118, yymsp[0].minor.yy118, TK_PLUS); } - yymsp[-2].minor.yy118 = yylhsminor.yy118; + case 246: /* expr ::= expr PLUS expr */ +{yylhsminor.yy178 = tSqlExprCreate(yymsp[-2].minor.yy178, yymsp[0].minor.yy178, TK_PLUS); } + yymsp[-2].minor.yy178 = yylhsminor.yy178; break; - case 244: /* expr ::= expr MINUS expr */ -{yylhsminor.yy118 = tSqlExprCreate(yymsp[-2].minor.yy118, yymsp[0].minor.yy118, TK_MINUS); } - yymsp[-2].minor.yy118 = yylhsminor.yy118; + case 247: /* expr ::= expr MINUS expr */ +{yylhsminor.yy178 = tSqlExprCreate(yymsp[-2].minor.yy178, yymsp[0].minor.yy178, TK_MINUS); } + yymsp[-2].minor.yy178 = yylhsminor.yy178; break; - case 245: /* expr ::= expr STAR expr */ -{yylhsminor.yy118 = tSqlExprCreate(yymsp[-2].minor.yy118, yymsp[0].minor.yy118, TK_STAR); } - yymsp[-2].minor.yy118 = yylhsminor.yy118; + case 248: /* expr ::= expr STAR expr */ +{yylhsminor.yy178 = tSqlExprCreate(yymsp[-2].minor.yy178, yymsp[0].minor.yy178, TK_STAR); } + yymsp[-2].minor.yy178 = yylhsminor.yy178; break; - case 246: /* expr ::= expr SLASH expr */ -{yylhsminor.yy118 = tSqlExprCreate(yymsp[-2].minor.yy118, yymsp[0].minor.yy118, TK_DIVIDE);} - yymsp[-2].minor.yy118 = yylhsminor.yy118; + case 249: /* expr ::= expr SLASH expr */ +{yylhsminor.yy178 = tSqlExprCreate(yymsp[-2].minor.yy178, yymsp[0].minor.yy178, TK_DIVIDE);} + yymsp[-2].minor.yy178 = yylhsminor.yy178; break; - case 247: /* expr ::= expr REM expr */ -{yylhsminor.yy118 = tSqlExprCreate(yymsp[-2].minor.yy118, yymsp[0].minor.yy118, TK_REM); } - yymsp[-2].minor.yy118 = yylhsminor.yy118; + case 250: /* expr ::= expr REM expr */ +{yylhsminor.yy178 = tSqlExprCreate(yymsp[-2].minor.yy178, yymsp[0].minor.yy178, TK_REM); } + yymsp[-2].minor.yy178 = yylhsminor.yy178; break; - case 248: /* expr ::= expr LIKE expr */ -{yylhsminor.yy118 = tSqlExprCreate(yymsp[-2].minor.yy118, yymsp[0].minor.yy118, TK_LIKE); } - yymsp[-2].minor.yy118 = yylhsminor.yy118; + case 251: /* expr ::= expr LIKE expr */ +{yylhsminor.yy178 = tSqlExprCreate(yymsp[-2].minor.yy178, yymsp[0].minor.yy178, TK_LIKE); } + yymsp[-2].minor.yy178 = yylhsminor.yy178; break; - case 249: /* expr ::= expr IN LP exprlist RP */ -{yylhsminor.yy118 = tSqlExprCreate(yymsp[-4].minor.yy118, (tSqlExpr*)yymsp[-1].minor.yy159, TK_IN); } - yymsp[-4].minor.yy118 = yylhsminor.yy118; + case 252: /* expr ::= expr IN LP exprlist RP */ +{yylhsminor.yy178 = tSqlExprCreate(yymsp[-4].minor.yy178, (tSqlExpr*)yymsp[-1].minor.yy285, TK_IN); } + yymsp[-4].minor.yy178 = yylhsminor.yy178; break; - case 250: /* exprlist ::= exprlist COMMA expritem */ -{yylhsminor.yy159 = tSqlExprListAppend(yymsp[-2].minor.yy159,yymsp[0].minor.yy118,0, 0);} - yymsp[-2].minor.yy159 = yylhsminor.yy159; + case 253: /* exprlist ::= exprlist COMMA expritem */ +{yylhsminor.yy285 = tSqlExprListAppend(yymsp[-2].minor.yy285,yymsp[0].minor.yy178,0, 0);} + yymsp[-2].minor.yy285 = yylhsminor.yy285; break; - case 251: /* exprlist ::= expritem */ -{yylhsminor.yy159 = tSqlExprListAppend(0,yymsp[0].minor.yy118,0, 0);} - yymsp[0].minor.yy159 = yylhsminor.yy159; + case 254: /* exprlist ::= expritem */ +{yylhsminor.yy285 = tSqlExprListAppend(0,yymsp[0].minor.yy178,0, 0);} + yymsp[0].minor.yy285 = yylhsminor.yy285; break; - case 252: /* expritem ::= expr */ -{yylhsminor.yy118 = yymsp[0].minor.yy118;} - yymsp[0].minor.yy118 = yylhsminor.yy118; + case 255: /* expritem ::= expr */ +{yylhsminor.yy178 = yymsp[0].minor.yy178;} + yymsp[0].minor.yy178 = yylhsminor.yy178; break; - case 254: /* cmd ::= RESET QUERY CACHE */ + case 257: /* cmd ::= RESET QUERY CACHE */ { setDCLSqlElems(pInfo, TSDB_SQL_RESET_CACHE, 0);} break; - case 255: /* cmd ::= SYNCDB ids REPLICA */ + case 258: /* cmd ::= SYNCDB ids REPLICA */ { setDCLSqlElems(pInfo, TSDB_SQL_SYNC_DB_REPLICA, 1, &yymsp[-1].minor.yy0);} break; - case 256: /* cmd ::= ALTER TABLE ids cpxName ADD COLUMN columnlist */ + case 259: /* cmd ::= ALTER TABLE ids cpxName ADD COLUMN columnlist */ { yymsp[-4].minor.yy0.n += yymsp[-3].minor.yy0.n; - SAlterTableInfo* pAlterTable = tSetAlterTableInfo(&yymsp[-4].minor.yy0, yymsp[0].minor.yy159, NULL, TSDB_ALTER_TABLE_ADD_COLUMN, -1); + SAlterTableInfo* pAlterTable = tSetAlterTableInfo(&yymsp[-4].minor.yy0, yymsp[0].minor.yy285, NULL, TSDB_ALTER_TABLE_ADD_COLUMN, -1); setSqlInfo(pInfo, pAlterTable, NULL, TSDB_SQL_ALTER_TABLE); } break; - case 257: /* cmd ::= ALTER TABLE ids cpxName DROP COLUMN ids */ + case 260: /* cmd ::= ALTER TABLE ids cpxName DROP COLUMN ids */ { yymsp[-4].minor.yy0.n += yymsp[-3].minor.yy0.n; @@ -3010,14 +3031,14 @@ static void yy_reduce( setSqlInfo(pInfo, pAlterTable, NULL, TSDB_SQL_ALTER_TABLE); } break; - case 258: /* cmd ::= ALTER TABLE ids cpxName ADD TAG columnlist */ + case 261: /* cmd ::= ALTER TABLE ids cpxName ADD TAG columnlist */ { yymsp[-4].minor.yy0.n += yymsp[-3].minor.yy0.n; - SAlterTableInfo* pAlterTable = tSetAlterTableInfo(&yymsp[-4].minor.yy0, yymsp[0].minor.yy159, NULL, TSDB_ALTER_TABLE_ADD_TAG_COLUMN, -1); + SAlterTableInfo* pAlterTable = tSetAlterTableInfo(&yymsp[-4].minor.yy0, yymsp[0].minor.yy285, NULL, TSDB_ALTER_TABLE_ADD_TAG_COLUMN, -1); setSqlInfo(pInfo, pAlterTable, NULL, TSDB_SQL_ALTER_TABLE); } break; - case 259: /* cmd ::= ALTER TABLE ids cpxName DROP TAG ids */ + case 262: /* cmd ::= ALTER TABLE ids cpxName DROP TAG ids */ { yymsp[-4].minor.yy0.n += yymsp[-3].minor.yy0.n; @@ -3028,7 +3049,7 @@ static void yy_reduce( setSqlInfo(pInfo, pAlterTable, NULL, TSDB_SQL_ALTER_TABLE); } break; - case 260: /* cmd ::= ALTER TABLE ids cpxName CHANGE TAG ids ids */ + case 263: /* cmd ::= ALTER TABLE ids cpxName CHANGE TAG ids ids */ { yymsp[-5].minor.yy0.n += yymsp[-4].minor.yy0.n; @@ -3042,26 +3063,26 @@ static void yy_reduce( setSqlInfo(pInfo, pAlterTable, NULL, TSDB_SQL_ALTER_TABLE); } break; - case 261: /* cmd ::= ALTER TABLE ids cpxName SET TAG ids EQ tagitem */ + case 264: /* cmd ::= ALTER TABLE ids cpxName SET TAG ids EQ tagitem */ { yymsp[-6].minor.yy0.n += yymsp[-5].minor.yy0.n; toTSDBType(yymsp[-2].minor.yy0.type); SArray* A = tVariantListAppendToken(NULL, &yymsp[-2].minor.yy0, -1); - A = tVariantListAppend(A, &yymsp[0].minor.yy488, -1); + A = tVariantListAppend(A, &yymsp[0].minor.yy362, -1); SAlterTableInfo* pAlterTable = tSetAlterTableInfo(&yymsp[-6].minor.yy0, NULL, A, TSDB_ALTER_TABLE_UPDATE_TAG_VAL, -1); setSqlInfo(pInfo, pAlterTable, NULL, TSDB_SQL_ALTER_TABLE); } break; - case 262: /* cmd ::= ALTER STABLE ids cpxName ADD COLUMN columnlist */ + case 265: /* cmd ::= ALTER STABLE ids cpxName ADD COLUMN columnlist */ { yymsp[-4].minor.yy0.n += yymsp[-3].minor.yy0.n; - SAlterTableInfo* pAlterTable = tSetAlterTableInfo(&yymsp[-4].minor.yy0, yymsp[0].minor.yy159, NULL, TSDB_ALTER_TABLE_ADD_COLUMN, TSDB_SUPER_TABLE); + SAlterTableInfo* pAlterTable = tSetAlterTableInfo(&yymsp[-4].minor.yy0, yymsp[0].minor.yy285, NULL, TSDB_ALTER_TABLE_ADD_COLUMN, TSDB_SUPER_TABLE); setSqlInfo(pInfo, pAlterTable, NULL, TSDB_SQL_ALTER_TABLE); } break; - case 263: /* cmd ::= ALTER STABLE ids cpxName DROP COLUMN ids */ + case 266: /* cmd ::= ALTER STABLE ids cpxName DROP COLUMN ids */ { yymsp[-4].minor.yy0.n += yymsp[-3].minor.yy0.n; @@ -3072,14 +3093,14 @@ static void yy_reduce( setSqlInfo(pInfo, pAlterTable, NULL, TSDB_SQL_ALTER_TABLE); } break; - case 264: /* cmd ::= ALTER STABLE ids cpxName ADD TAG columnlist */ + case 267: /* cmd ::= ALTER STABLE ids cpxName ADD TAG columnlist */ { yymsp[-4].minor.yy0.n += yymsp[-3].minor.yy0.n; - SAlterTableInfo* pAlterTable = tSetAlterTableInfo(&yymsp[-4].minor.yy0, yymsp[0].minor.yy159, NULL, TSDB_ALTER_TABLE_ADD_TAG_COLUMN, TSDB_SUPER_TABLE); + SAlterTableInfo* pAlterTable = tSetAlterTableInfo(&yymsp[-4].minor.yy0, yymsp[0].minor.yy285, NULL, TSDB_ALTER_TABLE_ADD_TAG_COLUMN, TSDB_SUPER_TABLE); setSqlInfo(pInfo, pAlterTable, NULL, TSDB_SQL_ALTER_TABLE); } break; - case 265: /* cmd ::= ALTER STABLE ids cpxName DROP TAG ids */ + case 268: /* cmd ::= ALTER STABLE ids cpxName DROP TAG ids */ { yymsp[-4].minor.yy0.n += yymsp[-3].minor.yy0.n; @@ -3090,7 +3111,7 @@ static void yy_reduce( setSqlInfo(pInfo, pAlterTable, NULL, TSDB_SQL_ALTER_TABLE); } break; - case 266: /* cmd ::= ALTER STABLE ids cpxName CHANGE TAG ids ids */ + case 269: /* cmd ::= ALTER STABLE ids cpxName CHANGE TAG ids ids */ { yymsp[-5].minor.yy0.n += yymsp[-4].minor.yy0.n; @@ -3104,13 +3125,13 @@ static void yy_reduce( setSqlInfo(pInfo, pAlterTable, NULL, TSDB_SQL_ALTER_TABLE); } break; - case 267: /* cmd ::= KILL CONNECTION INTEGER */ + case 270: /* cmd ::= KILL CONNECTION INTEGER */ {setKillSql(pInfo, TSDB_SQL_KILL_CONNECTION, &yymsp[0].minor.yy0);} break; - case 268: /* cmd ::= KILL STREAM INTEGER COLON INTEGER */ + case 271: /* cmd ::= KILL STREAM INTEGER COLON INTEGER */ {yymsp[-2].minor.yy0.n += (yymsp[-1].minor.yy0.n + yymsp[0].minor.yy0.n); setKillSql(pInfo, TSDB_SQL_KILL_STREAM, &yymsp[-2].minor.yy0);} break; - case 269: /* cmd ::= KILL QUERY INTEGER COLON INTEGER */ + case 272: /* cmd ::= KILL QUERY INTEGER COLON INTEGER */ {yymsp[-2].minor.yy0.n += (yymsp[-1].minor.yy0.n + yymsp[0].minor.yy0.n); setKillSql(pInfo, TSDB_SQL_KILL_QUERY, &yymsp[-2].minor.yy0);} break; default: diff --git a/src/query/tests/unitTest.cpp b/src/query/tests/unitTest.cpp index d2b058cf7cb9400c64149ff8f18f68788735ba96..33ba8200d3afb9cff00f150ab5bef799f3fa1e86 100644 --- a/src/query/tests/unitTest.cpp +++ b/src/query/tests/unitTest.cpp @@ -99,47 +99,47 @@ TEST(testCase, db_table_name) { EXPECT_EQ(testValidateName(t4), TSDB_CODE_SUCCESS); char t5[] = "table.'def'"; - EXPECT_EQ(testValidateName(t5), TSDB_CODE_TSC_INVALID_SQL); + EXPECT_EQ(testValidateName(t5), TSDB_CODE_TSC_INVALID_OPERATION); char t6[] = "'table'.'def'"; - EXPECT_EQ(testValidateName(t6), TSDB_CODE_TSC_INVALID_SQL); + EXPECT_EQ(testValidateName(t6), TSDB_CODE_TSC_INVALID_OPERATION); char t7[] = "'_ab1234'.'def'"; EXPECT_EQ(testValidateName(t7), TSDB_CODE_SUCCESS); printf("%s\n", t7); char t8[] = "'_ab&^%1234'.'def'"; - EXPECT_EQ(testValidateName(t8), TSDB_CODE_TSC_INVALID_SQL); + EXPECT_EQ(testValidateName(t8), TSDB_CODE_TSC_INVALID_OPERATION); char t9[] = "'_123'.'gtest中文'"; - EXPECT_EQ(testValidateName(t9), TSDB_CODE_TSC_INVALID_SQL); + EXPECT_EQ(testValidateName(t9), TSDB_CODE_TSC_INVALID_OPERATION); char t10[] = "abc.'gtest中文'"; - EXPECT_EQ(testValidateName(t10), TSDB_CODE_TSC_INVALID_SQL); + EXPECT_EQ(testValidateName(t10), TSDB_CODE_TSC_INVALID_OPERATION); char t10_1[] = "abc.'中文gtest'"; - EXPECT_EQ(testValidateName(t10_1), TSDB_CODE_TSC_INVALID_SQL); + EXPECT_EQ(testValidateName(t10_1), TSDB_CODE_TSC_INVALID_OPERATION); char t11[] = "'192.168.0.1'.abc"; - EXPECT_EQ(testValidateName(t11), TSDB_CODE_TSC_INVALID_SQL); + EXPECT_EQ(testValidateName(t11), TSDB_CODE_TSC_INVALID_OPERATION); char t12[] = "192.168.0.1.abc"; - EXPECT_EQ(testValidateName(t12), TSDB_CODE_TSC_INVALID_SQL); + EXPECT_EQ(testValidateName(t12), TSDB_CODE_TSC_INVALID_OPERATION); char t13[] = "abc."; - EXPECT_EQ(testValidateName(t13), TSDB_CODE_TSC_INVALID_SQL); + EXPECT_EQ(testValidateName(t13), TSDB_CODE_TSC_INVALID_OPERATION); char t14[] = ".abc"; - EXPECT_EQ(testValidateName(t14), TSDB_CODE_TSC_INVALID_SQL); + EXPECT_EQ(testValidateName(t14), TSDB_CODE_TSC_INVALID_OPERATION); char t15[] = ".'abc'"; - EXPECT_EQ(testValidateName(t15), TSDB_CODE_TSC_INVALID_SQL); + EXPECT_EQ(testValidateName(t15), TSDB_CODE_TSC_INVALID_OPERATION); char t16[] = ".abc'"; - EXPECT_EQ(testValidateName(t16), TSDB_CODE_TSC_INVALID_SQL); + EXPECT_EQ(testValidateName(t16), TSDB_CODE_TSC_INVALID_OPERATION); char t17[] = "123a.\"abc\""; - EXPECT_EQ(testValidateName(t17), TSDB_CODE_TSC_INVALID_SQL); + EXPECT_EQ(testValidateName(t17), TSDB_CODE_TSC_INVALID_OPERATION); printf("%s\n", t17); char t18[] = "a.\"abc\""; @@ -147,13 +147,13 @@ TEST(testCase, db_table_name) { printf("%s\n", t18); char t19[] = "'_ab1234'.'def'.'ab123'"; - EXPECT_EQ(testValidateName(t19), TSDB_CODE_TSC_INVALID_SQL); + EXPECT_EQ(testValidateName(t19), TSDB_CODE_TSC_INVALID_OPERATION); char t20[] = "'_ab1234*&^'"; - EXPECT_EQ(testValidateName(t20), TSDB_CODE_TSC_INVALID_SQL); + EXPECT_EQ(testValidateName(t20), TSDB_CODE_TSC_INVALID_OPERATION); char t21[] = "'1234_abc'"; - EXPECT_EQ(testValidateName(t21), TSDB_CODE_TSC_INVALID_SQL); + EXPECT_EQ(testValidateName(t21), TSDB_CODE_TSC_INVALID_OPERATION); // =======Containing capital letters================= @@ -167,10 +167,10 @@ TEST(testCase, db_table_name) { EXPECT_EQ(testValidateName(t32), TSDB_CODE_SUCCESS); char t33[] = "'ABC.def"; - EXPECT_EQ(testValidateName(t33), TSDB_CODE_TSC_INVALID_SQL); + EXPECT_EQ(testValidateName(t33), TSDB_CODE_TSC_INVALID_OPERATION); char t33_0[] = "abc.DEF'"; - EXPECT_EQ(testValidateName(t33_0), TSDB_CODE_TSC_INVALID_SQL); + EXPECT_EQ(testValidateName(t33_0), TSDB_CODE_TSC_INVALID_OPERATION); char t34[] = "'ABC.def'"; //int32_t tmp0 = testValidateName(t34); @@ -193,136 +193,136 @@ TEST(testCase, db_table_name) { // do not use key words char t39[] = "table.'DEF'"; - EXPECT_EQ(testValidateName(t39), TSDB_CODE_TSC_INVALID_SQL); + EXPECT_EQ(testValidateName(t39), TSDB_CODE_TSC_INVALID_OPERATION); char t40[] = "'table'.'DEF'"; - EXPECT_EQ(testValidateName(t40), TSDB_CODE_TSC_INVALID_SQL); + EXPECT_EQ(testValidateName(t40), TSDB_CODE_TSC_INVALID_OPERATION); char t41[] = "'_abXYZ1234'.'deFF'"; EXPECT_EQ(testValidateName(t41), TSDB_CODE_SUCCESS); char t42[] = "'_abDEF&^%1234'.'DIef'"; - EXPECT_EQ(testValidateName(t42), TSDB_CODE_TSC_INVALID_SQL); + EXPECT_EQ(testValidateName(t42), TSDB_CODE_TSC_INVALID_OPERATION); char t43[] = "'_123'.'Gtest中文'"; - EXPECT_EQ(testValidateName(t43), TSDB_CODE_TSC_INVALID_SQL); + EXPECT_EQ(testValidateName(t43), TSDB_CODE_TSC_INVALID_OPERATION); char t44[] = "'aABC'.'Gtest中文'"; - EXPECT_EQ(testValidateName(t44), TSDB_CODE_TSC_INVALID_SQL); + EXPECT_EQ(testValidateName(t44), TSDB_CODE_TSC_INVALID_OPERATION); char t45[] = "'ABC'."; - EXPECT_EQ(testValidateName(t45), TSDB_CODE_TSC_INVALID_SQL); + EXPECT_EQ(testValidateName(t45), TSDB_CODE_TSC_INVALID_OPERATION); char t46[] = ".'ABC'"; - EXPECT_EQ(testValidateName(t46), TSDB_CODE_TSC_INVALID_SQL); + EXPECT_EQ(testValidateName(t46), TSDB_CODE_TSC_INVALID_OPERATION); char t47[] = "a.\"aTWc\""; EXPECT_EQ(testValidateName(t47), TSDB_CODE_SUCCESS); // ================has space ================= char t60[] = " ABC "; - EXPECT_EQ(testValidateName(t60), TSDB_CODE_TSC_INVALID_SQL); + EXPECT_EQ(testValidateName(t60), TSDB_CODE_TSC_INVALID_OPERATION); char t60_1[] = " ABC "; - EXPECT_EQ(testValidateName(t60_1), TSDB_CODE_TSC_INVALID_SQL); + EXPECT_EQ(testValidateName(t60_1), TSDB_CODE_TSC_INVALID_OPERATION); char t61[] = "' ABC '"; - EXPECT_EQ(testValidateName(t61), TSDB_CODE_TSC_INVALID_SQL); + EXPECT_EQ(testValidateName(t61), TSDB_CODE_TSC_INVALID_OPERATION); char t61_1[] = "' ABC '"; - EXPECT_EQ(testValidateName(t61_1), TSDB_CODE_TSC_INVALID_SQL); + EXPECT_EQ(testValidateName(t61_1), TSDB_CODE_TSC_INVALID_OPERATION); char t62[] = " ABC . def "; - EXPECT_EQ(testValidateName(t62), TSDB_CODE_TSC_INVALID_SQL); + EXPECT_EQ(testValidateName(t62), TSDB_CODE_TSC_INVALID_OPERATION); char t63[] = "' ABC . def "; - EXPECT_EQ(testValidateName(t63), TSDB_CODE_TSC_INVALID_SQL); + EXPECT_EQ(testValidateName(t63), TSDB_CODE_TSC_INVALID_OPERATION); char t63_0[] = " abc . DEF ' "; - EXPECT_EQ(testValidateName(t63_0), TSDB_CODE_TSC_INVALID_SQL); + EXPECT_EQ(testValidateName(t63_0), TSDB_CODE_TSC_INVALID_OPERATION); char t64[] = " ' ABC . def ' "; //int32_t tmp1 = testValidateName(t64); - EXPECT_EQ(testValidateName(t64), TSDB_CODE_TSC_INVALID_SQL); + EXPECT_EQ(testValidateName(t64), TSDB_CODE_TSC_INVALID_OPERATION); char t65[] = " ' ABC '. def "; - EXPECT_EQ(testValidateName(t65), TSDB_CODE_TSC_INVALID_SQL); + EXPECT_EQ(testValidateName(t65), TSDB_CODE_TSC_INVALID_OPERATION); char t66[] = "' ABC '.' DEF '"; - EXPECT_EQ(testValidateName(t66), TSDB_CODE_TSC_INVALID_SQL); + EXPECT_EQ(testValidateName(t66), TSDB_CODE_TSC_INVALID_OPERATION); char t67[] = "abc . ' DEF '"; - EXPECT_EQ(testValidateName(t67), TSDB_CODE_TSC_INVALID_SQL); + EXPECT_EQ(testValidateName(t67), TSDB_CODE_TSC_INVALID_OPERATION); char t68[] = "' abc '.' DEF '"; - EXPECT_EQ(testValidateName(t68), TSDB_CODE_TSC_INVALID_SQL); + EXPECT_EQ(testValidateName(t68), TSDB_CODE_TSC_INVALID_OPERATION); // do not use key words char t69[] = "table.'DEF'"; - EXPECT_EQ(testValidateName(t69), TSDB_CODE_TSC_INVALID_SQL); + EXPECT_EQ(testValidateName(t69), TSDB_CODE_TSC_INVALID_OPERATION); char t70[] = "'table'.'DEF'"; - EXPECT_EQ(testValidateName(t70), TSDB_CODE_TSC_INVALID_SQL); + EXPECT_EQ(testValidateName(t70), TSDB_CODE_TSC_INVALID_OPERATION); char t71[] = "'_abXYZ1234 '.' deFF '"; - EXPECT_EQ(testValidateName(t71), TSDB_CODE_TSC_INVALID_SQL); + EXPECT_EQ(testValidateName(t71), TSDB_CODE_TSC_INVALID_OPERATION); char t72[] = "'_abDEF&^%1234'.' DIef'"; - EXPECT_EQ(testValidateName(t72), TSDB_CODE_TSC_INVALID_SQL); + EXPECT_EQ(testValidateName(t72), TSDB_CODE_TSC_INVALID_OPERATION); char t73[] = "'_123'.' Gtest中文'"; - EXPECT_EQ(testValidateName(t73), TSDB_CODE_TSC_INVALID_SQL); + EXPECT_EQ(testValidateName(t73), TSDB_CODE_TSC_INVALID_OPERATION); char t74[] = "' aABC'.'Gtest中文'"; - EXPECT_EQ(testValidateName(t74), TSDB_CODE_TSC_INVALID_SQL); + EXPECT_EQ(testValidateName(t74), TSDB_CODE_TSC_INVALID_OPERATION); char t75[] = "' ABC '."; - EXPECT_EQ(testValidateName(t75), TSDB_CODE_TSC_INVALID_SQL); + EXPECT_EQ(testValidateName(t75), TSDB_CODE_TSC_INVALID_OPERATION); char t76[] = ".' ABC'"; - EXPECT_EQ(testValidateName(t76), TSDB_CODE_TSC_INVALID_SQL); + EXPECT_EQ(testValidateName(t76), TSDB_CODE_TSC_INVALID_OPERATION); char t77[] = " a . \"aTWc\" "; - EXPECT_EQ(testValidateName(t77), TSDB_CODE_TSC_INVALID_SQL); + EXPECT_EQ(testValidateName(t77), TSDB_CODE_TSC_INVALID_OPERATION); char t78[] = " a.\"aTWc \""; - EXPECT_EQ(testValidateName(t78), TSDB_CODE_TSC_INVALID_SQL); + EXPECT_EQ(testValidateName(t78), TSDB_CODE_TSC_INVALID_OPERATION); // ===============muti string by space =================== // There's no such case. //char t160[] = "A BC"; - //EXPECT_EQ(testValidateName(t160), TSDB_CODE_TSC_INVALID_SQL); + //EXPECT_EQ(testValidateName(t160), TSDB_CODE_TSC_INVALID_OPERATION); //printf("end:%s\n", t160); // There's no such case. //char t161[] = "' A BC '"; - //EXPECT_EQ(testValidateName(t161), TSDB_CODE_TSC_INVALID_SQL); + //EXPECT_EQ(testValidateName(t161), TSDB_CODE_TSC_INVALID_OPERATION); char t162[] = " AB C . de f "; - EXPECT_EQ(testValidateName(t162), TSDB_CODE_TSC_INVALID_SQL); + EXPECT_EQ(testValidateName(t162), TSDB_CODE_TSC_INVALID_OPERATION); char t163[] = "' AB C . de f "; - EXPECT_EQ(testValidateName(t163), TSDB_CODE_TSC_INVALID_SQL); + EXPECT_EQ(testValidateName(t163), TSDB_CODE_TSC_INVALID_OPERATION); char t163_0[] = " ab c . DE F ' "; - EXPECT_EQ(testValidateName(t163_0), TSDB_CODE_TSC_INVALID_SQL); + EXPECT_EQ(testValidateName(t163_0), TSDB_CODE_TSC_INVALID_OPERATION); char t164[] = " ' AB C . de f ' "; //int32_t tmp2 = testValidateName(t164); - EXPECT_EQ(testValidateName(t164), TSDB_CODE_TSC_INVALID_SQL); + EXPECT_EQ(testValidateName(t164), TSDB_CODE_TSC_INVALID_OPERATION); char t165[] = " ' A BC '. de f "; - EXPECT_EQ(testValidateName(t165), TSDB_CODE_TSC_INVALID_SQL); + EXPECT_EQ(testValidateName(t165), TSDB_CODE_TSC_INVALID_OPERATION); char t166[] = "' AB C '.' DE F '"; - EXPECT_EQ(testValidateName(t166), TSDB_CODE_TSC_INVALID_SQL); + EXPECT_EQ(testValidateName(t166), TSDB_CODE_TSC_INVALID_OPERATION); char t167[] = "ab c . ' D EF '"; - EXPECT_EQ(testValidateName(t167), TSDB_CODE_TSC_INVALID_SQL); + EXPECT_EQ(testValidateName(t167), TSDB_CODE_TSC_INVALID_OPERATION); char t168[] = "' a bc '.' DE F '"; - EXPECT_EQ(testValidateName(t168), TSDB_CODE_TSC_INVALID_SQL); + EXPECT_EQ(testValidateName(t168), TSDB_CODE_TSC_INVALID_OPERATION); } diff --git a/src/sync/src/syncMain.c b/src/sync/src/syncMain.c index 3d8f54ad9b643143235804a68825c0803c8ce81b..493d5f117ac4d213da2703ed60825209f19f46b4 100644 --- a/src/sync/src/syncMain.c +++ b/src/sync/src/syncMain.c @@ -709,7 +709,7 @@ static void syncChooseMaster(SSyncNode *pNode) { } static SSyncPeer *syncCheckMaster(SSyncNode *pNode) { - int32_t onlineNum = 0; + int32_t onlineNum = 0, arbOnlineNum = 0; int32_t masterIndex = -1; int32_t replica = pNode->replica; @@ -723,13 +723,15 @@ static SSyncPeer *syncCheckMaster(SSyncNode *pNode) { SSyncPeer *pArb = pNode->peerInfo[TAOS_SYNC_MAX_REPLICA]; if (pArb && pArb->role != TAOS_SYNC_ROLE_OFFLINE) { onlineNum++; + ++arbOnlineNum; replica = pNode->replica + 1; } if (onlineNum <= replica * 0.5) { if (nodeRole != TAOS_SYNC_ROLE_UNSYNCED) { - if (nodeRole == TAOS_SYNC_ROLE_MASTER && onlineNum == replica * 0.5 && onlineNum >= 1) { + if (nodeRole == TAOS_SYNC_ROLE_MASTER && onlineNum == replica * 0.5 && ((replica > 2 && onlineNum - arbOnlineNum > 1) || pNode->replica < 3)) { sInfo("vgId:%d, self keep work as master, online:%d replica:%d", pNode->vgId, onlineNum, replica); + masterIndex = pNode->selfIndex; } else { nodeRole = TAOS_SYNC_ROLE_UNSYNCED; sInfo("vgId:%d, self change to unsynced state, online:%d replica:%d", pNode->vgId, onlineNum, replica); @@ -1002,6 +1004,7 @@ static void syncProcessForwardFromPeer(char *cont, SSyncPeer *pPeer) { if (nodeRole == TAOS_SYNC_ROLE_SLAVE) { // nodeVersion = pHead->version; code = (*pNode->writeToCacheFp)(pNode->vgId, pHead, TAOS_QTYPE_FWD, NULL); + syncConfirmForward(pNode->rid, pHead->version, code, false); } else { if (nodeSStatus != TAOS_SYNC_STATUS_INIT) { code = syncSaveIntoBuffer(pPeer, pHead); @@ -1404,7 +1407,7 @@ static void syncMonitorFwdInfos(void *param, void *tmrId) { pthread_mutex_lock(&pNode->mutex); for (int32_t i = 0; i < pSyncFwds->fwds; ++i) { SFwdInfo *pFwdInfo = pSyncFwds->fwdInfo + (pSyncFwds->first + i) % SYNC_MAX_FWDS; - if (ABS(time - pFwdInfo->time) < 2000) break; + if (ABS(time - pFwdInfo->time) < 10000) break; sDebug("vgId:%d, forward info expired, hver:%" PRIu64 " curtime:%" PRIu64 " savetime:%" PRIu64, pNode->vgId, pFwdInfo->version, time, pFwdInfo->time); diff --git a/src/tsdb/inc/tsdbMeta.h b/src/tsdb/inc/tsdbMeta.h index 7484071ce3c3bdb9e5aad449f22fc36d1809271a..45bbd5a7c6911fed4ea7309a77d3ac144109a34b 100644 --- a/src/tsdb/inc/tsdbMeta.h +++ b/src/tsdb/inc/tsdbMeta.h @@ -36,6 +36,12 @@ typedef struct STable { char* sql; void* cqhandle; SRWLatch latch; // TODO: implementa latch functions + + SDataCol *lastCols; + int16_t maxColNum; + int16_t restoreColumnNum; + bool hasRestoreLastColumn; + int lastColSVersion; T_REF_DECLARE() } STable; @@ -78,6 +84,11 @@ void tsdbUnRefTable(STable* pTable); void tsdbUpdateTableSchema(STsdbRepo* pRepo, STable* pTable, STSchema* pSchema, bool insertAct); int tsdbRestoreTable(STsdbRepo* pRepo, void* cont, int contLen); void tsdbOrgMeta(STsdbRepo* pRepo); +int tsdbInitColIdCacheWithSchema(STable* pTable, STSchema* pSchema); +int16_t tsdbGetLastColumnsIndexByColId(STable* pTable, int16_t colId); +int tsdbUpdateLastColSchema(STable *pTable, STSchema *pNewSchema); +STSchema* tsdbGetTableLatestSchema(STable *pTable); +void tsdbFreeLastColumns(STable* pTable); static FORCE_INLINE int tsdbCompareSchemaVersion(const void *key1, const void *key2) { if (*(int16_t *)key1 < schemaVersion(*(STSchema **)key2)) { diff --git a/src/tsdb/inc/tsdbint.h b/src/tsdb/inc/tsdbint.h index 4d62164df9920c916e7a01cd67496711707b76e1..e74c3238e2ac1d70087c615e4a3e6310a0cb3050 100644 --- a/src/tsdb/inc/tsdbint.h +++ b/src/tsdb/inc/tsdbint.h @@ -75,6 +75,9 @@ struct STsdbRepo { STsdbCfg save_config; // save apply config bool config_changed; // config changed flag pthread_mutex_t save_mutex; // protect save config + + uint8_t hasCachedLastRow; + uint8_t hasCachedLastColumn; STsdbAppH appH; STsdbStat stat; @@ -100,6 +103,7 @@ int tsdbUnlockRepo(STsdbRepo* pRepo); STsdbMeta* tsdbGetMeta(STsdbRepo* pRepo); int tsdbCheckCommit(STsdbRepo* pRepo); int tsdbRestoreInfo(STsdbRepo* pRepo); +int tsdbCacheLastData(STsdbRepo *pRepo, STsdbCfg* oldCfg); void tsdbGetRootDir(int repoid, char dirName[]); void tsdbGetDataDir(int repoid, char dirName[]); diff --git a/src/tsdb/src/tsdbCommit.c b/src/tsdb/src/tsdbCommit.c index 24f1c1162889ac15810de72e31b70dc2bf3219b0..734b47a718cbf43fc24982dd3bbc6f862c94afd2 100644 --- a/src/tsdb/src/tsdbCommit.c +++ b/src/tsdb/src/tsdbCommit.c @@ -90,6 +90,9 @@ static int tsdbApplyRtn(STsdbRepo *pRepo); static int tsdbApplyRtnOnFSet(STsdbRepo *pRepo, SDFileSet *pSet, SRtn *pRtn); void *tsdbCommitData(STsdbRepo *pRepo) { + if (pRepo->imem == NULL) { + return NULL; + } tsdbStartCommit(pRepo); // Commit to update meta file @@ -1149,7 +1152,7 @@ static int tsdbCommitAddBlock(SCommitH *pCommith, const SBlock *pSupBlock, const return -1; } - if (pSubBlocks && taosArrayPushBatch(pCommith->aSubBlk, pSubBlocks, nSubBlocks) == NULL) { + if (pSubBlocks && taosArrayAddBatch(pCommith->aSubBlk, pSubBlocks, nSubBlocks) == NULL) { terrno = TSDB_CODE_TDB_OUT_OF_MEMORY; return -1; } diff --git a/src/tsdb/src/tsdbCommitQueue.c b/src/tsdb/src/tsdbCommitQueue.c index e753a3211e5a789548ee6fc6f7caca6ccd3b59c3..abea79bc4f46129055af868af459c147dd93793f 100644 --- a/src/tsdb/src/tsdbCommitQueue.c +++ b/src/tsdb/src/tsdbCommitQueue.c @@ -113,11 +113,15 @@ int tsdbScheduleCommit(STsdbRepo *pRepo) { } static void tsdbApplyRepoConfig(STsdbRepo *pRepo) { + pthread_mutex_lock(&pRepo->save_mutex); + pRepo->config_changed = false; STsdbCfg * pSaveCfg = &pRepo->save_config; - + STsdbCfg oldCfg; int32_t oldTotalBlocks = pRepo->config.totalBlocks; + memcpy(&oldCfg, &(pRepo->config), sizeof(STsdbCfg)); + pRepo->config.compression = pRepo->save_config.compression; pRepo->config.keep = pRepo->save_config.keep; pRepo->config.keep1 = pRepo->save_config.keep1; @@ -125,10 +129,12 @@ static void tsdbApplyRepoConfig(STsdbRepo *pRepo) { pRepo->config.cacheLastRow = pRepo->save_config.cacheLastRow; pRepo->config.totalBlocks = pRepo->save_config.totalBlocks; - tsdbInfo("vgId:%d apply new config: compression(%d), keep(%d,%d,%d), totalBlocks(%d), cacheLastRow(%d),totalBlocks(%d)", + pthread_mutex_unlock(&pRepo->save_mutex); + + tsdbInfo("vgId:%d apply new config: compression(%d), keep(%d,%d,%d), totalBlocks(%d), cacheLastRow(%d->%d),totalBlocks(%d->%d)", REPO_ID(pRepo), pSaveCfg->compression, pSaveCfg->keep,pSaveCfg->keep1, pSaveCfg->keep2, - pSaveCfg->totalBlocks, pSaveCfg->cacheLastRow, pSaveCfg->totalBlocks); + pSaveCfg->totalBlocks, oldCfg.cacheLastRow, pSaveCfg->cacheLastRow, oldTotalBlocks, pSaveCfg->totalBlocks); int err = tsdbExpendPool(pRepo, oldTotalBlocks); if (!TAOS_SUCCEEDED(err)) { @@ -136,6 +142,12 @@ static void tsdbApplyRepoConfig(STsdbRepo *pRepo) { REPO_ID(pRepo), oldTotalBlocks, pSaveCfg->totalBlocks, tstrerror(err)); } + if (oldCfg.cacheLastRow != pRepo->config.cacheLastRow) { + if (tsdbLockRepo(pRepo) < 0) return; + tsdbCacheLastData(pRepo, &oldCfg); + tsdbUnlockRepo(pRepo); + } + } static void *tsdbLoopCommit(void *arg) { @@ -165,10 +177,8 @@ static void *tsdbLoopCommit(void *arg) { pRepo = ((SCommitReq *)pNode->data)->pRepo; // check if need to apply new config - if (pRepo->config_changed) { - pthread_mutex_lock(&pRepo->save_mutex); + if (pRepo->config_changed) { tsdbApplyRepoConfig(pRepo); - pthread_mutex_unlock(&pRepo->save_mutex); } tsdbCommitData(pRepo); diff --git a/src/tsdb/src/tsdbMain.c b/src/tsdb/src/tsdbMain.c index fd02a3c8b97d7506209d92661143a158d8d94951..afbedd5b2fd231606902db104916e4ff4f10ba67 100644 --- a/src/tsdb/src/tsdbMain.c +++ b/src/tsdb/src/tsdbMain.c @@ -26,6 +26,8 @@ static STsdbRepo *tsdbNewRepo(STsdbCfg *pCfg, STsdbAppH *pAppH); static void tsdbFreeRepo(STsdbRepo *pRepo); static void tsdbStartStream(STsdbRepo *pRepo); static void tsdbStopStream(STsdbRepo *pRepo); +static int tsdbRestoreLastColumns(STsdbRepo *pRepo, STable *pTable, SReadH* pReadh); +static int tsdbRestoreLastRow(STsdbRepo *pRepo, STable *pTable, SReadH* pReadh, SBlockIdx *pIdx); // Function declaration int32_t tsdbCreateRepo(int repoid) { @@ -267,6 +269,10 @@ int32_t tsdbConfigRepo(STsdbRepo *repo, STsdbCfg *pCfg) { repo->config_changed = true; pthread_mutex_unlock(&repo->save_mutex); + + // schedule a commit msg then the new config will be applied immediatly + tsdbAsyncCommit(repo); + return 0; #if 0 STsdbRepo *pRepo = (STsdbRepo *)repo; @@ -511,8 +517,10 @@ static int32_t tsdbCheckAndSetDefaultCfg(STsdbCfg *pCfg) { if (pCfg->update != 0) pCfg->update = 1; // update cacheLastRow - if (pCfg->cacheLastRow != 0) pCfg->cacheLastRow = 1; - + if (pCfg->cacheLastRow != 0) { + if (pCfg->cacheLastRow > 3) + pCfg->cacheLastRow = 1; + } return 0; } @@ -545,6 +553,8 @@ static STsdbRepo *tsdbNewRepo(STsdbCfg *pCfg, STsdbAppH *pAppH) { return NULL; } pRepo->config_changed = false; + atomic_store_8(&pRepo->hasCachedLastRow, 0); + atomic_store_8(&pRepo->hasCachedLastColumn, 0); code = tsem_init(&(pRepo->readyToCommit), 0, 1); if (code != 0) { @@ -614,13 +624,180 @@ static void tsdbStopStream(STsdbRepo *pRepo) { } } +static int tsdbRestoreLastColumns(STsdbRepo *pRepo, STable *pTable, SReadH* pReadh) { + //tsdbInfo("tsdbRestoreLastColumns of table %s", pTable->name->data); + + STSchema *pSchema = tsdbGetTableLatestSchema(pTable); + if (pSchema == NULL) { + tsdbError("tsdbGetTableLatestSchema of table %s fail", pTable->name->data); + return 0; + } + + SBlock* pBlock; + int numColumns; + int32_t blockIdx; + SDataStatis* pBlockStatis = NULL; + SDataRow row = NULL; + // restore last column data with last schema + + int err = 0; + + numColumns = schemaNCols(pSchema); + if (numColumns <= pTable->restoreColumnNum) { + pTable->hasRestoreLastColumn = true; + return 0; + } + if (pTable->lastColSVersion != schemaVersion(pSchema)) { + if (tsdbInitColIdCacheWithSchema(pTable, pSchema) < 0) { + return -1; + } + } + + row = taosTMalloc(dataRowMaxBytesFromSchema(pSchema)); + if (row == NULL) { + terrno = TSDB_CODE_TDB_OUT_OF_MEMORY; + err = -1; + goto out; + } + tdInitDataRow(row, pSchema); + + // first load block index info + if (tsdbLoadBlockInfo(pReadh, NULL) < 0) { + err = -1; + goto out; + } + + pBlockStatis = calloc(numColumns, sizeof(SDataStatis)); + if (pBlockStatis == NULL) { + terrno = TSDB_CODE_TDB_OUT_OF_MEMORY; + err = -1; + goto out; + } + memset(pBlockStatis, 0, numColumns * sizeof(SDataStatis)); + for(int32_t i = 0; i < numColumns; ++i) { + STColumn *pCol = schemaColAt(pSchema, i); + pBlockStatis[i].colId = pCol->colId; + } + + // load block from backward + SBlockIdx *pIdx = pReadh->pBlkIdx; + blockIdx = (int32_t)(pIdx->numOfBlocks - 1); + + while (numColumns > pTable->restoreColumnNum && blockIdx >= 0) { + bool loadStatisData = false; + pBlock = pReadh->pBlkInfo->blocks + blockIdx; + blockIdx -= 1; + + // load block data + if (tsdbLoadBlockData(pReadh, pBlock, NULL) < 0) { + err = -1; + goto out; + } + + // file block with sub-blocks has no statistics data + if (pBlock->numOfSubBlocks <= 1) { + tsdbLoadBlockStatis(pReadh, pBlock); + tsdbGetBlockStatis(pReadh, pBlockStatis, (int)numColumns); + loadStatisData = true; + } + + for (int16_t i = 0; i < numColumns && numColumns > pTable->restoreColumnNum; ++i) { + STColumn *pCol = schemaColAt(pSchema, i); + // ignore loaded columns + if (pTable->lastCols[i].bytes != 0) { + continue; + } + + // ignore block which has no not-null colId column + if (loadStatisData && pBlockStatis[i].numOfNull == pBlock->numOfRows) { + continue; + } + + // OK,let's load row from backward to get not-null column + for (int32_t rowId = pBlock->numOfRows - 1; rowId >= 0; rowId--) { + SDataCol *pDataCol = pReadh->pDCols[0]->cols + i; + tdAppendColVal(row, tdGetColDataOfRow(pDataCol, rowId), pCol->type, pCol->bytes, pCol->offset); + //SDataCol *pDataCol = readh.pDCols[0]->cols + j; + void* value = tdGetRowDataOfCol(row, (int8_t)pCol->type, TD_DATA_ROW_HEAD_SIZE + pCol->offset); + if (isNull(value, pCol->type)) { + continue; + } + + int16_t idx = tsdbGetLastColumnsIndexByColId(pTable, pCol->colId); + if (idx == -1) { + tsdbError("tsdbRestoreLastColumns restore vgId:%d,table:%s cache column %d fail", REPO_ID(pRepo), pTable->name->data, pCol->colId); + continue; + } + // save not-null column + SDataCol *pLastCol = &(pTable->lastCols[idx]); + pLastCol->pData = malloc(pCol->bytes); + pLastCol->bytes = pCol->bytes; + pLastCol->colId = pCol->colId; + memcpy(pLastCol->pData, value, pCol->bytes); + + // save row ts(in column 0) + pDataCol = pReadh->pDCols[0]->cols + 0; + pCol = schemaColAt(pSchema, 0); + tdAppendColVal(row, tdGetColDataOfRow(pDataCol, rowId), pCol->type, pCol->bytes, pCol->offset); + pLastCol->ts = dataRowKey(row); + + pTable->restoreColumnNum += 1; + + tsdbDebug("tsdbRestoreLastColumns restore vgId:%d,table:%s cache column %d, %" PRId64, REPO_ID(pRepo), pTable->name->data, pLastCol->colId, pLastCol->ts); + break; + } + } + } + +out: + taosTZfree(row); + tfree(pBlockStatis); + + if (err == 0 && numColumns <= pTable->restoreColumnNum) { + pTable->hasRestoreLastColumn = true; + } + + return err; +} + +static int tsdbRestoreLastRow(STsdbRepo *pRepo, STable *pTable, SReadH* pReadh, SBlockIdx *pIdx) { + ASSERT(pTable->lastRow == NULL); + if (tsdbLoadBlockInfo(pReadh, NULL) < 0) { + return -1; + } + + SBlock* pBlock = pReadh->pBlkInfo->blocks + pIdx->numOfBlocks - 1; + + if (tsdbLoadBlockData(pReadh, pBlock, NULL) < 0) { + return -1; + } + + // Get the data in row + + STSchema *pSchema = tsdbGetTableSchema(pTable); + pTable->lastRow = taosTMalloc(dataRowMaxBytesFromSchema(pSchema)); + if (pTable->lastRow == NULL) { + terrno = TSDB_CODE_TDB_OUT_OF_MEMORY; + return -1; + } + + tdInitDataRow(pTable->lastRow, pSchema); + for (int icol = 0; icol < schemaNCols(pSchema); icol++) { + STColumn *pCol = schemaColAt(pSchema, icol); + SDataCol *pDataCol = pReadh->pDCols[0]->cols + icol; + tdAppendColVal(pTable->lastRow, tdGetColDataOfRow(pDataCol, pBlock->numOfRows - 1), pCol->type, pCol->bytes, + pCol->offset); + } + + return 0; +} + int tsdbRestoreInfo(STsdbRepo *pRepo) { SFSIter fsiter; SReadH readh; SDFileSet *pSet; STsdbMeta *pMeta = pRepo->tsdbMeta; STsdbCfg * pCfg = REPO_CFG(pRepo); - SBlock * pBlock; if (tsdbInitReadH(&readh, pRepo) < 0) { return -1; @@ -628,6 +805,14 @@ int tsdbRestoreInfo(STsdbRepo *pRepo) { tsdbFSIterInit(&fsiter, REPO_FS(pRepo), TSDB_FS_ITER_BACKWARD); + if (CACHE_LAST_NULL_COLUMN(pCfg)) { + for (int i = 1; i < pMeta->maxTables; i++) { + STable *pTable = pMeta->tables[i]; + if (pTable == NULL) continue; + pTable->restoreColumnNum = 0; + } + } + while ((pSet = tsdbFSIterNext(&fsiter)) != NULL) { if (tsdbSetAndOpenReadFSet(&readh, pSet) < 0) { tsdbDestroyReadH(&readh); @@ -643,6 +828,8 @@ int tsdbRestoreInfo(STsdbRepo *pRepo) { STable *pTable = pMeta->tables[i]; if (pTable == NULL) continue; + //tsdbInfo("tsdbRestoreInfo restore vgId:%d,table:%s", REPO_ID(pRepo), pTable->name->data); + if (tsdbSetReadTable(&readh, pTable) < 0) { tsdbDestroyReadH(&readh); return -1; @@ -653,42 +840,155 @@ int tsdbRestoreInfo(STsdbRepo *pRepo) { if (pIdx && lastKey < pIdx->maxKey) { pTable->lastKey = pIdx->maxKey; - if (pCfg->cacheLastRow) { - if (tsdbLoadBlockInfo(&readh, NULL) < 0) { - tsdbDestroyReadH(&readh); - return -1; - } - - pBlock = readh.pBlkInfo->blocks + pIdx->numOfBlocks - 1; - - if (tsdbLoadBlockData(&readh, pBlock, NULL) < 0) { - tsdbDestroyReadH(&readh); - return -1; - } - - // Get the data in row - ASSERT(pTable->lastRow == NULL); - STSchema *pSchema = tsdbGetTableSchema(pTable); - pTable->lastRow = taosTMalloc(dataRowMaxBytesFromSchema(pSchema)); - if (pTable->lastRow == NULL) { - terrno = TSDB_CODE_TDB_OUT_OF_MEMORY; - tsdbDestroyReadH(&readh); - return -1; - } - - tdInitDataRow(pTable->lastRow, pSchema); - for (int icol = 0; icol < schemaNCols(pSchema); icol++) { - STColumn *pCol = schemaColAt(pSchema, icol); - SDataCol *pDataCol = readh.pDCols[0]->cols + icol; - tdAppendColVal(pTable->lastRow, tdGetColDataOfRow(pDataCol, pBlock->numOfRows - 1), pCol->type, pCol->bytes, - pCol->offset); - } + if (CACHE_LAST_ROW(pCfg) && tsdbRestoreLastRow(pRepo, pTable, &readh, pIdx) != 0) { + tsdbDestroyReadH(&readh); + return -1; } } + // restore NULL columns + if (pIdx && CACHE_LAST_NULL_COLUMN(pCfg) && !pTable->hasRestoreLastColumn) { + if (tsdbRestoreLastColumns(pRepo, pTable, &readh) != 0) { + tsdbDestroyReadH(&readh); + return -1; + } + } } } tsdbDestroyReadH(&readh); + if (CACHE_LAST_ROW(pCfg)) { + atomic_store_8(&pRepo->hasCachedLastRow, 1); + } + if (CACHE_LAST_NULL_COLUMN(pCfg)) { + atomic_store_8(&pRepo->hasCachedLastColumn, 1); + } + return 0; } + +int tsdbCacheLastData(STsdbRepo *pRepo, STsdbCfg* oldCfg) { + bool cacheLastRow = false, cacheLastCol = false; + SFSIter fsiter; + SReadH readh; + SDFileSet *pSet; + STsdbMeta *pMeta = pRepo->tsdbMeta; + int tableNum = 0; + int maxTableIdx = 0; + int cacheLastRowTableNum = 0; + int cacheLastColTableNum = 0; + + bool need_free_last_row = CACHE_LAST_ROW(oldCfg) && !CACHE_LAST_ROW(&(pRepo->config)); + bool need_free_last_col = CACHE_LAST_NULL_COLUMN(oldCfg) && !CACHE_LAST_NULL_COLUMN(&(pRepo->config)); + + if (CACHE_LAST_ROW(&(pRepo->config)) || CACHE_LAST_NULL_COLUMN(&(pRepo->config))) { + tsdbInfo("tsdbCacheLastData cache last data since cacheLast option changed"); + cacheLastRow = !CACHE_LAST_ROW(oldCfg) && CACHE_LAST_ROW(&(pRepo->config)); + cacheLastCol = !CACHE_LAST_NULL_COLUMN(oldCfg) && CACHE_LAST_NULL_COLUMN(&(pRepo->config)); + } + + // calc max table idx and table num + for (int i = 1; i < pMeta->maxTables; i++) { + STable *pTable = pMeta->tables[i]; + if (pTable == NULL) continue; + tableNum += 1; + maxTableIdx = i; + if (cacheLastCol) { + pTable->restoreColumnNum = 0; + } + } + + // if close last option,need to free data + if (need_free_last_row || need_free_last_col) { + if (need_free_last_row) { + atomic_store_8(&pRepo->hasCachedLastRow, 0); + } + if (need_free_last_col) { + atomic_store_8(&pRepo->hasCachedLastColumn, 0); + } + tsdbInfo("free cache last data since cacheLast option changed"); + for (int i = 1; i < maxTableIdx; i++) { + STable *pTable = pMeta->tables[i]; + if (pTable == NULL) continue; + if (need_free_last_row) { + taosTZfree(pTable->lastRow); + pTable->lastRow = NULL; + pTable->lastKey = TSKEY_INITIAL_VAL; + } + if (need_free_last_col) { + tsdbFreeLastColumns(pTable); + } + } + } + + if (!cacheLastRow && !cacheLastCol) { + return 0; + } + + cacheLastRowTableNum = cacheLastRow ? tableNum : 0; + cacheLastColTableNum = cacheLastCol ? tableNum : 0; + + if (tsdbInitReadH(&readh, pRepo) < 0) { + return -1; + } + + tsdbFSIterInit(&fsiter, REPO_FS(pRepo), TSDB_FS_ITER_BACKWARD); + + while ((pSet = tsdbFSIterNext(&fsiter)) != NULL && (cacheLastRowTableNum > 0 || cacheLastColTableNum > 0)) { + if (tsdbSetAndOpenReadFSet(&readh, pSet) < 0) { + tsdbDestroyReadH(&readh); + return -1; + } + + if (tsdbLoadBlockIdx(&readh) < 0) { + tsdbDestroyReadH(&readh); + return -1; + } + + for (int i = 1; i <= maxTableIdx; i++) { + STable *pTable = pMeta->tables[i]; + if (pTable == NULL) continue; + + //tsdbInfo("tsdbRestoreInfo restore vgId:%d,table:%s", REPO_ID(pRepo), pTable->name->data); + + if (tsdbSetReadTable(&readh, pTable) < 0) { + tsdbDestroyReadH(&readh); + return -1; + } + + SBlockIdx *pIdx = readh.pBlkIdx; + + if (pIdx && cacheLastRowTableNum > 0 && pTable->lastRow == NULL) { + pTable->lastKey = pIdx->maxKey; + + if (tsdbRestoreLastRow(pRepo, pTable, &readh, pIdx) != 0) { + tsdbDestroyReadH(&readh); + return -1; + } + cacheLastRowTableNum -= 1; + } + + // restore NULL columns + if (pIdx && cacheLastColTableNum > 0 && !pTable->hasRestoreLastColumn) { + if (tsdbRestoreLastColumns(pRepo, pTable, &readh) != 0) { + tsdbDestroyReadH(&readh); + return -1; + } + if (pTable->hasRestoreLastColumn) { + cacheLastColTableNum -= 1; + } + } + } + } + + tsdbDestroyReadH(&readh); + + if (cacheLastRow) { + atomic_store_8(&pRepo->hasCachedLastRow, 1); + } + if (cacheLastCol) { + atomic_store_8(&pRepo->hasCachedLastColumn, 1); + } + + return 0; +} \ No newline at end of file diff --git a/src/tsdb/src/tsdbMemTable.c b/src/tsdb/src/tsdbMemTable.c index c6fcf55686850432a90d82e1f50c856f29c8eac0..79dbb8be5d8719d7a9df7fe71695d013dffdefc1 100644 --- a/src/tsdb/src/tsdbMemTable.c +++ b/src/tsdb/src/tsdbMemTable.c @@ -274,7 +274,7 @@ void *tsdbAllocBytes(STsdbRepo *pRepo, int bytes) { int tsdbAsyncCommit(STsdbRepo *pRepo) { tsem_wait(&(pRepo->readyToCommit)); - ASSERT(pRepo->imem == NULL); + //ASSERT(pRepo->imem == NULL); if (pRepo->mem == NULL) { tsem_post(&(pRepo->readyToCommit)); return 0; @@ -964,6 +964,49 @@ static void tsdbFreeRows(STsdbRepo *pRepo, void **rows, int rowCounter) { } } +static void updateTableLatestColumn(STsdbRepo *pRepo, STable *pTable, SDataRow row) { + tsdbDebug("vgId:%d updateTableLatestColumn, %s row version:%d", REPO_ID(pRepo), pTable->name->data, dataRowVersion(row)); + + STSchema* pSchema = tsdbGetTableLatestSchema(pTable); + if (tsdbUpdateLastColSchema(pTable, pSchema) < 0) { + return; + } + + pSchema = tsdbGetTableSchemaByVersion(pTable, dataRowVersion(row)); + if (pSchema == NULL) { + return; + } + + SDataCol *pLatestCols = pTable->lastCols; + + for (int16_t j = 0; j < schemaNCols(pSchema); j++) { + STColumn *pTCol = schemaColAt(pSchema, j); + // ignore not exist colId + int16_t idx = tsdbGetLastColumnsIndexByColId(pTable, pTCol->colId); + if (idx == -1) { + continue; + } + + void* value = tdGetRowDataOfCol(row, (int8_t)pTCol->type, TD_DATA_ROW_HEAD_SIZE + pSchema->columns[j].offset); + if (isNull(value, pTCol->type)) { + continue; + } + + SDataCol *pDataCol = &(pLatestCols[idx]); + if (pDataCol->pData == NULL) { + pDataCol->pData = malloc(pSchema->columns[j].bytes); + pDataCol->bytes = pSchema->columns[j].bytes; + } else if (pDataCol->bytes < pSchema->columns[j].bytes) { + pDataCol->pData = realloc(pDataCol->pData, pSchema->columns[j].bytes); + pDataCol->bytes = pSchema->columns[j].bytes; + } + + memcpy(pDataCol->pData, value, pDataCol->bytes); + //tsdbInfo("updateTableLatestColumn vgId:%d cache column %d for %d,%s", REPO_ID(pRepo), j, pDataCol->bytes, (char*)pDataCol->pData); + pDataCol->ts = dataRowKey(row); + } +} + static int tsdbUpdateTableLatestInfo(STsdbRepo *pRepo, STable *pTable, SDataRow row) { STsdbCfg *pCfg = &pRepo->config; @@ -977,7 +1020,7 @@ static int tsdbUpdateTableLatestInfo(STsdbRepo *pRepo, STable *pTable, SDataRow } if (tsdbGetTableLastKeyImpl(pTable) < dataRowKey(row)) { - if (pCfg->cacheLastRow || pTable->lastRow != NULL) { + if (CACHE_LAST_ROW(pCfg) || pTable->lastRow != NULL) { SDataRow nrow = pTable->lastRow; if (taosTSizeof(nrow) < dataRowLen(row)) { SDataRow orow = nrow; @@ -1002,7 +1045,10 @@ static int tsdbUpdateTableLatestInfo(STsdbRepo *pRepo, STable *pTable, SDataRow } else { pTable->lastKey = dataRowKey(row); } - } + if (CACHE_LAST_NULL_COLUMN(pCfg)) { + updateTableLatestColumn(pRepo, pTable, row); + } + } return 0; } diff --git a/src/tsdb/src/tsdbMeta.c b/src/tsdb/src/tsdbMeta.c index e6cbc4da9e6a3d2000e395e8ae8b9a52a01d1f6c..324a7c79c5b7dbfa69bbdf240301c3c710f90b59 100644 --- a/src/tsdb/src/tsdbMeta.c +++ b/src/tsdb/src/tsdbMeta.c @@ -589,6 +589,131 @@ void tsdbUnRefTable(STable *pTable) { } } +void tsdbFreeLastColumns(STable* pTable) { + if (pTable->lastCols == NULL) { + return; + } + + for (int i = 0; i < pTable->maxColNum; ++i) { + if (pTable->lastCols[i].bytes == 0) { + continue; + } + tfree(pTable->lastCols[i].pData); + pTable->lastCols[i].bytes = 0; + pTable->lastCols[i].pData = NULL; + } + tfree(pTable->lastCols); + pTable->lastCols = NULL; + pTable->maxColNum = 0; + pTable->lastColSVersion = -1; + pTable->restoreColumnNum = 0; +} + +int16_t tsdbGetLastColumnsIndexByColId(STable* pTable, int16_t colId) { + if (pTable->lastCols == NULL) { + return -1; + } + for (int16_t i = 0; i < pTable->maxColNum; ++i) { + if (pTable->lastCols[i].colId == colId) { + return i; + } + } + + return -1; +} + +int tsdbInitColIdCacheWithSchema(STable* pTable, STSchema* pSchema) { + ASSERT(pTable->lastCols == NULL); + + int16_t numOfColumn = pSchema->numOfCols; + + pTable->lastCols = (SDataCol*)malloc(numOfColumn * sizeof(SDataCol)); + if (pTable->lastCols == NULL) { + return -1; + } + + for (int16_t i = 0; i < numOfColumn; ++i) { + STColumn *pCol = schemaColAt(pSchema, i); + SDataCol* pDataCol = &(pTable->lastCols[i]); + pDataCol->bytes = 0; + pDataCol->pData = NULL; + pDataCol->colId = pCol->colId; + } + + pTable->lastColSVersion = schemaVersion(pSchema); + pTable->maxColNum = numOfColumn; + pTable->restoreColumnNum = 0; + return 0; +} + +STSchema* tsdbGetTableLatestSchema(STable *pTable) { + return tsdbGetTableSchemaByVersion(pTable, -1); +} + +int tsdbUpdateLastColSchema(STable *pTable, STSchema *pNewSchema) { + if (pTable->lastColSVersion == schemaVersion(pNewSchema)) { + return 0; + } + + tsdbInfo("tsdbUpdateLastColSchema:%s,%d->%d", pTable->name->data, pTable->lastColSVersion, schemaVersion(pNewSchema)); + + int16_t numOfCols = pNewSchema->numOfCols; + SDataCol *lastCols = (SDataCol*)malloc(numOfCols * sizeof(SDataCol)); + if (lastCols == NULL) { + return -1; + } + + TSDB_WLOCK_TABLE(pTable); + + for (int16_t i = 0; i < numOfCols; ++i) { + STColumn *pCol = schemaColAt(pNewSchema, i); + int16_t idx = tsdbGetLastColumnsIndexByColId(pTable, pCol->colId); + + SDataCol* pDataCol = &(lastCols[i]); + if (idx != -1) { + // move col data to new last column array + SDataCol* pOldDataCol = &(pTable->lastCols[idx]); + memcpy(pDataCol, pOldDataCol, sizeof(SDataCol)); + } else { + // init new colid data + pDataCol->colId = pCol->colId; + pDataCol->bytes = 0; + pDataCol->pData = NULL; + } + } + + SDataCol *oldLastCols = pTable->lastCols; + int16_t oldLastColNum = pTable->maxColNum; + + pTable->lastColSVersion = schemaVersion(pNewSchema); + pTable->lastCols = lastCols; + pTable->maxColNum = numOfCols; + + if (oldLastCols == NULL) { + TSDB_WUNLOCK_TABLE(pTable); + return 0; + } + + // free old schema last column datas + for (int16_t i = 0; i < oldLastColNum; ++i) { + SDataCol* pDataCol = &(oldLastCols[i]); + if (pDataCol->bytes == 0) { + continue; + } + int16_t idx = tsdbGetLastColumnsIndexByColId(pTable, pDataCol->colId); + if (idx != -1) { + continue; + } + + // free not exist column data + tfree(pDataCol->pData); + } + TSDB_WUNLOCK_TABLE(pTable); + tfree(oldLastCols); + + return 0; +} + void tsdbUpdateTableSchema(STsdbRepo *pRepo, STable *pTable, STSchema *pSchema, bool insertAct) { ASSERT(TABLE_TYPE(pTable) != TSDB_STREAM_TABLE && TABLE_TYPE(pTable) != TSDB_SUPER_TABLE); STsdbMeta *pMeta = pRepo->tsdbMeta; @@ -672,6 +797,10 @@ static STable *tsdbNewTable() { pTable->lastKey = TSKEY_INITIAL_VAL; + pTable->lastCols = NULL; + pTable->restoreColumnNum = 0; + pTable->maxColNum = 0; + pTable->lastColSVersion = -1; return pTable; } @@ -785,8 +914,10 @@ static void tsdbFreeTable(STable *pTable) { kvRowFree(pTable->tagVal); tSkipListDestroy(pTable->pIndex); - taosTZfree(pTable->lastRow); + taosTZfree(pTable->lastRow); tfree(pTable->sql); + + tsdbFreeLastColumns(pTable); free(pTable); } } diff --git a/src/tsdb/src/tsdbRead.c b/src/tsdb/src/tsdbRead.c index 9df25409de62f9b2579d43800fd6a5709bd41f1a..0a39dba021c92510e9e6d5f1420fe88792645cad 100644 --- a/src/tsdb/src/tsdbRead.c +++ b/src/tsdb/src/tsdbRead.c @@ -62,6 +62,7 @@ typedef struct SLoadCompBlockInfo { int32_t fileId; } SLoadCompBlockInfo; + typedef struct STableCheckInfo { STableId tableId; TSKEY lastKey; @@ -107,7 +108,7 @@ typedef struct STsdbQueryHandle { SArray* pTableCheckInfo; // SArray int32_t activeIndex; bool checkFiles; // check file stage - bool cachelastrow; // check if last row cached + int8_t cachelastrow; // check if last row cached bool loadExternalRow; // load time window external data rows bool currentLoadExternalRows; // current load external rows int32_t loadType; // block load type @@ -117,7 +118,6 @@ typedef struct STsdbQueryHandle { SFSIter fileIter; SReadH rhelper; STableBlockInfo* pDataBlockInfo; - SDataCols *pDataCols; // in order to hold current file data block int32_t allocSize; // allocated data block size SMemRef *pMemRef; @@ -138,6 +138,7 @@ typedef struct STableGroupSupporter { static STimeWindow updateLastrowForEachGroup(STableGroupInfo *groupList); static int32_t checkForCachedLastRow(STsdbQueryHandle* pQueryHandle, STableGroupInfo *groupList); +static int32_t checkForCachedLast(STsdbQueryHandle* pQueryHandle); static int32_t tsdbGetCachedLastRow(STable* pTable, SDataRow* pRes, TSKEY* lastKey); static void changeQueryHandleForInterpQuery(TsdbQueryHandleT pHandle); @@ -512,6 +513,8 @@ void tsdbResetQueryHandleForNewTable(TsdbQueryHandleT queryHandle, STsdbQueryCon pQueryHandle->next = doFreeColumnInfoData(pQueryHandle->next); } + + TsdbQueryHandleT tsdbQueryLastRow(STsdbRepo *tsdb, STsdbQueryCond *pCond, STableGroupInfo *groupList, uint64_t qId, SMemRef* pMemRef) { pCond->twindow = updateLastrowForEachGroup(groupList); @@ -528,10 +531,30 @@ TsdbQueryHandleT tsdbQueryLastRow(STsdbRepo *tsdb, STsdbQueryCond *pCond, STable } assert(pCond->order == TSDB_ORDER_ASC && pCond->twindow.skey <= pCond->twindow.ekey); - pQueryHandle->type = TSDB_QUERY_TYPE_LAST; + if (pQueryHandle->cachelastrow) { + pQueryHandle->type = TSDB_QUERY_TYPE_LAST; + } + + return pQueryHandle; +} + + +TsdbQueryHandleT tsdbQueryCacheLast(STsdbRepo *tsdb, STsdbQueryCond *pCond, STableGroupInfo *groupList, uint64_t qId, SMemRef* pMemRef) { + STsdbQueryHandle *pQueryHandle = (STsdbQueryHandle*) tsdbQueryTables(tsdb, pCond, groupList, qId, pMemRef); + int32_t code = checkForCachedLast(pQueryHandle); + if (code != TSDB_CODE_SUCCESS) { // set the numOfTables to be 0 + terrno = code; + return NULL; + } + + if (pQueryHandle->cachelastrow) { + pQueryHandle->type = TSDB_QUERY_TYPE_LAST; + } + return pQueryHandle; } + SArray* tsdbGetQueriedTableList(TsdbQueryHandleT *pHandle) { assert(pHandle != NULL); @@ -2460,6 +2483,159 @@ static bool loadCachedLastRow(STsdbQueryHandle* pQueryHandle) { return false; } + + +static bool loadCachedLast(STsdbQueryHandle* pQueryHandle) { + // the last row is cached in buffer, return it directly. + // here note that the pQueryHandle->window must be the TS_INITIALIZER + int32_t tgNumOfCols = (int32_t)QH_GET_NUM_OF_COLS(pQueryHandle); + size_t numOfTables = taosArrayGetSize(pQueryHandle->pTableCheckInfo); + int32_t numOfRows = 0; + assert(numOfTables > 0 && tgNumOfCols > 0); + SQueryFilePos* cur = &pQueryHandle->cur; + TSKEY priKey = TSKEY_INITIAL_VAL; + int32_t priIdx = -1; + SColumnInfoData* pColInfo = NULL; + + while (++pQueryHandle->activeIndex < numOfTables) { + STableCheckInfo* pCheckInfo = taosArrayGet(pQueryHandle->pTableCheckInfo, pQueryHandle->activeIndex); + STable* pTable = pCheckInfo->pTableObj; + char* pData = NULL; + + int32_t numOfCols = pTable->maxColNum; + + if (pTable->lastCols == NULL || pTable->maxColNum <= 0) { + tsdbWarn("no last cached for table, uid:%" PRIu64 ",tid:%d", pTable->tableId.uid, pTable->tableId.tid); + continue; + } + + int32_t i = 0, j = 0; + while(i < tgNumOfCols && j < numOfCols) { + pColInfo = taosArrayGet(pQueryHandle->pColumns, i); + if (pTable->lastCols[j].colId < pColInfo->info.colId) { + j++; + continue; + } else if (pTable->lastCols[j].colId > pColInfo->info.colId) { + i++; + continue; + } + + pData = (char*)pColInfo->pData + numOfRows * pColInfo->info.bytes; + + if (pTable->lastCols[j].bytes > 0) { + void* value = pTable->lastCols[j].pData; + switch (pColInfo->info.type) { + case TSDB_DATA_TYPE_BINARY: + case TSDB_DATA_TYPE_NCHAR: + memcpy(pData, value, varDataTLen(value)); + break; + case TSDB_DATA_TYPE_NULL: + case TSDB_DATA_TYPE_BOOL: + case TSDB_DATA_TYPE_TINYINT: + case TSDB_DATA_TYPE_UTINYINT: + *(uint8_t *)pData = *(uint8_t *)value; + break; + case TSDB_DATA_TYPE_SMALLINT: + case TSDB_DATA_TYPE_USMALLINT: + *(uint16_t *)pData = *(uint16_t *)value; + break; + case TSDB_DATA_TYPE_INT: + case TSDB_DATA_TYPE_UINT: + *(uint32_t *)pData = *(uint32_t *)value; + break; + case TSDB_DATA_TYPE_BIGINT: + case TSDB_DATA_TYPE_UBIGINT: + *(uint64_t *)pData = *(uint64_t *)value; + break; + case TSDB_DATA_TYPE_FLOAT: + SET_FLOAT_PTR(pData, value); + break; + case TSDB_DATA_TYPE_DOUBLE: + SET_DOUBLE_PTR(pData, value); + break; + case TSDB_DATA_TYPE_TIMESTAMP: + if (pColInfo->info.colId == PRIMARYKEY_TIMESTAMP_COL_INDEX) { + priKey = tdGetKey(*(TKEY *)value); + priIdx = i; + + i++; + j++; + continue; + } else { + *(TSKEY *)pData = *(TSKEY *)value; + } + break; + default: + memcpy(pData, value, pColInfo->info.bytes); + } + + for (int32_t n = 0; n < tgNumOfCols; ++n) { + if (n == i) { + continue; + } + + pColInfo = taosArrayGet(pQueryHandle->pColumns, n); + pData = (char*)pColInfo->pData + numOfRows * pColInfo->info.bytes;; + + if (pColInfo->info.colId == PRIMARYKEY_TIMESTAMP_COL_INDEX) { + *(TSKEY *)pData = pTable->lastCols[j].ts; + continue; + } + + if (pColInfo->info.type == TSDB_DATA_TYPE_BINARY || pColInfo->info.type == TSDB_DATA_TYPE_NCHAR) { + setVardataNull(pData, pColInfo->info.type); + } else { + setNull(pData, pColInfo->info.type, pColInfo->info.bytes); + } + } + + numOfRows++; + assert(numOfRows < pQueryHandle->outputCapacity); + } + + i++; + j++; + } + + // leave the real ts column as the last row, because last function only (not stable) use the last row as res + if (priKey != TSKEY_INITIAL_VAL) { + pColInfo = taosArrayGet(pQueryHandle->pColumns, priIdx); + pData = (char*)pColInfo->pData + numOfRows * pColInfo->info.bytes; + + *(TSKEY *)pData = priKey; + + for (int32_t n = 0; n < tgNumOfCols; ++n) { + if (n == priIdx) { + continue; + } + + pColInfo = taosArrayGet(pQueryHandle->pColumns, n); + pData = (char*)pColInfo->pData + numOfRows * pColInfo->info.bytes;; + + assert (pColInfo->info.colId != PRIMARYKEY_TIMESTAMP_COL_INDEX); + + if (pColInfo->info.type == TSDB_DATA_TYPE_BINARY || pColInfo->info.type == TSDB_DATA_TYPE_NCHAR) { + setVardataNull(pData, pColInfo->info.type); + } else { + setNull(pData, pColInfo->info.type, pColInfo->info.bytes); + } + } + + numOfRows++; + } + + if (numOfRows > 0) { + cur->rows = numOfRows; + cur->mixBlock = true; + + return true; + } + } + + return false; +} + + static bool loadDataBlockFromTableSeq(STsdbQueryHandle* pQueryHandle) { size_t numOfTables = taosArrayGetSize(pQueryHandle->pTableCheckInfo); assert(numOfTables > 0); @@ -2496,8 +2672,12 @@ bool tsdbNextDataBlock(TsdbQueryHandleT pHandle) { int64_t stime = taosGetTimestampUs(); int64_t elapsedTime = stime; - if (pQueryHandle->type == TSDB_QUERY_TYPE_LAST && pQueryHandle->cachelastrow) { - return loadCachedLastRow(pQueryHandle); + if (pQueryHandle->type == TSDB_QUERY_TYPE_LAST) { + if (pQueryHandle->cachelastrow == 1) { + return loadCachedLastRow(pQueryHandle); + } else if (pQueryHandle->cachelastrow == 2) { + return loadCachedLast(pQueryHandle); + } } if (pQueryHandle->loadType == BLOCK_LOAD_TABLE_SEQ_ORDER) { @@ -2695,6 +2875,10 @@ int32_t tsdbGetCachedLastRow(STable* pTable, SDataRow* pRes, TSKEY* lastKey) { return TSDB_CODE_SUCCESS; } +bool isTsdbCacheLastRow(TsdbQueryHandleT* pQueryHandle) { + return ((STsdbQueryHandle *)pQueryHandle)->cachelastrow > 0; +} + int32_t checkForCachedLastRow(STsdbQueryHandle* pQueryHandle, STableGroupInfo *groupList) { assert(pQueryHandle != NULL && groupList != NULL); @@ -2706,11 +2890,15 @@ int32_t checkForCachedLastRow(STsdbQueryHandle* pQueryHandle, STableGroupInfo *g STableKeyInfo* pInfo = (STableKeyInfo*)taosArrayGet(group, 0); - int32_t code = tsdbGetCachedLastRow(pInfo->pTable, &pRow, &key); - if (code != TSDB_CODE_SUCCESS) { - pQueryHandle->cachelastrow = false; - } else { - pQueryHandle->cachelastrow = (pRow != NULL); + int32_t code = 0; + + if (((STable*)pInfo->pTable)->lastRow) { + code = tsdbGetCachedLastRow(pInfo->pTable, &pRow, &key); + if (code != TSDB_CODE_SUCCESS) { + pQueryHandle->cachelastrow = 0; + } else { + pQueryHandle->cachelastrow = 1; + } } // update the tsdb query time range @@ -2724,6 +2912,26 @@ int32_t checkForCachedLastRow(STsdbQueryHandle* pQueryHandle, STableGroupInfo *g return code; } +int32_t checkForCachedLast(STsdbQueryHandle* pQueryHandle) { + assert(pQueryHandle != NULL); + + int32_t code = 0; + + if (pQueryHandle->pTsdb && atomic_load_8(&pQueryHandle->pTsdb->hasCachedLastColumn)){ + pQueryHandle->cachelastrow = 2; + } + + // update the tsdb query time range + if (pQueryHandle->cachelastrow) { + pQueryHandle->window = TSWINDOW_INITIALIZER; + pQueryHandle->checkFiles = false; + pQueryHandle->activeIndex = -1; // start from -1 + } + + return code; +} + + STimeWindow updateLastrowForEachGroup(STableGroupInfo *groupList) { STimeWindow window = {INT64_MAX, INT64_MIN}; diff --git a/src/util/inc/hash.h b/src/util/inc/hash.h index b6b49693f60b15a51eaa35c95555d0f35d36f017..cd4850e47e855cde4c1a7c281edb0afc23404370 100644 --- a/src/util/inc/hash.h +++ b/src/util/inc/hash.h @@ -148,6 +148,7 @@ int32_t taosHashGetMaxOverflowLinkLength(const SHashObj *pHashObj); size_t taosHashGetMemSize(const SHashObj *pHashObj); void *taosHashIterate(SHashObj *pHashObj, void *p); + void taosHashCancelIterate(SHashObj *pHashObj, void *p); #ifdef __cplusplus diff --git a/src/util/inc/tarray.h b/src/util/inc/tarray.h index f2e268c2d4dfe210dfbfd9b94ee74a4f87848361..bc25776caaae50e320b036f662ed4780fd36d90f 100644 --- a/src/util/inc/tarray.h +++ b/src/util/inc/tarray.h @@ -50,7 +50,15 @@ void* taosArrayInit(size_t size, size_t elemSize); * @param nEles * @return */ -void *taosArrayPushBatch(SArray *pArray, const void *pData, int nEles); +void *taosArrayAddBatch(SArray *pArray, const void *pData, int nEles); + +/** + * add all element from the source array list into the destination + * @param pArray + * @param pInput + * @return + */ +void* taosArrayAddAll(SArray* pArray, const SArray* pInput); /** * @@ -59,7 +67,7 @@ void *taosArrayPushBatch(SArray *pArray, const void *pData, int nEles); * @return */ static FORCE_INLINE void* taosArrayPush(SArray* pArray, const void* pData) { - return taosArrayPushBatch(pArray, pData, 1); + return taosArrayAddBatch(pArray, pData, 1); } /** diff --git a/src/util/inc/ttoken.h b/src/util/inc/ttoken.h index d5e45e60a148726d0e71c38fcd8262d1d67a380a..3bf030a9ebf96a27f16b27eb6d55505212274ba6 100644 --- a/src/util/inc/ttoken.h +++ b/src/util/inc/ttoken.h @@ -37,8 +37,6 @@ typedef struct SStrToken { char *z; } SStrToken; -extern const char escapeChar[]; - /** * check if it is a number or not * @param pToken @@ -47,8 +45,6 @@ extern const char escapeChar[]; #define isNumber(tk) \ ((tk)->type == TK_INTEGER || (tk)->type == TK_FLOAT || (tk)->type == TK_HEX || (tk)->type == TK_BIN) -#define GET_ESCAPE_CHAR(c) (escapeChar[(uint8_t)(c)]) - /** * tokenizer for sql string * @param z diff --git a/src/util/src/tarray.c b/src/util/src/tarray.c index f4587b33e75c556d6dbc03e1577436afb728cba7..45cb878668e2058c92c1ba2821a88886b87e2044 100644 --- a/src/util/src/tarray.c +++ b/src/util/src/tarray.c @@ -56,7 +56,7 @@ static int32_t taosArrayResize(SArray* pArray) { return 0; } -void* taosArrayPushBatch(SArray* pArray, const void* pData, int nEles) { +void* taosArrayAddBatch(SArray* pArray, const void* pData, int nEles) { if (pArray == NULL || pData == NULL) { return NULL; } @@ -82,6 +82,10 @@ void* taosArrayPushBatch(SArray* pArray, const void* pData, int nEles) { return dst; } +void* taosArrayAddAll(SArray* pArray, const SArray* pInput) { + return taosArrayAddBatch(pArray, pInput->pData, (int32_t) taosArrayGetSize(pInput)); +} + void* taosArrayPop(SArray* pArray) { assert( pArray != NULL ); diff --git a/src/util/src/tcache.c b/src/util/src/tcache.c index 57d45cc8c00dab5b760f8e915b967e37217f9b64..569f9b01bda6d3a8cf5de6ba4b089e7bab49ca9f 100644 --- a/src/util/src/tcache.c +++ b/src/util/src/tcache.c @@ -613,7 +613,7 @@ void doCleanupDataCache(SCacheObj *pCacheObj) { // todo memory leak if there are object with refcount greater than 0 in hash table? taosHashCleanup(pCacheObj->pHashTable); - taosTrashcanEmpty(pCacheObj, true); + taosTrashcanEmpty(pCacheObj, false); __cache_lock_destroy(pCacheObj); diff --git a/src/util/src/terror.c b/src/util/src/terror.c index d88393777ccd1de829e8f4e53f9f4563bc8f7235..887b231ec6161ee99e5c64330595345a01ab34c0 100644 --- a/src/util/src/terror.c +++ b/src/util/src/terror.c @@ -83,7 +83,7 @@ TAOS_DEFINE_ERROR(TSDB_CODE_REF_ALREADY_EXIST, "Ref is already there" TAOS_DEFINE_ERROR(TSDB_CODE_REF_NOT_EXIST, "Ref is not there") //client -TAOS_DEFINE_ERROR(TSDB_CODE_TSC_INVALID_SQL, "Invalid SQL statement") +TAOS_DEFINE_ERROR(TSDB_CODE_TSC_INVALID_OPERATION, "Invalid operation") TAOS_DEFINE_ERROR(TSDB_CODE_TSC_INVALID_QHANDLE, "Invalid qhandle") TAOS_DEFINE_ERROR(TSDB_CODE_TSC_INVALID_TIME_STAMP, "Invalid combination of client/service time") TAOS_DEFINE_ERROR(TSDB_CODE_TSC_INVALID_VALUE, "Invalid value in client") diff --git a/src/util/src/ttokenizer.c b/src/util/src/ttokenizer.c index 93d4570ea8de5c41edcedce7fab3183d5f54f7e8..138a79511c36eaab4087bf51007717944656b268 100644 --- a/src/util/src/ttokenizer.c +++ b/src/util/src/ttokenizer.c @@ -232,18 +232,6 @@ static const char isIdChar[] = { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, /* 7x */ }; -const char escapeChar[] = { - /* x0 x1 x2 x3 x4 x5 x6 x7 x8 x9 xA xB xC xD xE xF */ - 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0A, 0x0B, 0x0C, 0x0D, 0x0E, 0x0F, /* 0x */ - 0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, 0x18, 0x19, 0x1A, 0x1B, 0x1C, 0x1D, 0x1E, 0x1F, /* 1x */ - 0x20, 0x21, 0x22, 0x23, 0x24, 0x25, 0x26, 0x27, 0x28, 0x29, 0x2A, 0x2B, 0x2C, 0x2D, 0x2E, 0x2F, /* 2x */ - 0x30, 0x31, 0x32, 0x33, 0x34, 0x35, 0x36, 0x37, 0x38, 0x39, 0x3A, 0x3B, 0x3C, 0x3D, 0x3E, 0x3F, /* 3x */ - 0x40, 0x41, 0x42, 0x43, 0x44, 0x45, 0x46, 0x47, 0x48, 0x49, 0x4A, 0x4B, 0x4C, 0x4D, 0x4E, 0x4F,/* 4x */ - 0x50, 0x51, 0x52, 0x53, 0x54, 0x55, 0x56, 0x57, 0x58, 0x59, 0x5A, 0x5B, 0x5C, 0x5D, 0x5E, 0x5F,/* 5x */ - 0x60, 0x07, 0x08, 0x63, 0x64, 0x65, 0x0C, 0x67, 0x68, 0x69, 0x6A, 0x6B, 0x6C, 0x6D, 0x0A, 0x6F,/* 6x */ - 0x70, 0x71, 0x0D, 0x73, 0x09, 0x75, 0x0B, 0x77, 0x78, 0x79, 0x7A, 0x7B, 0x7C, 0x7D, 0x7E, 0x7F,/* 7x */ -}; - static void* keywordHashTable = NULL; static void doInitKeywordsTable(void) { @@ -593,7 +581,6 @@ SStrToken tscReplaceStrToken(char **str, SStrToken *token, const char* newToken) return ntoken; } - SStrToken tStrGetToken(char* str, int32_t* i, bool isPrevOptr) { SStrToken t0 = {0}; diff --git a/src/vnode/src/vnodeMain.c b/src/vnode/src/vnodeMain.c index f133101ccad3c7eaa7bd206f3355ff5ff8b82298..fdf215a966582254978daf5997234efa9d2cfb8b 100644 --- a/src/vnode/src/vnodeMain.c +++ b/src/vnode/src/vnodeMain.c @@ -454,7 +454,11 @@ void vnodeDestroy(SVnodeObj *pVnode) { } if (pVnode->tsdb) { - code = tsdbCloseRepo(pVnode->tsdb, 1); + // the deleted vnode does not need to commit, so as to speed up the deletion + int toCommit = 1; + if (pVnode->dropped) toCommit = 0; + + code = tsdbCloseRepo(pVnode->tsdb, toCommit); pVnode->tsdb = NULL; } diff --git a/src/vnode/src/vnodeRead.c b/src/vnode/src/vnodeRead.c index 2831090267004ba92cd08b91b0f968b144025256..3964975e355db3ced1c86e28611ef202dd24d9e3 100644 --- a/src/vnode/src/vnodeRead.c +++ b/src/vnode/src/vnodeRead.c @@ -357,7 +357,7 @@ static int32_t vnodeProcessFetchMsg(SVnodeObj *pVnode, SVReadMsg *pRead) { // kill current query and free corresponding resources. if (pRetrieve->free == 1) { - vWarn("vgId:%d, QInfo:%"PRIu64 "-%p, retrieve msg received to kill query and free qhandle", pVnode->vgId, pRetrieve->qId, *handle); + vWarn("vgId:%d, QInfo:%"PRIx64 "-%p, retrieve msg received to kill query and free qhandle", pVnode->vgId, pRetrieve->qId, *handle); qKillQuery(*handle); qReleaseQInfo(pVnode->qMgmt, (void **)&handle, true); diff --git a/src/vnode/src/vnodeSync.c b/src/vnode/src/vnodeSync.c index e5a1964915646c7d6da106b003abbc9909ed71db..4197428fec6b5d24e7791b2a5f8cb7df229cbca5 100644 --- a/src/vnode/src/vnodeSync.c +++ b/src/vnode/src/vnodeSync.c @@ -126,11 +126,16 @@ void vnodeStopSyncFile(int32_t vgId, uint64_t fversion) { } void vnodeConfirmForard(int32_t vgId, void *wparam, int32_t code) { - void *pVnode = vnodeAcquire(vgId); + SVnodeObj *pVnode = vnodeAcquire(vgId); if (pVnode == NULL) { vError("vgId:%d, vnode not found while confirm forward", vgId); } + if (code == TSDB_CODE_SYN_CONFIRM_EXPIRED && pVnode->status == TAOS_VN_STATUS_CLOSING) { + vDebug("vgId:%d, db:%s, vnode is closing while confirm forward", vgId, pVnode->db); + code = TSDB_CODE_VND_IS_CLOSING; + } + dnodeSendRpcVWriteRsp(pVnode, wparam, code); vnodeRelease(pVnode); } diff --git a/tests/examples/c/CMakeLists.txt b/tests/examples/c/CMakeLists.txt index 954fe468b1b8fc88ce93fa2474e2f69a33415e6f..7f941b8c87a2c256a82aa37f9e937a41cd9a4c77 100644 --- a/tests/examples/c/CMakeLists.txt +++ b/tests/examples/c/CMakeLists.txt @@ -5,6 +5,8 @@ IF (TD_LINUX) AUX_SOURCE_DIRECTORY(. SRC) ADD_EXECUTABLE(demo apitest.c) TARGET_LINK_LIBRARIES(demo taos_static trpc tutil pthread ) + ADD_EXECUTABLE(subscribe subscribe.c) + TARGET_LINK_LIBRARIES(subscribe taos_static trpc tutil pthread ) ADD_EXECUTABLE(epoll epoll.c) TARGET_LINK_LIBRARIES(epoll taos_static trpc tutil pthread ) ENDIF () diff --git a/tests/examples/c/apitest.c b/tests/examples/c/apitest.c index f20c0321c455da9c430aa3a4eb1d32af7d71da8b..0ca92eaf1d018bc8b370706752cf0d139a9bfaf8 100644 --- a/tests/examples/c/apitest.c +++ b/tests/examples/c/apitest.c @@ -7,7 +7,6 @@ #include #include - static void prepare_data(TAOS* taos) { TAOS_RES *result; result = taos_query(taos, "drop database if exists test;"); @@ -69,7 +68,6 @@ static void prepare_data(TAOS* taos) { usleep(1000000); } - static int print_result(TAOS_RES* res, int blockFetch) { TAOS_ROW row = NULL; int num_fields = taos_num_fields(res); @@ -99,7 +97,6 @@ static int print_result(TAOS_RES* res, int blockFetch) { return nRows; } - static void check_row_count(int line, TAOS_RES* res, int expected) { int actual = print_result(res, expected % 2); if (actual != expected) { @@ -109,7 +106,6 @@ static void check_row_count(int line, TAOS_RES* res, int expected) { } } - static void verify_query(TAOS* taos) { prepare_data(taos); @@ -153,7 +149,6 @@ static void verify_query(TAOS* taos) { taos_free_result(res); } - void subscribe_callback(TAOS_SUB* tsub, TAOS_RES *res, void* param, int code) { int rows = print_result(res, *(int*)param); printf("%d rows consumed in subscribe_callback\n", rows); @@ -235,10 +230,10 @@ static void verify_subscribe(TAOS* taos) { taos_unsubscribe(tsub, 0); } - void verify_prepare(TAOS* taos) { TAOS_RES* result = taos_query(taos, "drop database if exists test;"); taos_free_result(result); + usleep(100000); result = taos_query(taos, "create database test;"); @@ -248,6 +243,7 @@ void verify_prepare(TAOS* taos) { taos_free_result(result); return; } + taos_free_result(result); usleep(100000); @@ -369,6 +365,7 @@ void verify_prepare(TAOS* taos) { taos_stmt_add_batch(stmt); } if (taos_stmt_execute(stmt) != 0) { + taos_stmt_close(stmt); printf("\033[31mfailed to execute insert statement.\033[0m\n"); return; } @@ -381,6 +378,7 @@ void verify_prepare(TAOS* taos) { v.v2 = 15; taos_stmt_bind_param(stmt, params + 2); if (taos_stmt_execute(stmt) != 0) { + taos_stmt_close(stmt); printf("\033[31mfailed to execute select statement.\033[0m\n"); return; } diff --git a/tests/mas/Jenkinsfile b/tests/mas/Jenkinsfile new file mode 100644 index 0000000000000000000000000000000000000000..b2a1a5e1167844e777909cc9688186d85b90a707 --- /dev/null +++ b/tests/mas/Jenkinsfile @@ -0,0 +1,309 @@ +def pre_test(){ + + sh ''' + sudo rmtaos||echo 'no taosd installed' + ''' + sh ''' + cd ${WKC} + git reset --hard + git checkout $BRANCH_NAME + git pull + git submodule update + cd ${WK} + git reset --hard + git checkout $BRANCH_NAME + git pull + export TZ=Asia/Harbin + date + rm -rf ${WK}/debug + mkdir debug + cd debug + cmake .. > /dev/null + make > /dev/null + make install > /dev/null + pip3 install ${WKC}/src/connector/python/linux/python3/ + ''' + return 1 +} +pipeline { + agent none + environment{ + + WK = '/var/lib/jenkins/workspace/TDinternal' + WKC= '/var/lib/jenkins/workspace/TDinternal/community' + } + + stages { + stage('Parallel test stage') { + parallel { + stage('pytest') { + agent{label 'slam1'} + steps { + pre_test() + sh ''' + cd ${WKC}/tests + find pytest -name '*'sql|xargs rm -rf + ./test-all.sh pytest + date''' + } + } + stage('test_b1') { + agent{label 'slam2'} + steps { + pre_test() + + sh ''' + cd ${WKC}/tests + ./test-all.sh b1 + date''' + + + } + } + + stage('test_crash_gen') { + agent{label "slam3"} + steps { + pre_test() + sh ''' + cd ${WKC}/tests/pytest + ''' + catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE') { + sh ''' + cd ${WKC}/tests/pytest + ./crash_gen.sh -a -p -t 4 -s 2000 + ''' + } + catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE') { + sh ''' + cd ${WKC}/tests/pytest + rm -rf /var/lib/taos/* + rm -rf /var/log/taos/* + ./handle_crash_gen_val_log.sh + ''' + } + catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE') { + sh ''' + cd ${WKC}/tests/pytest + rm -rf /var/lib/taos/* + rm -rf /var/log/taos/* + ./handle_taosd_val_log.sh + ''' + } + + sh''' + systemctl start taosd + sleep 10 + ''' + catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE') { + sh ''' + cd ${WKC}/tests/gotest + bash batchtest.sh + ''' + } + catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE') { + sh ''' + cd ${WKC}/tests/examples/python/PYTHONConnectorChecker + python3 PythonChecker.py + ''' + } + catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE') { + sh ''' + cd ${WKC}/tests/examples/JDBC/JDBCDemo/ + mvn clean package assembly:single -DskipTests >/dev/null + java -jar target/JDBCDemo-SNAPSHOT-jar-with-dependencies.jar -host 127.0.0.1 + ''' + } + catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE') { + sh ''' + cd ${WKC}/src/connector/jdbc + mvn clean package -Dmaven.test.skip=true >/dev/null + cd ${WKC}/tests/examples/JDBC/JDBCDemo/ + java --class-path=../../../../src/connector/jdbc/target:$JAVA_HOME/jre/lib/ext -jar target/JDBCDemo-SNAPSHOT-jar-with-dependencies.jar -host 127.0.0.1 + ''' + } + catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE') { + sh ''' + cp -rf ${WKC}/tests/examples/nodejs ${JENKINS_HOME}/workspace/ + cd ${JENKINS_HOME}/workspace/nodejs + node nodejsChecker.js host=localhost + ''' + } + catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE') { + sh ''' + cd ${JENKINS_HOME}/workspace/C#NET/src/CheckC# + dotnet run + ''' + } + sh ''' + systemctl stop taosd + cd ${WKC}/tests + ./test-all.sh b2 + date + ''' + sh ''' + cd ${WKC}/tests + ./test-all.sh full unit + date''' + } + } + + stage('test_valgrind') { + agent{label "slam4"} + + steps { + pre_test() + catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE') { + sh ''' + cd ${WKC}/tests/pytest + nohup taosd >/dev/null & + sleep 10 + python3 concurrent_inquiry.py -c 1 + + ''' + } + sh ''' + cd ${WKC}/tests + ./test-all.sh full jdbc + date''' + sh ''' + cd ${WKC}/tests/pytest + ./valgrind-test.sh 2>&1 > mem-error-out.log + ./handle_val_log.sh + + date + cd ${WKC}/tests + ./test-all.sh b3 + date''' + sh ''' + date + cd ${WKC}/tests + ./test-all.sh full example + date''' + } + } + + stage('arm64_build'){ + agent{label 'arm64'} + steps{ + sh ''' + cd ${WK} + git fetch + git checkout develop + git pull + cd ${WKC} + git fetch + git checkout develop + git pull + git submodule update + cd ${WKC}/packaging + ./release.sh -v cluster -c aarch64 -n 2.0.0.0 -m 2.0.0.0 + + ''' + } + } + stage('arm32_build'){ + agent{label 'arm32'} + steps{ + catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE') { + sh ''' + cd ${WK} + git fetch + git checkout develop + git pull + cd ${WKC} + git fetch + git checkout develop + git pull + git submodule update + cd ${WKC}/packaging + ./release.sh -v cluster -c aarch32 -n 2.0.0.0 -m 2.0.0.0 + + ''' + } + + } + } + } + } + + } + post { + success { + emailext ( + subject: "SUCCESSFUL: Job '${env.JOB_NAME} [${env.BUILD_NUMBER}]'", + body: ''' + + + + + + + + + + + + +

+ 构建信息 +
+
    +
    +
  • 构建名称>>分支：${PROJECT_NAME}
    • +
  • 构建结果： Successful
    • +
  • 构建编号：${BUILD_NUMBER}
    • +
  • 触发用户：${CAUSE}
    • +
  • 变更概要：${CHANGES}
    • +
  • 构建地址：${BUILD_URL}
    • +
  • 构建日志：${BUILD_URL}console
    • +
  • 变更集：${JELLY_SCRIPT}
    • +
    +
+
+ + ''', + to: "yqliu@taosdata.com,pxiao@taosdata.com", + from: "support@taosdata.com" + ) + } + failure { + emailext ( + subject: "FAILED: Job '${env.JOB_NAME} [${env.BUILD_NUMBER}]'", + body: ''' + + + + + + + + + + + + +

+ 构建信息 +
+
    +
    +
  • 构建名称>>分支：${PROJECT_NAME}
    • +
  • 构建结果： Successful
    • +
  • 构建编号：${BUILD_NUMBER}
    • +
  • 触发用户：${CAUSE}
    • +
  • 变更概要：${CHANGES}
    • +
  • 构建地址：${BUILD_URL}
    • +
  • 构建日志：${BUILD_URL}console
    • +
  • 变更集：${JELLY_SCRIPT}
    • +
    +
+
+ + ''', + to: "yqliu@taosdata.com,pxiao@taosdata.com", + from: "support@taosdata.com" + ) + } + } +} \ No newline at end of file diff --git a/tests/perftest-scripts/perftest-query.sh b/tests/perftest-scripts/perftest-query.sh index c6d4687ed700fc266e4911019ccf7882c086bc0f..bf62f401bffa1cd4d3189d551d02c8a3e227be4b 100755 --- a/tests/perftest-scripts/perftest-query.sh +++ b/tests/perftest-scripts/perftest-query.sh @@ -64,18 +64,25 @@ function runQueryPerfTest { [ -f $PERFORMANCE_TEST_REPORT ] && rm $PERFORMANCE_TEST_REPORT nohup $WORK_DIR/TDengine/debug/build/bin/taosd -c /etc/taosperf/ > /dev/null 2>&1 & echoInfo "Wait TDengine to start" - sleep 300 + sleep 60 echoInfo "Run Performance Test" cd $WORK_DIR/TDengine/tests/pytest python3 query/queryPerformance.py -c $LOCAL_COMMIT | tee -a $PERFORMANCE_TEST_REPORT + mkdir -p /var/lib/perf/ + mkdir -p /var/log/perf/ + rm -rf /var/lib/perf/* + rm -rf /var/log/perf/* + nohup $WORK_DIR/TDengine/debug/build/bin/taosd -c /etc/perf/ > /dev/null 2>&1 & + echoInfo "Wait TDengine to start" + sleep 10 + echoInfo "Run Performance Test" + cd $WORK_DIR/TDengine/tests/pytest + python3 insert/insertFromCSVPerformance.py -c $LOCAL_COMMIT | tee -a $PERFORMANCE_TEST_REPORT python3 tools/taosdemoPerformance.py -c $LOCAL_COMMIT | tee -a $PERFORMANCE_TEST_REPORT - - #python3 perfbenchmark/joinPerformance.py | tee -a $PERFORMANCE_TEST_REPORT - } diff --git a/tests/pytest/crash_gen/service_manager.py b/tests/pytest/crash_gen/service_manager.py index 95507f0142036f36f9a5249a48800bd481513844..1cd65c1dde4e9c2027a82730cf4fc12290048bbe 100644 --- a/tests/pytest/crash_gen/service_manager.py +++ b/tests/pytest/crash_gen/service_manager.py @@ -22,7 +22,7 @@ from queue import Queue, Empty from .shared.config import Config from .shared.db import DbTarget, DbConn from .shared.misc import Logging, Helper, CrashGenError, Status, Progress, Dice -from .shared.types import DirPath +from .shared.types import DirPath, IpcStream # from crash_gen.misc import CrashGenError, Dice, Helper, Logging, Progress, Status # from crash_gen.db import DbConn, DbTarget @@ -177,13 +177,12 @@ quorum 2 return "127.0.0.1" def getServiceCmdLine(self): # to start the instance - cmdLine = [] if Config.getConfig().track_memory_leaks: Logging.info("Invoking VALGRIND on service...") - cmdLine = ['valgrind', '--leak-check=yes'] - # TODO: move "exec -c" into Popen(), we can both "use shell" and NOT fork so ask to lose kill control - cmdLine += ["exec " + self.getExecFile(), '-c', self.getCfgDir()] # used in subproce.Popen() - return cmdLine + return ['exec /usr/bin/valgrind', '--leak-check=yes', self.getExecFile(), '-c', self.getCfgDir()] + else: + # TODO: move "exec -c" into Popen(), we can both "use shell" and NOT fork so ask to lose kill control + return ["exec " + self.getExecFile(), '-c', self.getCfgDir()] # used in subproce.Popen() def _getDnodes(self, dbc): dbc.query("show dnodes") @@ -281,16 +280,16 @@ class TdeSubProcess: return '[TdeSubProc: pid = {}, status = {}]'.format( self.getPid(), self.getStatus() ) - def getStdOut(self) -> BinaryIO : + def getIpcStdOut(self) -> IpcStream : if self._popen.universal_newlines : # alias of text_mode raise CrashGenError("We need binary mode for STDOUT IPC") # Logging.info("Type of stdout is: {}".format(type(self._popen.stdout))) - return typing.cast(BinaryIO, self._popen.stdout) + return typing.cast(IpcStream, self._popen.stdout) - def getStdErr(self) -> BinaryIO : + def getIpcStdErr(self) -> IpcStream : if self._popen.universal_newlines : # alias of text_mode raise CrashGenError("We need binary mode for STDERR IPC") - return typing.cast(BinaryIO, self._popen.stderr) + return typing.cast(IpcStream, self._popen.stderr) # Now it's always running, since we matched the life cycle # def isRunning(self): @@ -301,11 +300,6 @@ class TdeSubProcess: def _start(self, cmdLine) -> Popen : ON_POSIX = 'posix' in sys.builtin_module_names - - # Sanity check - # if self.subProcess: # already there - # raise RuntimeError("Corrupt process state") - # Prepare environment variables for coverage information # Ref: https://stackoverflow.com/questions/2231227/python-subprocess-popen-with-a-modified-environment @@ -314,9 +308,8 @@ class TdeSubProcess: # print(myEnv) # print("Starting TDengine with env: ", myEnv.items()) - # print("Starting TDengine via Shell: {}".format(cmdLineStr)) + print("Starting TDengine: {}".format(cmdLine)) - # useShell = True # Needed to pass environments into it return Popen( ' '.join(cmdLine), # ' '.join(cmdLine) if useShell else cmdLine, shell=True, # Always use shell, since we need to pass ENV vars @@ -732,19 +725,19 @@ class ServiceManagerThread: self._ipcQueue = Queue() # type: Queue self._thread = threading.Thread( # First thread captures server OUTPUT target=self.svcOutputReader, - args=(subProc.getStdOut(), self._ipcQueue, logDir)) + args=(subProc.getIpcStdOut(), self._ipcQueue, logDir)) self._thread.daemon = True # thread dies with the program self._thread.start() time.sleep(0.01) if not self._thread.is_alive(): # What happened? - Logging.info("Failed to started process to monitor STDOUT") + Logging.info("Failed to start process to monitor STDOUT") self.stop() raise CrashGenError("Failed to start thread to monitor STDOUT") Logging.info("Successfully started process to monitor STDOUT") self._thread2 = threading.Thread( # 2nd thread captures server ERRORs target=self.svcErrorReader, - args=(subProc.getStdErr(), self._ipcQueue, logDir)) + args=(subProc.getIpcStdErr(), self._ipcQueue, logDir)) self._thread2.daemon = True # thread dies with the program self._thread2.start() time.sleep(0.01) @@ -887,14 +880,19 @@ class ServiceManagerThread: print("\nNon-UTF8 server output: {}\n".format(bChunk.decode('cp437'))) return None - def _textChunkGenerator(self, streamIn: BinaryIO, logDir: str, logFile: str + def _textChunkGenerator(self, streamIn: IpcStream, logDir: str, logFile: str ) -> Generator[TextChunk, None, None]: ''' - Take an input stream with binary data, produced a generator of decoded - "text chunks", and also save the original binary data in a log file. + Take an input stream with binary data (likely from Popen), produced a generator of decoded + "text chunks". + + Side effect: it also save the original binary data in a log file. ''' os.makedirs(logDir, exist_ok=True) logF = open(os.path.join(logDir, logFile), 'wb') + if logF is None: + Logging.error("Failed to open log file (binary write): {}/{}".format(logDir, logFile)) + return for bChunk in iter(streamIn.readline, b''): logF.write(bChunk) # Write to log file immediately tChunk = self._decodeBinaryChunk(bChunk) # decode @@ -902,14 +900,14 @@ class ServiceManagerThread: yield tChunk # TODO: split into actual text lines # At the end... - streamIn.close() # Close the stream - logF.close() # Close the output file + streamIn.close() # Close the incoming stream + logF.close() # Close the log file - def svcOutputReader(self, stdOut: BinaryIO, queue, logDir: str): + def svcOutputReader(self, ipcStdOut: IpcStream, queue, logDir: str): ''' The infinite routine that processes the STDOUT stream for the sub process being managed. - :param stdOut: the IO stream object used to fetch the data from + :param ipcStdOut: the IO stream object used to fetch the data from :param queue: the queue where we dump the roughly parsed chunk-by-chunk text data :param logDir: where we should dump a verbatim output file ''' @@ -917,7 +915,7 @@ class ServiceManagerThread: # Important Reference: https://stackoverflow.com/questions/375427/non-blocking-read-on-a-subprocess-pipe-in-python # print("This is the svcOutput Reader...") # stdOut.readline() # Skip the first output? TODO: remove? - for tChunk in self._textChunkGenerator(stdOut, logDir, 'stdout.log') : + for tChunk in self._textChunkGenerator(ipcStdOut, logDir, 'stdout.log') : queue.put(tChunk) # tChunk garanteed not to be None self._printProgress("_i") @@ -940,12 +938,12 @@ class ServiceManagerThread: Logging.info("EOF found TDengine STDOUT, marking the process as terminated") self.setStatus(Status.STATUS_STOPPED) - def svcErrorReader(self, stdErr: BinaryIO, queue, logDir: str): + def svcErrorReader(self, ipcStdErr: IpcStream, queue, logDir: str): # os.makedirs(logDir, exist_ok=True) # logFile = os.path.join(logDir,'stderr.log') # fErr = open(logFile, 'wb') # for line in iter(err.readline, b''): - for tChunk in self._textChunkGenerator(stdErr, logDir, 'stderr.log') : + for tChunk in self._textChunkGenerator(ipcStdErr, logDir, 'stderr.log') : queue.put(tChunk) # tChunk garanteed not to be None # fErr.write(line) Logging.info("TDengine STDERR: {}".format(tChunk)) diff --git a/tests/pytest/crash_gen/shared/types.py b/tests/pytest/crash_gen/shared/types.py index 814a8219178d1c01ee0291447939984bdf81df9d..42fd2a1617cf729e4f23fc61a685027f738bc4a3 100644 --- a/tests/pytest/crash_gen/shared/types.py +++ b/tests/pytest/crash_gen/shared/types.py @@ -1,4 +1,4 @@ -from typing import Any, List, Dict, NewType +from typing import Any, BinaryIO, List, Dict, NewType from enum import Enum DirPath = NewType('DirPath', str) @@ -26,3 +26,5 @@ class TdDataType(Enum): TdColumns = Dict[str, TdDataType] TdTags = Dict[str, TdDataType] + +IpcStream = NewType('IpcStream', BinaryIO) \ No newline at end of file diff --git a/tests/pytest/crash_gen/valgrind_taos.supp b/tests/pytest/crash_gen/valgrind_taos.supp index a00b2d830c2e4a3261fcb6fc9a1769b2b583799f..5f6604ba776b03eb5a00280cdf8ea2e8cde36f99 100644 --- a/tests/pytest/crash_gen/valgrind_taos.supp +++ b/tests/pytest/crash_gen/valgrind_taos.supp @@ -17496,4 +17496,25 @@ obj:/usr/bin/python3.8 obj:/usr/bin/python3.8 fun:PyVectorcall_Call +} +{ + + Memcheck:Leak + match-leak-kinds: definite + fun:malloc + fun:__libc_alloc_buffer_allocate + fun:alloc_buffer_allocate + fun:__resolv_conf_allocate + fun:__resolv_conf_load + fun:__resolv_conf_get_current + fun:__res_vinit + fun:maybe_init + fun:context_get + fun:context_get + fun:__resolv_context_get + fun:gaih_inet.constprop.0 + fun:getaddrinfo + fun:taosGetFqdn + fun:taosCheckGlobalCfg + fun:taos_init_imp } \ No newline at end of file diff --git a/tests/pytest/fulltest.sh b/tests/pytest/fulltest.sh index a748c9dd2da623b2648b35ca7dd526dde8beebdb..d8e2a31e70bd14b8feef6c8a45df671473e68537 100755 --- a/tests/pytest/fulltest.sh +++ b/tests/pytest/fulltest.sh @@ -183,7 +183,7 @@ python3 ./test.py -f stable/query_after_reset.py # perfbenchmark python3 ./test.py -f perfbenchmark/bug3433.py #python3 ./test.py -f perfbenchmark/bug3589.py - +python3 ./test.py -f perfbenchmark/taosdemoInsert.py #query python3 ./test.py -f query/filter.py diff --git a/tests/pytest/insert/insertFromCSVPerformance.py b/tests/pytest/insert/insertFromCSVPerformance.py index 84fd1d7ccab85d7e4469c779f10f047160ee0565..e1f741bd1203e93cd25832866f97be2e87d2b37d 100644 --- a/tests/pytest/insert/insertFromCSVPerformance.py +++ b/tests/pytest/insert/insertFromCSVPerformance.py @@ -31,7 +31,7 @@ class insertFromCSVPerformace: self.host = "127.0.0.1" self.user = "root" self.password = "taosdata" - self.config = "/etc/taosperf" + self.config = "/etc/perf" self.conn = taos.connect( self.host, self.user, diff --git a/tests/pytest/perfbenchmark/taosdemoInsert.py b/tests/pytest/perfbenchmark/taosdemoInsert.py new file mode 100644 index 0000000000000000000000000000000000000000..59a8143d5a34014ecbdfe57c6ecc073c0eaeb39c --- /dev/null +++ b/tests/pytest/perfbenchmark/taosdemoInsert.py @@ -0,0 +1,387 @@ +################################################################### +# Copyright (c) 2016 by TAOS Technologies, Inc. +# All rights reserved. +# +# This file is proprietary and confidential to TAOS Technologies. +# No part of this file may be reproduced, stored, transmitted, +# disclosed or used in any form or by any means other than as +# expressly provided by the written permission from Jianhui Tao +# +################################################################### + +# -*- coding: utf-8 -*- + +import taos +import sys +import os +import json +import argparse +import subprocess +import datetime +import re + + +from multiprocessing import cpu_count +from util.log import * +from util.sql import * +from util.cases import * +from util.dnodes import * +from util.dnodes import TDDnode + +class Taosdemo: + def __init__(self, clearCache, dbName, keep): + self.clearCache = clearCache + self.dbname = dbName + self.drop = "yes" + self.keep = keep + self.host = "127.0.0.1" + self.user = "root" + self.password = "taosdata" + # self.config = "/etc/taosperf" + # self.conn = taos.connect( + # self.host, + # self.user, + # self.password, + # self.config) + + # env config + def getBuildPath(self) -> str: + selfPath = os.path.dirname(os.path.realpath(__file__)) + + if ("community" in selfPath): + projPath = selfPath[:selfPath.find("community")] + else: + projPath = selfPath[:selfPath.find("tests")] + + for root, dirs, files in os.walk(projPath): + if ("taosd" in files): + rootRealPath = os.path.dirname(os.path.realpath(root)) + if ("packaging" not in rootRealPath): + buildPath = root[:len(root) - len("/debug/build/bin")] + break + return buildPath + + def getExeToolsDir(self) -> str: + self.debugdir = self.getBuildPath() + "/debug/build/bin" + return self.debugdir + + def getCfgDir(self) -> str: + self.config = self.getBuildPath() + "/sim/dnode1/cfg" + return self.config + + # taodemo insert file config + def dbinfocfg(self) -> dict: + return { + "name": self.dbname, + "drop": self.drop, + "replica": 1, + "days": 10, + "cache": 16, + "blocks": 8, + "precision": "ms", + "keep": self.keep, + "minRows": 100, + "maxRows": 4096, + "comp": 2, + "walLevel": 1, + "cachelast": 0, + "quorum": 1, + "fsync": 3000, + "update": 0 + } + + def type_check(func): + def wrapper(self, **kwargs): + num_types = ["int", "float", "bigint", "tinyint", "smallint", "double"] + str_types = ["binary", "nchar"] + for k ,v in kwargs.items(): + if k.lower() not in num_types and k.lower() not in str_types: + return f"args {k} type error, not allowed" + elif not isinstance(v, (int, list, tuple)): + return f"value {v} type error, not allowed" + elif k.lower() in num_types and not isinstance(v, int): + return f"arg {v} takes 1 positional argument must be type int " + elif isinstance(v, (list,tuple)) and len(v) > 2: + return f"arg {v} takes from 1 to 2 positional arguments but more than 2 were given " + elif isinstance(v,(list,tuple)) and [ False for _ in v if not isinstance(_, int) ]: + return f"arg {v} takes from 1 to 2 positional arguments must be type int " + else: + pass + return func(self, **kwargs) + return wrapper + + @type_check + def column_tag_count(self, **column_tag) -> list : + init_column_tag = [] + for k, v in column_tag.items(): + if re.search(k, "int, float, bigint, tinyint, smallint, double", re.IGNORECASE): + init_column_tag.append({"type": k, "count": v}) + elif re.search(k, "binary, nchar", re.IGNORECASE): + if isinstance(v, int): + init_column_tag.append({"type": k, "count": v, "len":8}) + elif len(v) == 1: + init_column_tag.append({"type": k, "count": v[0], "len": 8}) + else: + init_column_tag.append({"type": k, "count": v[0], "len": v[1]}) + return init_column_tag + + def stbcfg(self, stb: str, child_tab_count: int, rows: int, prechildtab: str, columns: dict, tags: dict) -> dict: + return { + "name": stb, + "child_table_exists": "no", + "childtable_count": child_tab_count, + "childtable_prefix": prechildtab, + "auto_create_table": "no", + "batch_create_tbl_num": 10, + "data_source": "rand", + "insert_mode": "taosc", + "insert_rows": rows, + "childtable_limit": 0, + "childtable_offset": 0, + "rows_per_tbl": 1, + "max_sql_len": 65480, + "disorder_ratio": 0, + "disorder_range": 1000, + "timestamp_step": 10, + "start_timestamp": f"{datetime.datetime.now():%F %X}", + "sample_format": "csv", + "sample_file": "./sample.csv", + "tags_file": "", + "columns": self.column_tag_count(**columns), + "tags": self.column_tag_count(**tags) + } + + def schemecfg(self,intcount=1,floatcount=0,bcount=0,tcount=0,scount=0,doublecount=0,binarycount=0,ncharcount=0): + return { + "INT": intcount, + "FLOAT": floatcount, + "BIGINT": bcount, + "TINYINT": tcount, + "SMALLINT": scount, + "DOUBLE": doublecount, + "BINARY": binarycount, + "NCHAR": ncharcount + } + + def insertcfg(self,db: dict, stbs: list) -> dict: + return { + "filetype": "insert", + "cfgdir": self.config, + "host": self.host, + "port": 6030, + "user": self.user, + "password": self.password, + "thread_count": cpu_count(), + "thread_count_create_tbl": cpu_count(), + "result_file": "/tmp/insert_res.txt", + "confirm_parameter_prompt": "no", + "insert_interval": 0, + "num_of_records_per_req": 100, + "max_sql_len": 1024000, + "databases": [{ + "dbinfo": db, + "super_tables": stbs + }] + } + + def createinsertfile(self,db: dict, stbs: list) -> str: + date = datetime.datetime.now() + file_create_table = f"/tmp/insert_{date:%F-%H%M}.json" + + with open(file_create_table, 'w') as f: + json.dump(self.insertcfg(db, stbs), f) + + return file_create_table + + # taosdemo query file config + def querysqls(self, sql: str) -> list: + return [{"sql":sql,"result":""}] + + def querycfg(self, sql: str) -> dict: + return { + "filetype": "query", + "cfgdir": self.config, + "host": self.host, + "port": 6030, + "user": self.user, + "password": self.password, + "confirm_parameter_prompt": "yes", + "query_times": 10, + "query_mode": "taosc", + "databases": self.dbname, + "specified_table_query": { + "query_interval": 0, + "concurrent": cpu_count(), + "sqls": self.querysqls(sql) + } + } + + def createqueryfile(self, sql: str): + date = datetime.datetime.now() + file_query_table = f"/tmp/query_{date:%F-%H%M}.json" + + with open(file_query_table,"w") as f: + json.dump(self.querycfg(sql), f) + + return file_query_table + + # Execute taosdemo, and delete temporary files when finished + def taosdemotable(self, filepath: str, resultfile="/dev/null"): + taosdemopath = self.getBuildPath() + "/debug/build/bin" + with open(filepath,"r") as f: + filetype = json.load(f)["filetype"] + if filetype == "insert": + taosdemo_table_cmd = f"{taosdemopath}/taosdemo -f {filepath} > {resultfile} 2>&1" + else: + taosdemo_table_cmd = f"yes | {taosdemopath}/taosdemo -f {filepath} > {resultfile} 2>&1" + try: + _ = subprocess.check_output(taosdemo_table_cmd, shell=True).decode("utf-8") + except subprocess.CalledProcessError as e: + _ = e.output + + def droptmpfile(self, filepath: str): + drop_file_cmd = f"[ -f {filepath} ] && rm -f {filepath}" + try: + _ = subprocess.check_output(drop_file_cmd, shell=True).decode("utf-8") + except subprocess.CalledProcessError as e: + _ = e.output + + # TODO:需要完成TD-4153的数据插入和客户端请求的性能查询。 + def td4153insert(self): + + tdLog.printNoPrefix("========== start to create table and insert data ==========") + self.dbname = "td4153" + db = self.dbinfocfg() + stblist = [] + + columntype = self.schemecfg(intcount=1, ncharcount=100) + tagtype = self.schemecfg(intcount=1) + stbname = "stb1" + prechild = "t1" + stable = self.stbcfg( + stb=stbname, + prechildtab=prechild, + child_tab_count=2, + rows=10000, + columns=columntype, + tags=tagtype + ) + stblist.append(stable) + insertfile = self.createinsertfile(db=db, stbs=stblist) + + nmon_file = f"/tmp/insert_{datetime.datetime.now():%F-%H%M}.nmon" + cmd = f"nmon -s5 -F {nmon_file} -m /tmp/" + try: + _ = subprocess.check_output(cmd, shell=True).decode("utf-8") + except subprocess.CalledProcessError as e: + _ = e.output + + self.taosdemotable(insertfile) + self.droptmpfile(insertfile) + self.droptmpfile("/tmp/insert_res.txt") + + # In order to prevent too many performance files from being generated, the nmon file is deleted. + # and the delete statement can be cancelled during the actual test. + self.droptmpfile(nmon_file) + + cmd = f"ps -ef|grep -w nmon| grep -v grep | awk '{{print $2}}'" + try: + time.sleep(10) + _ = subprocess.check_output(cmd,shell=True).decode("utf-8") + except BaseException as e: + raise e + + def td4153query(self): + tdLog.printNoPrefix("========== start to query operation ==========") + + sqls = { + "select_all": "select * from stb1", + "select_join": "select * from t10, t11 where t10.ts=t11.ts" + } + for type, sql in sqls.items(): + result_file = f"/tmp/queryResult_{type}.log" + query_file = self.createqueryfile(sql) + try: + self.taosdemotable(query_file, resultfile=result_file) + except subprocess.CalledProcessError as e: + out_put = e.output + if result_file: + print(f"execute rows {type.split('_')[1]} sql, the sql is: {sql}") + max_sql_time_cmd = f''' + grep -o Spent.*s {result_file} |awk 'NR==1{{max=$2;next}}{{max=max>$2?max:$2}}END{{print "Max=",max,"s"}}' + ''' + max_sql_time = subprocess.check_output(max_sql_time_cmd, shell=True).decode("UTF-8") + print(f"{type.split('_')[1]} rows sql time : {max_sql_time}") + + min_sql_time_cmd = f''' + grep -o Spent.*s {result_file} |awk 'NR==1{{min=$2;next}}{{min=min<$2?min:$2}}END{{print "Min=",min,"s"}}' + ''' + min_sql_time = subprocess.check_output(min_sql_time_cmd, shell=True).decode("UTF-8") + print(f"{type.split('_')[1]} rows sql time : {min_sql_time}") + + avg_sql_time_cmd = f''' + grep -o Spent.*s {result_file} |awk '{{sum+=$2}}END{{print "Average=",sum/NR,"s"}}' + ''' + avg_sql_time = subprocess.check_output(avg_sql_time_cmd, shell=True).decode("UTF-8") + print(f"{type.split('_')[1]} rows sql time : {avg_sql_time}") + + self.droptmpfile(query_file) + self.droptmpfile(result_file) + + drop_query_tmt_file_cmd = " find ./ -name 'querySystemInfo-*' -type f -exec rm {} \; " + try: + _ = subprocess.check_output(drop_query_tmt_file_cmd, shell=True).decode("utf-8") + except subprocess.CalledProcessError as e: + _ = e.output + pass + + def td4153(self): + self.td4153insert() + self.td4153query() + + +if __name__ == '__main__': + parser = argparse.ArgumentParser() + parser.add_argument( + '-r', + '--remove-cache', + action='store_true', + default=False, + help='clear cache before query (default: False)') + parser.add_argument( + '-d', + '--database-name', + action='store', + default='db', + type=str, + help='Database name to be created (default: db)') + parser.add_argument( + '-k', + '--keep-time', + action='store', + default=3650, + type=int, + help='Database keep parameters (default: 3650)') + + args = parser.parse_args() + taosdemo = Taosdemo(args.remove_cache, args.database_name, args.keep_time) + # taosdemo.conn = taos.connect( + # taosdemo.host, + # taosdemo.user, + # taosdemo.password, + # taosdemo.config + # ) + + debugdir = taosdemo.getExeToolsDir() + cfgdir = taosdemo.getCfgDir() + cmd = f"{debugdir}/taosd -c {cfgdir} >/dev/null 2>&1 &" + try: + _ = subprocess.check_output(cmd, shell=True).decode("utf-8") + except subprocess.CalledProcessError as e: + _ = e.output + + if taosdemo.clearCache: + # must be root permission + subprocess.check_output("echo 3 > /proc/sys/vm/drop_caches", shell=True).decode("utf-8") + + taosdemo.td4153() diff --git a/tests/pytest/tools/taosdemoPerformance.py b/tests/pytest/tools/taosdemoPerformance.py index a32cba167ef83a0036683f28297ca15d53939514..a45393e22284d675584c5dddd71fc507bcb2563f 100644 --- a/tests/pytest/tools/taosdemoPerformance.py +++ b/tests/pytest/tools/taosdemoPerformance.py @@ -24,7 +24,7 @@ class taosdemoPerformace: self.host = "127.0.0.1" self.user = "root" self.password = "taosdata" - self.config = "/etc/taosperf" + self.config = "/etc/perf" self.conn = taos.connect( self.host, self.user, @@ -77,7 +77,7 @@ class taosdemoPerformace: insert_data = { "filetype": "insert", - "cfgdir": "/etc/taosperf", + "cfgdir": "/etc/perf", "host": "127.0.0.1", "port": 6030, "user": "root", diff --git a/tests/script/general/parser/binary_escapeCharacter.sim b/tests/script/general/parser/binary_escapeCharacter.sim index b5bb10284b9b1b09456f4fe9dd3e07cb14a33721..176edd4e298d92689140644587622b20a267f63e 100644 --- a/tests/script/general/parser/binary_escapeCharacter.sim +++ b/tests/script/general/parser/binary_escapeCharacter.sim @@ -1,3 +1,4 @@ + system sh/stop_dnodes.sh system sh/deploy.sh -n dnode1 -i 1 diff --git a/tests/script/general/parser/function.sim b/tests/script/general/parser/function.sim index 65058333fb6a6dea4d0dd583b86f2927bdcc7979..56ce15c36fbbe71753028f60a03d5cdf73a03571 100644 --- a/tests/script/general/parser/function.sim +++ b/tests/script/general/parser/function.sim @@ -809,3 +809,5 @@ endi if $data00 != 1 then return -1 endi + +print ====================> TODO stddev + normal column filter diff --git a/tests/script/general/parser/last_cache.sim b/tests/script/general/parser/last_cache.sim new file mode 100644 index 0000000000000000000000000000000000000000..9d7da9ddbabbd668382740a230ecc9582c27fe84 --- /dev/null +++ b/tests/script/general/parser/last_cache.sim @@ -0,0 +1,71 @@ +system sh/stop_dnodes.sh +system sh/deploy.sh -n dnode1 -i 1 +system sh/cfg.sh -n dnode1 -c walLevel -v 0 +system sh/cfg.sh -n dnode1 -c maxtablesPerVnode -v 4 +system sh/exec.sh -n dnode1 -s start + +sleep 100 +sql connect +print ======================== dnode1 start + +$db = testdb + +sql create database $db cachelast 2 +sql use $db + +sql create stable st2 (ts timestamp, f1 int, f2 double, f3 binary(10), f4 timestamp) tags (id int) + +sql create table tb1 using st2 tags (1); +sql create table tb2 using st2 tags (2); +sql create table tb3 using st2 tags (3); +sql create table tb4 using st2 tags (4); +sql create table tb5 using st2 tags (1); +sql create table tb6 using st2 tags (2); +sql create table tb7 using st2 tags (3); +sql create table tb8 using st2 tags (4); +sql create table tb9 using st2 tags (5); +sql create table tba using st2 tags (5); +sql create table tbb using st2 tags (5); +sql create table tbc using st2 tags (5); +sql create table tbd using st2 tags (5); +sql create table tbe using st2 tags (5); + +sql insert into tb1 values ("2021-05-09 10:10:10", 1, 2.0, '3', -1000) +sql insert into tb1 values ("2021-05-10 10:10:11", 4, 5.0, NULL, -2000) +sql insert into tb1 values ("2021-05-12 10:10:12", 6,NULL, NULL, -3000) + +sql insert into tb2 values ("2021-05-09 10:11:13",-1,-2.0,'-3', -1001) +sql insert into tb2 values ("2021-05-10 10:11:14",-4,-5.0, NULL, -2001) +sql insert into tb2 values ("2021-05-11 10:11:15",-6, -7, '-8', -3001) + +sql insert into tb3 values ("2021-05-09 10:12:17", 7, 8.0, '9' , -1002) +sql insert into tb3 values ("2021-05-09 10:12:17",10,11.0, NULL, -2002) +sql insert into tb3 values ("2021-05-09 10:12:18",12,NULL, NULL, -3002) + +sql insert into tb4 values ("2021-05-09 10:12:19",13,14.0,'15' , -1003) +sql insert into tb4 values ("2021-05-10 10:12:20",16,17.0, NULL, -2003) +sql insert into tb4 values ("2021-05-11 10:12:21",18,NULL, NULL, -3003) + +sql insert into tb5 values ("2021-05-09 10:12:22",19, 20, '21', -1004) +sql insert into tb6 values ("2021-05-11 10:12:23",22, 23, NULL, -2004) +sql insert into tb7 values ("2021-05-10 10:12:24",24,NULL, '25', -3004) +sql insert into tb8 values ("2021-05-11 10:12:25",26,NULL, '27', -4004) + +sql insert into tb9 values ("2021-05-09 10:12:26",28, 29, '30', -1005) +sql insert into tba values ("2021-05-10 10:12:27",31, 32, NULL, -2005) +sql insert into tbb values ("2021-05-10 10:12:28",33,NULL, '35', -3005) +sql insert into tbc values ("2021-05-11 10:12:29",36, 37, NULL, -4005) +sql insert into tbd values ("2021-05-11 10:12:29",NULL,NULL,NULL,NULL ) + +run general/parser/last_cache_query.sim + +system sh/exec.sh -n dnode1 -s stop -x SIGINT + +system sh/exec.sh -n dnode1 -s start + +run general/parser/last_cache_query.sim + +system sh/exec.sh -n dnode1 -s stop -x SIGINT + + + diff --git a/tests/script/general/parser/last_cache_query.sim b/tests/script/general/parser/last_cache_query.sim new file mode 100644 index 0000000000000000000000000000000000000000..2acd00058592c3d4f70600aad1b2ee41a688f1e1 --- /dev/null +++ b/tests/script/general/parser/last_cache_query.sim @@ -0,0 +1,416 @@ + +sleep 100 +sql connect + +$db = testdb + +sql use $db + +print "test tb1" + +sql select last(ts) from tb1 +if $rows != 1 then + return -1 +endi +if $data00 != @21-05-12 10:10:12.000@ then + print $data00 + return -1 +endi + + +sql select last(f1) from tb1 +if $rows != 1 then + return -1 +endi +if $data00 != 6 then + print $data00 + return -1 +endi + +sql select last(*) from tb1 +if $rows != 1 then + return -1 +endi +if $data00 != @21-05-12 10:10:12.000@ then + print $data00 + return -1 +endi +if $data01 != 6 then + return -1 +endi +if $data02 != 5.000000000 then + print $data02 + return -1 +endi +if $data03 != 3 then + return -1 +endi +if $data04 != @70-01-01 07:59:57.000@ then + return -1 +endi + + +sql select last(tb1.*,ts,f4) from tb1 +if $rows != 1 then + return -1 +endi +if $data00 != @21-05-12 10:10:12.000@ then + print $data00 + return -1 +endi +if $data01 != 6 then + return -1 +endi +if $data02 != 5.000000000 then + print $data02 + return -1 +endi +if $data03 != 3 then + return -1 +endi +if $data04 != @70-01-01 07:59:57.000@ then + return -1 +endi +if $data05 != @21-05-12 10:10:12.000@ then + print $data00 + return -1 +endi +if $data06 != @70-01-01 07:59:57.000@ then + return -1 +endi + + + + +print "test tb2" + +sql select last(ts) from tb2 +if $rows != 1 then + return -1 +endi +if $data00 != @21-05-11 10:11:15.000@ then + print $data00 + return -1 +endi + + +sql select last(f1) from tb2 +if $rows != 1 then + return -1 +endi +if $data00 != -6 then + print $data00 + return -1 +endi + +sql select last(*) from tb2 +if $rows != 1 then + return -1 +endi +if $data00 != @21-05-11 10:11:15.000@ then + print $data00 + return -1 +endi +if $data01 != -6 then + return -1 +endi +if $data02 != -7.000000000 then + print $data02 + return -1 +endi +if $data03 != -8 then + return -1 +endi +if $data04 != @70-01-01 07:59:56.999@ then + if $data04 != @70-01-01 07:59:57.-01@ then + return -1 + endi +endi + + +sql select last(tb2.*,ts,f4) from tb2 +if $rows != 1 then + return -1 +endi +if $data00 != @21-05-11 10:11:15.000@ then + print $data00 + return -1 +endi +if $data01 != -6 then + return -1 +endi +if $data02 != -7.000000000 then + print $data02 + return -1 +endi +if $data03 != -8 then + return -1 +endi +if $data04 != @70-01-01 07:59:56.999@ then + if $data04 != @70-01-01 07:59:57.-01@ then + return -1 + endi +endi +if $data05 != @21-05-11 10:11:15.000@ then + print $data00 + return -1 +endi +if $data06 != @70-01-01 07:59:56.999@ then + if $data04 != @70-01-01 07:59:57.-01@ then + return -1 + endi +endi + + + + + + + +print "test tbd" +sql select last(*) from tbd +if $rows != 1 then + return -1 +endi +if $data00 != @21-05-11 10:12:29.000@ then + print $data00 + return -1 +endi +if $data01 != NULL then + return -1 +endi +if $data02 != NULL then + print $data02 + return -1 +endi +if $data03 != NULL then + return -1 +endi +if $data04 != NULL then + return -1 +endi + + + +print "test tbe" +sql select last(*) from tbe +if $rows != 0 then + return -1 +endi + + + + + +print "test stable" +sql select last(ts) from st2 +if $rows != 1 then + return -1 +endi +if $data00 != @21-05-12 10:10:12.000@ then + print $data00 + return -1 +endi + + +sql select last(f1) from st2 +if $rows != 1 then + return -1 +endi +if $data00 != 6 then + print $data00 + return -1 +endi + +sql select last(*) from st2 +if $rows != 1 then + return -1 +endi +if $data00 != @21-05-12 10:10:12.000@ then + print $data00 + return -1 +endi +if $data01 != 6 then + return -1 +endi +if $data02 != 37.000000000 then + print $data02 + return -1 +endi +if $data03 != 27 then + return -1 +endi +if $data04 != @70-01-01 07:59:57.000@ then + return -1 +endi + + +sql select last(st2.*,ts,f4) from st2 +if $rows != 1 then + return -1 +endi +if $data00 != @21-05-12 10:10:12.000@ then + print $data00 + return -1 +endi +if $data01 != 6 then + return -1 +endi +if $data02 != 37.000000000 then + print $data02 + return -1 +endi +if $data03 != 27 then + return -1 +endi +if $data04 != @70-01-01 07:59:57.000@ then + return -1 +endi +if $data05 != @21-05-12 10:10:12.000@ then + print $data00 + return -1 +endi +if $data06 != @70-01-01 07:59:57.000@ then + return -1 +endi + + +sql select last(*) from st2 group by id +if $rows != 5 then + return -1 +endi +if $data00 != @21-05-12 10:10:12.000@ then + return -1 +endi +if $data01 != 6 then + return -1 +endi +if $data02 != 5.000000000 then + print $data02 + return -1 +endi +if $data03 != 21 then + return -1 +endi +if $data04 != @70-01-01 07:59:57.000@ then + return -1 +endi +if $data05 != 1 then + return -1 +endi +if $data10 != @21-05-11 10:12:23.000@ then + return -1 +endi +if $data11 != 22 then + return -1 +endi +if $data12 != 23.000000000 then + print $data02 + return -1 +endi +if $data13 != -8 then + return -1 +endi +if $data14 != @70-01-01 07:59:57.996@ then +if $data14 != @70-01-01 07:59:58.-04@ then + print $data14 + return -1 +endi +endi +if $data15 != 2 then + return -1 +endi +if $data20 != @21-05-10 10:12:24.000@ then + return -1 +endi +if $data21 != 24 then + return -1 +endi +if $data22 != 8.000000000 then + print $data02 + return -1 +endi +if $data23 != 25 then + return -1 +endi +if $data24 != @70-01-01 07:59:56.996@ then +if $data24 != @70-01-01 07:59:57.-04@ then + return -1 +endi +endi +if $data25 != 3 then + return -1 +endi +if $data30 != @21-05-11 10:12:25.000@ then + return -1 +endi +if $data31 != 26 then + return -1 +endi +if $data32 != 17.000000000 then + print $data02 + return -1 +endi +if $data33 != 27 then + return -1 +endi +if $data34 != @70-01-01 07:59:55.996@ then +if $data34 != @70-01-01 07:59:56.-04@ then + return -1 +endi +endi +if $data35 != 4 then + return -1 +endi +if $data40 != @21-05-11 10:12:29.000@ then + return -1 +endi +if $data41 != 36 then + return -1 +endi +if $data42 != 37.000000000 then + print $data02 + return -1 +endi +if $data43 != 35 then + return -1 +endi +if $data44 != @70-01-01 07:59:55.995@ then +if $data44 != @70-01-01 07:59:56.-05@ then + return -1 +endi +endi +if $data45 != 5 then + return -1 +endi + + +print "test tbn" +sql create table tbn (ts timestamp, f1 int, f2 double, f3 binary(10), f4 timestamp) +sql insert into tbn values ("2021-05-09 10:10:10", 1, 2.0, '3', -1000) +sql insert into tbn values ("2021-05-10 10:10:11", 4, 5.0, NULL, -2000) +sql insert into tbn values ("2021-05-12 10:10:12", 6,NULL, NULL, -3000) +sql insert into tbn values ("2021-05-13 10:10:12", NULL,NULL, NULL,NULL) + +sql select last(*) from tbn; +if $rows != 1 then + return -1 +endi +if $data00 != @21-05-13 10:10:12.000@ then + print $data00 + return -1 +endi +if $data01 != 6 then + return -1 +endi +if $data02 != 5.000000000 then + print $data02 + return -1 +endi +if $data03 != 3 then + return -1 +endi +if $data04 != @70-01-01 07:59:57.000@ then + return -1 +endi + diff --git a/tests/script/general/parser/nchar.sim b/tests/script/general/parser/nchar.sim index 786cee651b793b23ec8519be400554567af49852..84719efcab9a18e6f554cdfb84942adf1de5add9 100644 --- a/tests/script/general/parser/nchar.sim +++ b/tests/script/general/parser/nchar.sim @@ -1,6 +1,5 @@ system sh/stop_dnodes.sh - system sh/deploy.sh -n dnode1 -i 1 system sh/cfg.sh -n dnode1 -c walLevel -v 1 system sh/exec.sh -n dnode1 -s start diff --git a/tests/script/general/parser/nestquery.sim b/tests/script/general/parser/nestquery.sim new file mode 100644 index 0000000000000000000000000000000000000000..9e2736833f56cb6224cc1a1b351b2171213f5f8f --- /dev/null +++ b/tests/script/general/parser/nestquery.sim @@ -0,0 +1,207 @@ +system sh/stop_dnodes.sh + +system sh/deploy.sh -n dnode1 -i 1 +system sh/cfg.sh -n dnode1 -c walLevel -v 1 +system sh/exec.sh -n dnode1 -s start + +sleep 100 +sql connect + +print ======================== dnode1 start + +$dbPrefix = nest_query +$tbPrefix = nest_tb +$mtPrefix = nest_mt +$tbNum = 10 +$rowNum = 10000 +$totalNum = $tbNum * $rowNum + +print =============== nestquery.sim + +$i = 0 +$db = $dbPrefix . $i +$mt = $mtPrefix . $i + +sql drop database if exists $db +sql create database if not exists $db + +sql use $db +sql create table $mt (ts timestamp, c1 int, c2 float, c3 bigint, c4 smallint, c5 tinyint, c6 double, c7 bool, c8 binary(10), c9 nchar(9)) TAGS(t1 int) + +$half = $tbNum / 2 + +$i = 0 +while $i < $half + $tb = $tbPrefix . $i + + $nextSuffix = $i + $half + $tb1 = $tbPrefix . $nextSuffix + + sql create table $tb using $mt tags( $i ) + sql create table $tb1 using $mt tags( $nextSuffix ) + + $x = 0 + while $x < $rowNum + $y = $x * 60000 + $ms = 1600099200000 + $y + $c = $x / 100 + $c = $c * 100 + $c = $x - $c + $binary = 'binary . $c + $binary = $binary . ' + $nchar = 'nchar . $c + $nchar = $nchar . ' + sql insert into $tb values ($ms , $c , $c , $c , $c , $c , $c , $c , $binary , $nchar ) $tb1 values ($ms , $c , $c , $c , $c , $c , $c , $c , $binary , $nchar ) + $x = $x + 1 + endw + + $i = $i + 1 +endw + +sleep 100 + +$i = 1 +$tb = $tbPrefix . $i + +print ==============> simple nest query test +sql select count(*) from (select count(*) from nest_mt0) +if $rows != 1 then + return -1 +endi + +if $data00 != 1 then + return -1 +endi + +sql select count(*) from (select count(*) from nest_mt0 group by tbname) +if $rows != 1 then + return -1 +endi + +if $data00 != 10 then + return -1 +endi + +sql select count(*) from (select count(*) from nest_mt0 interval(10h) group by tbname) +if $rows != 1 then + return -1 +endi + +if $data00 != 170 then + return -1 +endi + +sql select sum(a) from (select count(*) a from nest_mt0 interval(10h) group by tbname) +if $rows != 1 then + return -1 +endi + +if $data00 != 100000 then + return -1 +endi + +print =================> alias name test +sql select ts from (select count(*) a from nest_tb0 interval(1h)) +if $rows != 167 then + return -1 +endi + +if $data00 != @20-09-15 00:00:00.000@ then + return -1 +endi + +sql select count(a) from (select count(*) a from nest_tb0 interval(1h)) +if $rows != 1 then + return -1 +endi + +if $data00 != 167 then + return -1 +endi + +print ================>master query + filter +sql select t.* from (select count(*) a from nest_tb0 interval(10h)) t where t.a <= 520; +if $rows != 2 then + return -1 +endi + + +print ===================> nest query interval + + + +print ===================> complex query + + + +print ===================> group by + having + + + + + +print =========================> nest query join +sql select a.ts,a.k,b.ts from (select count(*) k from nest_tb0 interval(30a)) a, (select count(*) f from nest_tb1 interval(30a)) b where a.ts = b.ts ; +if $rows != 10000 then + return -1 +endi + +if $data00 != @20-09-15 00:00:00.000@ then + return -1 +endi + +if $data01 != 1 then + return -1 +endi + +if $data02 != @20-09-15 00:00:00.000@ then + return -1 +endi + +if $data10 != @20-09-15 00:01:00.000@ then + return -1 +endi + +if $data11 != 1 then + return -1 +endi + +if $data12 != @20-09-15 00:01:00.000@ then + return -1 +endi + +sql select sum(a.k), sum(b.f) from (select count(*) k from nest_tb0 interval(30a)) a, (select count(*) f from nest_tb1 interval(30a)) b where a.ts = b.ts ; +if $rows != 1 then + return -1 +endi + +if $data00 != 10000 then + return -1 +endi + +if $data01 != 10000 then + return -1 +endi + +sql select a.ts,a.k,b.ts,c.ts,c.ts,c.x from (select count(*) k from nest_tb0 interval(30a)) a, (select count(*) f from nest_tb1 interval(30a)) b, (select count(*) x from nest_tb2 interval(30a)) c where a.ts = b.ts and a.ts = c.ts +if $rows != 10000 then + return -1 +endi + +if $data00 != @20-09-15 00:00:00.000@ then + return -1 +endi + +if $data01 != 1 then + return -1 +endi + +if $data02 != @20-09-15 00:00:00.000@ then + return -1 +endi + +if $data03 != @20-09-15 00:00:00.000@ then + return -1 +endi + +system sh/exec.sh -n dnode1 -s stop -x SIGINT \ No newline at end of file diff --git a/tests/script/general/parser/select_with_tags.sim b/tests/script/general/parser/select_with_tags.sim index 7e5f31f7594bdde5e068e0d754ae863662696759..9d5214bfaaa6fdca44def066a85f49b87731c836 100644 --- a/tests/script/general/parser/select_with_tags.sim +++ b/tests/script/general/parser/select_with_tags.sim @@ -887,10 +887,16 @@ sql_error select tbname, t1 from select_tags_mt0 interval(1y); #valid sql: select first(c1), last(c2), count(*) from select_tags_mt0 group by tbname, t1; #valid sql: select first(c1), tbname, t1 from select_tags_mt0 group by t2; +print ==================================>TD-4231 +sql_error select t1,tbname from select_tags_mt0 where c1<0 +sql_error select t1,tbname from select_tags_mt0 where c1<0 and tbname in ('select_tags_tb12') + +sql select tbname from select_tags_mt0 where tbname in ('select_tags_tb12'); + sql_error select first(c1), last(c2), t1 from select_tags_mt0 group by tbname; sql_error select first(c1), last(c2), tbname, t2 from select_tags_mt0 group by tbname; sql_error select first(c1), count(*), t2, t1, tbname from select_tags_mt0 group by tbname; -# this sql is valid: select first(c1), t2 from select_tags_mt0 group by tbname; +#valid sql: select first(c1), t2 from select_tags_mt0 group by tbname; #sql select first(ts), tbname from select_tags_mt0 group by tbname; #sql select count(c1) from select_tags_mt0 where c1=99 group by tbname; diff --git a/tests/script/general/parser/testSuite.sim b/tests/script/general/parser/testSuite.sim index 1294b093c27aaaa8ecbcafa50dfacb3b10f7dfdd..afb76c799ecc2c7c55058cd4f0e287975c0d7d45 100644 --- a/tests/script/general/parser/testSuite.sim +++ b/tests/script/general/parser/testSuite.sim @@ -54,9 +54,10 @@ run general/parser/timestamp.sim run general/parser/sliding.sim run general/parser/function.sim run general/parser/stableOp.sim - run general/parser/having.sim run general/parser/having_child.sim run general/parser/slimit_alter_tags.sim run general/parser/binary_escapeCharacter.sim +run general/parser/between_and.sim +run general/parser/last_cache.sim diff --git a/tests/script/unique/arbitrator/dn3_mn1_vnode_nomaster.sim b/tests/script/unique/arbitrator/dn3_mn1_vnode_nomaster.sim index 8e15c4f527db044d52cd725714f15b98161da2ce..b9ee508f78602cee7d6c9f17dbc3e250e7014f72 100644 --- a/tests/script/unique/arbitrator/dn3_mn1_vnode_nomaster.sim +++ b/tests/script/unique/arbitrator/dn3_mn1_vnode_nomaster.sim @@ -158,7 +158,7 @@ if $dnode4Vtatus != offline then sleep 2000 goto wait_dnode4_vgroup_offline endi -if $dnode3Vtatus != master then +if $dnode3Vtatus != unsynced then sleep 2000 goto wait_dnode4_vgroup_offline endi