diff --git a/docs/en/14-reference/03-connector/rust.mdx b/docs/en/14-reference/03-connector/rust.mdx index a5cbaeac8077cda42690d9cc232062a685a51f41..56ca586c7e8ada6e4422596906e01887d4726fd0 100644 --- a/docs/en/14-reference/03-connector/rust.mdx +++ b/docs/en/14-reference/03-connector/rust.mdx @@ -250,7 +250,7 @@ The [Taos] structure is the connection manager in [libtaos] and provides two mai Column information is stored using [ColumnMeta]. - ``rust + ```rust let cols = &q.column_meta; for col in cols { println!("name: {}, type: {:?} , bytes: {}", col.name, col.type_, col.bytes); diff --git a/docs/en/14-reference/05-taosbenchmark.md b/docs/en/14-reference/05-taosbenchmark.md index 7cf1f95eb116b5f87b3bc1e05b647b9b0da3c544..5d47cc06e8034d8c49669d71c3f98c1f587acb33 100644 --- a/docs/en/14-reference/05-taosbenchmark.md +++ b/docs/en/14-reference/05-taosbenchmark.md @@ -7,7 +7,7 @@ description: "taosBenchmark (once called taosdemo ) is a tool for testing the pe ## Introduction -taosBenchmark (formerly taosdemo ) is a tool for testing the performance of TDengine products. taosBenchmark can test the performance of TDengine's insert, query, and subscription functions and simulate large amounts of data generated by many devices. taosBenchmark can flexibly control the number and type of databases, supertables, tag columns, number and type of data columns, and sub-tables, and types of databases, super tables, the number and types of data columns, the number of sub-tables, the amount of data per sub-table, the time interval for inserting data, the number of working threads, whether and how to insert disordered data, and so on. The installer provides taosdemo as a soft link to taosBenchmark for compatibility and for the convenience of past users. +taosBenchmark (formerly taosdemo ) is a tool for testing the performance of TDengine products. taosBenchmark can test the performance of TDengine's insert, query, and subscription functions and simulate large amounts of data generated by many devices. taosBenchmark can be configured to generate user defined databases, supertables, subtables, and the time series data to populate these for performance benchmarking. taosBenchmark is highly configurable and some of the configurations include the time interval for inserting data, the number of working threads and the capability to insert disordered data. The installer provides taosdemo as a soft link to taosBenchmark for compatibility with past users. ## Installation @@ -21,9 +21,13 @@ There are two ways to install taosBenchmark: ### Configuration and running methods -TaosBenchmark needs to be executed on the terminal of the operating system, it supports two configuration methods: [Command-line arguments](#Command-line arguments in detailed) and [JSON configuration file](#Configuration file arguments in detailed). These two methods are mutually exclusive. Users can use `-f ` to specify a configuration file. When running taosBenchmark with command-line arguments to control its behavior, users should use other parameters for configuration, but not the `-f` parameter. In addition, taosBenchmark offers a special way of running without parameters. +TaosBenchmark needs to be executed on the terminal of the operating system, it supports two configuration methods: [Command-line arguments](#command-line-arguments-in-detail) and [JSON configuration file](#configuration-file-parameters-in-detail). These two methods are mutually exclusive. Users can use `-f ` to specify a configuration file. When running taosBenchmark with command-line arguments to control its behavior, users should use other parameters for configuration, but not the `-f` parameter. In addition, taosBenchmark offers a special way of running without parameters. +<<<<<<< HEAD taosBenchmark supports complete performance testing of TDengine. taosBenchmark supports the TDengine functions in three categories: write, query, and subscribe. These three functions are mutually exclusive, and users can select only one of them each time taosBenchmark runs. It is important to note that the type of functionality to be tested is not configurable when using the command-line configuration method, which can only test writing performance. To test the query and subscription performance of the TDengine, you must use the configuration file method and specify the function type to test via the parameter `filetype` in the configuration file. +======= +taosBenchmark supports the complete performance testing of TDengine by providing functionally to write, query, and subscribe. These three functions are mutually exclusive, users can only select one of them each time taosBenchmark runs. The query and subscribe functionalities are only configurable using a json configuration file by specifying the parameter `filetype`, while write can be performed through both the command-line and a configuration file. +>>>>>>> 108548b4d6 (docs: typo) **Make sure that the TDengine cluster is running correctly before running taosBenchmark. ** @@ -57,9 +61,8 @@ Use the following command-line to run taosBenchmark and control its behavior via taosBenchmark -f ``` -**Here are a few examples of configuration files:** - -#### Example of inserting a scenario JSON configuration file +#### Configuration file examples +##### Example of inserting a scenario JSON configuration file
insert.json @@ -70,7 +73,7 @@ taosBenchmark -f
-#### Query Scenario JSON Profile Example +##### Query Scenario JSON Profile Example
query.json @@ -81,7 +84,7 @@ taosBenchmark -f
-#### Subscription JSON configuration example +##### Subscription JSON configuration example
subscribe.json @@ -172,7 +175,7 @@ taosBenchmark -A INT,DOUBLE,NCHAR,BINARY\(16\) Switch parameter specifying whether to use escape characters in the super table and sub-table names. By default is not used. - **-C/--chinese** : - Switch specifying whether to use Unicode Chinese characters in nchar and binary. By default is not used. + specify whether to use Unicode Chinese characters in nchar and binary, the default is no. - **-N/--normal-table** : This parameter indicates that taosBenchmark will create only normal tables instead of super tables. The default value is false. It can be used if the insert mode is taosc, stmt, and rest. @@ -373,7 +376,7 @@ The configuration parameters for querying the sub-tables or the normal tables ar - **sqls**. - **sql**: the SQL command to be executed. - - **result**: the file to save the query result. If it is unspecified, taosBenchark will not save the result. + - **result**: the file to save the query result. If it is unspecified, taosBenchmark will not save the result. #### Configuration parameters of query super table diff --git a/docs/en/20-third-party/01-grafana.mdx b/docs/en/20-third-party/01-grafana.mdx index 9076f163bea60e87b5d1aaed79d966572463d56d..5dbeb31a231464e48b4f977420f03f0ede81e78e 100644 --- a/docs/en/20-third-party/01-grafana.mdx +++ b/docs/en/20-third-party/01-grafana.mdx @@ -31,38 +31,41 @@ TDengine currently supports Grafana versions 7.5 and above. Users can go to the ### Install Grafana Plugin and Configure Data Source - + -Set the url and authorization environment variables by `export` or a [`.env`(dotenv) file](https://hexdocs.pm/dotenvy/dotenv-file-format.html): +Under Grafana 8, plugin catalog allows you to [browse and manage plugins within Grafana](https://grafana.com/docs/grafana/next/administration/plugin-management/#plugin-catalog) (but for Grafana 7.x, use **With Script** or **Install & Configure Manually**). Find the page at **Configurations > Plugins**, search **TDengine** and click it to install. -```sh -export TDENGINE_API=http://tdengine.local:6041 -# user + password -export TDENGINE_USER=user -export TDENGINE_PASSWORD=password - -# Other useful variables -# - If to install TDengine data source, default is true -export TDENGINE_DS_ENABLED=false -# - Data source name to be created, default is TDengine -export TDENGINE_DS_NAME=TDengine -# - Data source organization id, default is 1 -export GF_ORG_ID=1 -# - Data source is editable in admin ui or not, default is 0 (false) -export TDENGINE_EDITABLE=1 -``` +![Search tdengine in grafana plugins](./grafana/grafana-plugin-search-tdengine.png) + +Installation may cost some minutes, then you can **Create a TDengine data source**: + +![Install and configure Grafana data source](./grafana/grafana-install-and-config.png) + +Then you can add a TDengine data source by filling up the configuration options. + +![TDengine Database Grafana plugin add data source](./grafana/grafana-data-source.png) + +You can create dashboards with TDengine now. + + + -Run `install.sh`: +On a server with Grafana installed, run `install.sh` with TDengine url and username/passwords will install TDengine data source plugin and add a data source named TDengine. This is the recommended way for Grafana 7.x or [Grafana provisioning](https://grafana.com/docs/grafana/latest/administration/provisioning/) users. ```sh -bash -c "$(curl -fsSL https://raw.githubusercontent.com/taosdata/grafanaplugin/master/install.sh)" +bash -c "$(curl -fsSL \ + https://raw.githubusercontent.com/taosdata/grafanaplugin/master/install.sh)" -- \ + -a http://localhost:6041 \ + -u root \ + -p taosdata ``` -With this script, TDengine data source plugin and the Grafana data source will be installed and created automatically with Grafana provisioning configurations. Save the script and type `./install.sh --help` for the full usage of the script. +Restart Grafana service and open Grafana in web-browser, usually . -And then, restart Grafana service and open Grafana in web-browser, usually . +Save the script and type `./install.sh --help` for the full usage of the script. + Follow the installation steps in [Grafana](https://grafana.com/grafana/plugins/tdengine-datasource/?tab=installation) with the [``grafana-cli`` command-line tool](https://grafana.com/docs/grafana/latest/administration/cli/) for plugin installation. @@ -115,6 +118,73 @@ Click `Save & Test` to test. You should see a success message if the test worked ![TDengine Database TDinsight plugin add database 4](./grafana/add_datasource4.webp) + + + +Please refer to [Install plugins in the Docker container](https://grafana.com/docs/grafana/next/setup-grafana/installation/docker/#install-plugins-in-the-docker-container). This will install `tdengine-datasource` plugin when Grafana container starts: + +```bash +docker run -d \ + -p 3000:3000 \ + --name=grafana \ + -e "GF_INSTALL_PLUGINS=tdengine-datasource" \ + grafana/grafana +``` + +You can setup a zero-configuration stack for TDengine + Grafana by [docker-compose](https://docs.docker.com/compose/) and [Grafana provisioning](https://grafana.com/docs/grafana/latest/administration/provisioning/) file： + +1. Save the provisioning configuration file to `tdengine.yml`. + + ```yml + apiVersion: 1 + datasources: + - name: TDengine + type: tdengine-datasource + orgId: 1 + url: "$TDENGINE_API" + isDefault: true + secureJsonData: + url: "$TDENGINE_URL" + basicAuth: "$TDENGINE_BASIC_AUTH" + token: "$TDENGINE_CLOUD_TOKEN" + version: 1 + editable: true + ``` + +2. Write `docker-compose.yml` with [TDengine](https://hub.docker.com/r/tdengine/tdengine) and [Grafana](https://hub.docker.com/r/grafana/grafana) image. + + ```yml + version: "3.7" + + services: + tdengine: + image: tdengine/tdengine:2.6.0.2 + environment: + TAOS_FQDN: tdengine + volumes: + - tdengine-data:/var/lib/taos/ + grafana: + image: grafana/grafana:8.5.6 + volumes: + - ./tdengine.yml/:/etc/grafana/provisioning/tdengine.yml + - grafana-data:/var/lib/grafana + environment: + # install tdengine plugin at start + GF_INSTALL_PLUGINS: "tdengine-datasource" + TDENGINE_URL: "http://tdengine:6041" + #printf "$TDENGINE_USER:$TDENGINE_PASSWORD" | base64 + TDENGINE_BASIC_AUTH: "cm9vdDp0YmFzZTEyNQ==" + ports: + - 3000:3000 + volumes: + grafana-data: + tdengine-data: + ``` + +3. Start TDengine and Grafana services: `docker-compose up -d`. + +Open Grafana , and you can add dashboard with TDengine now. + diff --git a/docs/en/20-third-party/grafana/grafana-data-source.png b/docs/en/20-third-party/grafana/grafana-data-source.png new file mode 100644 index 0000000000000000000000000000000000000000..989ffcca0bf5baae8798b0695e259aca35f0442a Binary files /dev/null and b/docs/en/20-third-party/grafana/grafana-data-source.png differ diff --git a/docs/en/20-third-party/grafana/grafana-install-and-config.png b/docs/en/20-third-party/grafana/grafana-install-and-config.png new file mode 100644 index 0000000000000000000000000000000000000000..b918da8b2d62e694fe1797e09cf8f23f103bc97e Binary files /dev/null and b/docs/en/20-third-party/grafana/grafana-install-and-config.png differ diff --git a/docs/en/20-third-party/grafana/grafana-plugin-search-tdengine.png b/docs/en/20-third-party/grafana/grafana-plugin-search-tdengine.png new file mode 100644 index 0000000000000000000000000000000000000000..cf3b66977b64f7dcd617f06024a66066cd62810e Binary files /dev/null and b/docs/en/20-third-party/grafana/grafana-plugin-search-tdengine.png differ diff --git a/docs/zh/20-third-party/01-grafana.mdx b/docs/zh/20-third-party/01-grafana.mdx index 09c0d786cf5ac3fc9a6c5e6a72087be0b658454d..93090ffd38c3ce66488826c486584dd305dbc20c 100644 --- a/docs/zh/20-third-party/01-grafana.mdx +++ b/docs/zh/20-third-party/01-grafana.mdx @@ -29,39 +29,41 @@ TDengine 能够与开源数据可视化系统 [Grafana](https://www.grafana.com/ ### 安装 Grafana Plugin 并配置数据源 - + -将集群信息设置为环境变量；也可以使用 `.env` 文件，请参考 [dotenv](https://hexdocs.pm/dotenvy/dotenv-file-format.html)： +使用 Grafana 最新版本（8.5+），您可以在 Grafana 中[浏览和管理插件](https://grafana.com/docs/grafana/next/administration/plugin-management/#plugin-catalog)（对于 7.x 版本，请使用 **安装脚本** 或 **手动安装并配置** 方式）。在 Grafana 管理界面中的 **Configurations > Plugins** 页面直接搜索并按照提示安装 TDengine。 -```sh -export TDENGINE_API=http://tdengine.local:6041 -# user + password -export TDENGINE_USER=user -export TDENGINE_PASSWORD=password - -# 其他环境变量： -# - 是否安装数据源，默认为 true，表示安装 -export TDENGINE_DS_ENABLED=false -# - 数据源名称，默认为 TDengine -export TDENGINE_DS_NAME=TDengine -# - 数据源所属组织 ID，默认为 1 -export GF_ORG_ID=1 -# - 数据源是否可通过管理面板编辑，默认为 0，表示不可编辑 -export TDENGINE_EDITABLE=1 -``` +![Search tdengine in grafana plugins](grafana-plugin-search-tdengine.png) + +如图示即安装完毕，按照指示 **Create a TDengine data source** 添加数据源。 + +![Install and configure Grafana data source](grafana-install-and-config.png) + +输入 TDengine 相关配置，完成数据源配置。 + +![TDengine Database Grafana plugin add data source](./grafana-data-source.png) + +配置完毕，现在可以使用 TDengine 创建 Dashboard 了。 + + + -运行安装脚本： +对于使用 Grafana 7.x 版本或使用 [Grafana Provisioning](https://grafana.com/docs/grafana/latest/administration/provisioning/) 配置的用户，可以在 Grafana 服务器上使用安装脚本自动安装插件即添加数据源 Provisioning 配置文件。 ```sh -bash -c "$(curl -fsSL https://raw.githubusercontent.com/taosdata/grafanaplugin/master/install.sh)" +bash -c "$(curl -fsSL \ + https://raw.githubusercontent.com/taosdata/grafanaplugin/master/install.sh)" -- \ + -a http://localhost:6041 \ + -u root \ + -p taosdata ``` -该脚本将自动安装 Grafana 插件并配置数据源。安装完毕后，需要重启 Grafana 服务后生效。 +安装完毕后，需要重启 Grafana 服务后方可生效。 保存该脚本并执行 `./install.sh --help` 可查看详细帮助文档。 - + 使用 [`grafana-cli` 命令行工具](https://grafana.com/docs/grafana/latest/administration/cli/) 进行插件[安装](https://grafana.com/grafana/plugins/tdengine-datasource/?tab=installation)。 @@ -113,6 +115,73 @@ GF_INSTALL_PLUGINS=tdengine-datasource ![TDengine Database Grafana plugin add data source](./add_datasource4.webp) + + + +参考 [Grafana 容器化安装说明](https://grafana.com/docs/grafana/next/setup-grafana/installation/docker/#install-plugins-in-the-docker-container)。使用如下命令启动一个容器，并自动安装 TDengine 插件： + +```bash +docker run -d \ + -p 3000:3000 \ + --name=grafana \ + -e "GF_INSTALL_PLUGINS=tdengine-datasource" \ + grafana/grafana +``` + +使用 docker-compose，配置 Grafana Provisioning 自动化配置，体验 TDengine + Grafana 组合的零配置启动： + +1. 保存该文件为 `tdengine.yml`。 + + ```yml + apiVersion: 1 + datasources: + - name: TDengine + type: tdengine-datasource + orgId: 1 + url: "$TDENGINE_API" + isDefault: true + secureJsonData: + url: "$TDENGINE_URL" + basicAuth: "$TDENGINE_BASIC_AUTH" + token: "$TDENGINE_CLOUD_TOKEN" + version: 1 + editable: true + ``` + +2. 保存该文件为 `docker-compose.yml`。 + + ```yml + version: "3.7" + + services: + tdengine: + image: tdengine/tdengine:2.6.0.2 + environment: + TAOS_FQDN: tdengine + volumes: + - tdengine-data:/var/lib/taos/ + grafana: + image: grafana/grafana:8.5.6 + volumes: + - ./tdengine.yml/:/etc/grafana/provisioning/tdengine.yml + - grafana-data:/var/lib/grafana + environment: + # install tdengine plugin at start + GF_INSTALL_PLUGINS: "tdengine-datasource" + TDENGINE_URL: "http://tdengine:6041" + #printf "$TDENGINE_USER:$TDENGINE_PASSWORD" | base64 + TDENGINE_BASIC_AUTH: "cm9vdDp0YmFzZTEyNQ==" + ports: + - 3000:3000 + volumes: + grafana-data: + tdengine-data: + ``` + +3. 使用 docker-compose 命令启动 TDengine + Grafana ：`docker-compose up -d`。 + +打开 Grafana ，现在可以添加 Dashboard 了。 + diff --git a/docs/zh/20-third-party/grafana-data-source.png b/docs/zh/20-third-party/grafana-data-source.png new file mode 100644 index 0000000000000000000000000000000000000000..989ffcca0bf5baae8798b0695e259aca35f0442a Binary files /dev/null and b/docs/zh/20-third-party/grafana-data-source.png differ diff --git a/docs/zh/20-third-party/grafana-install-and-config.png b/docs/zh/20-third-party/grafana-install-and-config.png new file mode 100644 index 0000000000000000000000000000000000000000..b918da8b2d62e694fe1797e09cf8f23f103bc97e Binary files /dev/null and b/docs/zh/20-third-party/grafana-install-and-config.png differ diff --git a/docs/zh/20-third-party/grafana-plugin-search-tdengine.png b/docs/zh/20-third-party/grafana-plugin-search-tdengine.png new file mode 100644 index 0000000000000000000000000000000000000000..cf3b66977b64f7dcd617f06024a66066cd62810e Binary files /dev/null and b/docs/zh/20-third-party/grafana-plugin-search-tdengine.png differ diff --git a/examples/c/stream_demo.c b/examples/c/stream_demo.c index ab59fa5e474cb8e6dde123b18c8cc265570aeeaa..5a141867e7b56693eece323a8e039e7afe94b1dd 100644 --- a/examples/c/stream_demo.c +++ b/examples/c/stream_demo.c @@ -32,6 +32,13 @@ int32_t init_env() { } taos_free_result(pRes); + pRes = taos_query(pConn, "create database if not exists abc2 vgroups 20"); + if (taos_errno(pRes) != 0) { + printf("error in create db, reason:%s\n", taos_errstr(pRes)); + return -1; + } + taos_free_result(pRes); + pRes = taos_query(pConn, "use abc1"); if (taos_errno(pRes) != 0) { printf("error in use db, reason:%s\n", taos_errstr(pRes)); @@ -81,9 +88,9 @@ int32_t create_stream() { /*const char* sql = "select min(k), max(k), sum(k) as sum_of_k from st1";*/ /*const char* sql = "select sum(k) from tu1 interval(10m)";*/ /*pRes = tmq_create_stream(pConn, "stream1", "out1", sql);*/ - pRes = taos_query( - pConn, - "create stream stream1 trigger max_delay 10s into outstb as select _wstartts, sum(k) from st1 interval(10m)"); + pRes = taos_query(pConn, + "create stream stream1 trigger at_once into abc2.outstb as select _wstartts, sum(k) from st1 " + "partition by tbname interval(10m) "); if (taos_errno(pRes) != 0) { printf("failed to create stream stream1, reason:%s\n", taos_errstr(pRes)); return -1; diff --git a/examples/c/tmq.c b/examples/c/tmq.c index 2e8aa21da7a2bdd83e4a995beccb99ac40228a48..7870d7d9a188030806fc113fbadcb2f76172ef66 100644 --- a/examples/c/tmq.c +++ b/examples/c/tmq.c @@ -47,7 +47,7 @@ int32_t init_env() { return -1; } - TAOS_RES* pRes = taos_query(pConn, "create database if not exists abc1 vgroups 2"); + TAOS_RES* pRes = taos_query(pConn, "create database if not exists abc1 vgroups 1"); if (taos_errno(pRes) != 0) { printf("error in create db, reason:%s\n", taos_errstr(pRes)); return -1; @@ -146,8 +146,8 @@ int32_t create_topic() { return 0; } -void tmq_commit_cb_print(tmq_t* tmq, tmq_resp_err_t resp, tmq_topic_vgroup_list_t* offsets, void* param) { - printf("commit %d tmq %p offsets %p param %p\n", resp, tmq, offsets, param); +void tmq_commit_cb_print(tmq_t* tmq, int32_t code, void* param) { + printf("commit %d tmq %p param %p\n", code, tmq, param); } tmq_t* build_consumer() { @@ -167,7 +167,7 @@ tmq_t* build_consumer() { tmq_conf_set(conf, "td.connect.user", "root"); tmq_conf_set(conf, "td.connect.pass", "taosdata"); tmq_conf_set(conf, "msg.with.table.name", "true"); - tmq_conf_set(conf, "enable.auto.commit", "false"); + tmq_conf_set(conf, "enable.auto.commit", "true"); tmq_conf_set_auto_commit_cb(conf, tmq_commit_cb_print, NULL); tmq_t* tmq = tmq_consumer_new(conf, NULL, 0); assert(tmq); @@ -183,10 +183,10 @@ tmq_list_t* build_topic_list() { } void basic_consume_loop(tmq_t* tmq, tmq_list_t* topics) { - tmq_resp_err_t err; + int32_t code; - if ((err = tmq_subscribe(tmq, topics))) { - fprintf(stderr, "%% Failed to start consuming topics: %s\n", tmq_err2str(err)); + if ((code = tmq_subscribe(tmq, topics))) { + fprintf(stderr, "%% Failed to start consuming topics: %s\n", tmq_err2str(code)); printf("subscribe err\n"); return; } @@ -201,12 +201,13 @@ void basic_consume_loop(tmq_t* tmq, tmq_list_t* topics) { taos_free_result(tmqmessage); /*} else {*/ /*break;*/ + /*tmq_commit_sync(tmq, NULL);*/ } } - err = tmq_consumer_close(tmq); - if (err) - fprintf(stderr, "%% Failed to close consumer: %s\n", tmq_err2str(err)); + code = tmq_consumer_close(tmq); + if (code) + fprintf(stderr, "%% Failed to close consumer: %s\n", tmq_err2str(code)); else fprintf(stderr, "%% Consumer closed\n"); } @@ -214,11 +215,11 @@ void basic_consume_loop(tmq_t* tmq, tmq_list_t* topics) { void sync_consume_loop(tmq_t* tmq, tmq_list_t* topics) { static const int MIN_COMMIT_COUNT = 1; - int msg_count = 0; - tmq_resp_err_t err; + int msg_count = 0; + int32_t code; - if ((err = tmq_subscribe(tmq, topics))) { - fprintf(stderr, "%% Failed to start consuming topics: %s\n", tmq_err2str(err)); + if ((code = tmq_subscribe(tmq, topics))) { + fprintf(stderr, "%% Failed to start consuming topics: %s\n", tmq_err2str(code)); return; } @@ -239,14 +240,14 @@ void sync_consume_loop(tmq_t* tmq, tmq_list_t* topics) { msg_process(tmqmessage); taos_free_result(tmqmessage); - /*tmq_commit_async(tmq, NULL, tmq_commit_cb_print, NULL);*/ + /*tmq_commit_sync(tmq, NULL);*/ /*if ((++msg_count % MIN_COMMIT_COUNT) == 0) tmq_commit(tmq, NULL, 0);*/ } } - err = tmq_consumer_close(tmq); - if (err) - fprintf(stderr, "%% Failed to close consumer: %s\n", tmq_err2str(err)); + code = tmq_consumer_close(tmq); + if (code) + fprintf(stderr, "%% Failed to close consumer: %s\n", tmq_err2str(code)); else fprintf(stderr, "%% Consumer closed\n"); } diff --git a/include/client/taos.h b/include/client/taos.h index deb4276b54356df5a125aba16b245410114921da..61538e392a0daa14b3f2def6ed74b0ee51d8d7ef 100644 --- a/include/client/taos.h +++ b/include/client/taos.h @@ -126,47 +126,47 @@ typedef struct setConfRet { char retMsg[RET_MSG_LENGTH]; } setConfRet; -DLL_EXPORT void taos_cleanup(void); -DLL_EXPORT int taos_options(TSDB_OPTION option, const void *arg, ...); -DLL_EXPORT setConfRet taos_set_config(const char *config); -DLL_EXPORT int taos_init(void); -DLL_EXPORT TAOS *taos_connect(const char *ip, const char *user, const char *pass, const char *db, uint16_t port); -DLL_EXPORT TAOS *taos_connect_auth(const char *ip, const char *user, const char *auth, const char *db, uint16_t port); -DLL_EXPORT void taos_close(TAOS *taos); - -const char *taos_data_type(int type); - -DLL_EXPORT TAOS_STMT *taos_stmt_init(TAOS *taos); -DLL_EXPORT int taos_stmt_prepare(TAOS_STMT *stmt, const char *sql, unsigned long length); -DLL_EXPORT int taos_stmt_set_tbname_tags(TAOS_STMT *stmt, const char *name, TAOS_MULTI_BIND *tags); -DLL_EXPORT int taos_stmt_set_tbname(TAOS_STMT *stmt, const char *name); -DLL_EXPORT int taos_stmt_set_tags(TAOS_STMT *stmt, TAOS_MULTI_BIND *tags); -DLL_EXPORT int taos_stmt_set_sub_tbname(TAOS_STMT *stmt, const char *name); -DLL_EXPORT int taos_stmt_get_tag_fields(TAOS_STMT *stmt, int *fieldNum, TAOS_FIELD_E **fields); -DLL_EXPORT int taos_stmt_get_col_fields(TAOS_STMT *stmt, int *fieldNum, TAOS_FIELD_E **fields); - -DLL_EXPORT int taos_stmt_is_insert(TAOS_STMT *stmt, int *insert); -DLL_EXPORT int taos_stmt_num_params(TAOS_STMT *stmt, int *nums); -DLL_EXPORT int taos_stmt_get_param(TAOS_STMT *stmt, int idx, int *type, int *bytes); -DLL_EXPORT int taos_stmt_bind_param(TAOS_STMT *stmt, TAOS_MULTI_BIND *bind); -DLL_EXPORT int taos_stmt_bind_param_batch(TAOS_STMT *stmt, TAOS_MULTI_BIND *bind); -DLL_EXPORT int taos_stmt_bind_single_param_batch(TAOS_STMT *stmt, TAOS_MULTI_BIND *bind, int colIdx); -DLL_EXPORT int taos_stmt_add_batch(TAOS_STMT *stmt); -DLL_EXPORT int taos_stmt_execute(TAOS_STMT *stmt); -DLL_EXPORT TAOS_RES *taos_stmt_use_result(TAOS_STMT *stmt); -DLL_EXPORT int taos_stmt_close(TAOS_STMT *stmt); -DLL_EXPORT char *taos_stmt_errstr(TAOS_STMT *stmt); -DLL_EXPORT int taos_stmt_affected_rows(TAOS_STMT *stmt); -DLL_EXPORT int taos_stmt_affected_rows_once(TAOS_STMT *stmt); - -DLL_EXPORT TAOS_RES *taos_query(TAOS *taos, const char *sql); - -DLL_EXPORT TAOS_ROW taos_fetch_row(TAOS_RES *res); -DLL_EXPORT int taos_result_precision(TAOS_RES *res); // get the time precision of result -DLL_EXPORT void taos_free_result(TAOS_RES *res); -DLL_EXPORT int taos_field_count(TAOS_RES *res); -DLL_EXPORT int taos_num_fields(TAOS_RES *res); -DLL_EXPORT int taos_affected_rows(TAOS_RES *res); +DLL_EXPORT void taos_cleanup(void); +DLL_EXPORT int taos_options(TSDB_OPTION option, const void *arg, ...); +DLL_EXPORT setConfRet taos_set_config(const char *config); +DLL_EXPORT int taos_init(void); +DLL_EXPORT TAOS *taos_connect(const char *ip, const char *user, const char *pass, const char *db, uint16_t port); +DLL_EXPORT TAOS *taos_connect_auth(const char *ip, const char *user, const char *auth, const char *db, uint16_t port); +DLL_EXPORT void taos_close(TAOS *taos); + +const char *taos_data_type(int type); + +DLL_EXPORT TAOS_STMT *taos_stmt_init(TAOS *taos); +DLL_EXPORT int taos_stmt_prepare(TAOS_STMT *stmt, const char *sql, unsigned long length); +DLL_EXPORT int taos_stmt_set_tbname_tags(TAOS_STMT *stmt, const char *name, TAOS_MULTI_BIND *tags); +DLL_EXPORT int taos_stmt_set_tbname(TAOS_STMT *stmt, const char *name); +DLL_EXPORT int taos_stmt_set_tags(TAOS_STMT *stmt, TAOS_MULTI_BIND *tags); +DLL_EXPORT int taos_stmt_set_sub_tbname(TAOS_STMT *stmt, const char *name); +DLL_EXPORT int taos_stmt_get_tag_fields(TAOS_STMT *stmt, int *fieldNum, TAOS_FIELD_E **fields); +DLL_EXPORT int taos_stmt_get_col_fields(TAOS_STMT *stmt, int *fieldNum, TAOS_FIELD_E **fields); + +DLL_EXPORT int taos_stmt_is_insert(TAOS_STMT *stmt, int *insert); +DLL_EXPORT int taos_stmt_num_params(TAOS_STMT *stmt, int *nums); +DLL_EXPORT int taos_stmt_get_param(TAOS_STMT *stmt, int idx, int *type, int *bytes); +DLL_EXPORT int taos_stmt_bind_param(TAOS_STMT *stmt, TAOS_MULTI_BIND *bind); +DLL_EXPORT int taos_stmt_bind_param_batch(TAOS_STMT *stmt, TAOS_MULTI_BIND *bind); +DLL_EXPORT int taos_stmt_bind_single_param_batch(TAOS_STMT *stmt, TAOS_MULTI_BIND *bind, int colIdx); +DLL_EXPORT int taos_stmt_add_batch(TAOS_STMT *stmt); +DLL_EXPORT int taos_stmt_execute(TAOS_STMT *stmt); +DLL_EXPORT TAOS_RES *taos_stmt_use_result(TAOS_STMT *stmt); +DLL_EXPORT int taos_stmt_close(TAOS_STMT *stmt); +DLL_EXPORT char *taos_stmt_errstr(TAOS_STMT *stmt); +DLL_EXPORT int taos_stmt_affected_rows(TAOS_STMT *stmt); +DLL_EXPORT int taos_stmt_affected_rows_once(TAOS_STMT *stmt); + +DLL_EXPORT TAOS_RES *taos_query(TAOS *taos, const char *sql); + +DLL_EXPORT TAOS_ROW taos_fetch_row(TAOS_RES *res); +DLL_EXPORT int taos_result_precision(TAOS_RES *res); // get the time precision of result +DLL_EXPORT void taos_free_result(TAOS_RES *res); +DLL_EXPORT int taos_field_count(TAOS_RES *res); +DLL_EXPORT int taos_num_fields(TAOS_RES *res); +DLL_EXPORT int taos_affected_rows(TAOS_RES *res); DLL_EXPORT TAOS_FIELD *taos_fetch_fields(TAOS_RES *res); DLL_EXPORT int taos_select_db(TAOS *taos, const char *db); @@ -181,8 +181,8 @@ DLL_EXPORT int *taos_get_column_data_offset(TAOS_RES *res, int columnInde DLL_EXPORT int taos_validate_sql(TAOS *taos, const char *sql); DLL_EXPORT void taos_reset_current_db(TAOS *taos); -DLL_EXPORT int *taos_fetch_lengths(TAOS_RES *res); -DLL_EXPORT TAOS_ROW *taos_result_block(TAOS_RES *res); +DLL_EXPORT int *taos_fetch_lengths(TAOS_RES *res); +DLL_EXPORT TAOS_ROW *taos_result_block(TAOS_RES *res); DLL_EXPORT const char *taos_get_server_info(TAOS *taos); DLL_EXPORT const char *taos_get_client_info(); @@ -192,8 +192,8 @@ DLL_EXPORT int taos_errno(TAOS_RES *tres); DLL_EXPORT void taos_query_a(TAOS *taos, const char *sql, __taos_async_fn_t fp, void *param); DLL_EXPORT void taos_fetch_rows_a(TAOS_RES *res, __taos_async_fn_t fp, void *param); -DLL_EXPORT void taos_fetch_raw_block_a(TAOS_RES* res, __taos_async_fn_t fp, void* param); -DLL_EXPORT const void *taos_get_raw_block(TAOS_RES* res); +DLL_EXPORT void taos_fetch_raw_block_a(TAOS_RES *res, __taos_async_fn_t fp, void *param); +DLL_EXPORT const void *taos_get_raw_block(TAOS_RES *res); // Shuduo: temporary enable for app build #if 1 @@ -209,21 +209,11 @@ DLL_EXPORT TAOS_RES *taos_schemaless_insert(TAOS *taos, char *lines[], int numLi /* --------------------------TMQ INTERFACE------------------------------- */ -enum { - TMQ_RESP_ERR__FAIL = -1, - TMQ_RESP_ERR__SUCCESS = 0, -}; - -typedef int32_t tmq_resp_err_t; - -typedef struct tmq_t tmq_t; -typedef struct tmq_topic_vgroup_t tmq_topic_vgroup_t; -typedef struct tmq_topic_vgroup_list_t tmq_topic_vgroup_list_t; - +typedef struct tmq_t tmq_t; typedef struct tmq_conf_t tmq_conf_t; typedef struct tmq_list_t tmq_list_t; -typedef void(tmq_commit_cb(tmq_t *, tmq_resp_err_t, tmq_topic_vgroup_list_t *, void *param)); +typedef void(tmq_commit_cb(tmq_t *, int32_t code, void *param)); DLL_EXPORT tmq_list_t *tmq_list_new(); DLL_EXPORT int32_t tmq_list_append(tmq_list_t *, const char *); @@ -233,25 +223,17 @@ DLL_EXPORT char **tmq_list_to_c_array(const tmq_list_t *); DLL_EXPORT tmq_t *tmq_consumer_new(tmq_conf_t *conf, char *errstr, int32_t errstrLen); -DLL_EXPORT const char *tmq_err2str(tmq_resp_err_t); +DLL_EXPORT const char *tmq_err2str(int32_t code); /* ------------------------TMQ CONSUMER INTERFACE------------------------ */ -DLL_EXPORT tmq_resp_err_t tmq_subscribe(tmq_t *tmq, const tmq_list_t *topic_list); -DLL_EXPORT tmq_resp_err_t tmq_unsubscribe(tmq_t *tmq); -DLL_EXPORT tmq_resp_err_t tmq_subscription(tmq_t *tmq, tmq_list_t **topics); - -// timeout: -1 means infinitely waiting +DLL_EXPORT int32_t tmq_subscribe(tmq_t *tmq, const tmq_list_t *topic_list); +DLL_EXPORT int32_t tmq_unsubscribe(tmq_t *tmq); +DLL_EXPORT int32_t tmq_subscription(tmq_t *tmq, tmq_list_t **topics); DLL_EXPORT TAOS_RES *tmq_consumer_poll(tmq_t *tmq, int64_t timeout); - -DLL_EXPORT tmq_resp_err_t tmq_consumer_close(tmq_t *tmq); -DLL_EXPORT tmq_resp_err_t tmq_commit_sync(tmq_t *tmq, const tmq_topic_vgroup_list_t *offsets); -DLL_EXPORT void tmq_commit_async(tmq_t *tmq, const tmq_topic_vgroup_list_t *offsets, tmq_commit_cb *cb, void *param); - -#if 0 -DLL_EXPORT tmq_resp_err_t tmq_commit(tmq_t *tmq, const tmq_topic_vgroup_list_t *offsets, int32_t async); -DLL_EXPORT tmq_resp_err_t tmq_seek(tmq_t *tmq, const tmq_topic_vgroup_t *offset); -#endif +DLL_EXPORT int32_t tmq_consumer_close(tmq_t *tmq); +DLL_EXPORT int32_t tmq_commit_sync(tmq_t *tmq, const TAOS_RES *msg); +DLL_EXPORT void tmq_commit_async(tmq_t *tmq, const TAOS_RES *msg, tmq_commit_cb *cb, void *param); /* ----------------------TMQ CONFIGURATION INTERFACE---------------------- */ @@ -275,11 +257,6 @@ DLL_EXPORT const char *tmq_get_db_name(TAOS_RES *res); DLL_EXPORT int32_t tmq_get_vgroup_id(TAOS_RES *res); DLL_EXPORT const char *tmq_get_table_name(TAOS_RES *res); -#if 0 -DLL_EXPORT int64_t tmq_get_request_offset(tmq_message_t *message); -DLL_EXPORT int64_t tmq_get_response_offset(tmq_message_t *message); -#endif - /* ------------------------------ TMQ END -------------------------------- */ #if 1 // Shuduo: temporary enable for app build diff --git a/include/common/tcommon.h b/include/common/tcommon.h index 2e646f4769607947102be1dbae767410a167a6e8..a05287761ef5dbf8d28c075bbc6cb9cac9910651 100644 --- a/include/common/tcommon.h +++ b/include/common/tcommon.h @@ -35,6 +35,7 @@ enum { TMQ_MSG_TYPE__DUMMY = 0, TMQ_MSG_TYPE__POLL_RSP, TMQ_MSG_TYPE__EP_RSP, + TMQ_MSG_TYPE__END_RSP, }; typedef enum EStreamType { diff --git a/include/common/tglobal.h b/include/common/tglobal.h index 1b44b6d7ea226c79e8adde0cbc076bb593be8958..8c03d3ff42b5b0b28f8f1958acfcd7dc8520d464 100644 --- a/include/common/tglobal.h +++ b/include/common/tglobal.h @@ -61,6 +61,7 @@ extern int32_t tsNumOfRpcThreads; extern int32_t tsNumOfCommitThreads; extern int32_t tsNumOfTaskQueueThreads; extern int32_t tsNumOfMnodeQueryThreads; +extern int32_t tsNumOfMnodeFetchThreads; extern int32_t tsNumOfMnodeReadThreads; extern int32_t tsNumOfVnodeQueryThreads; extern int32_t tsNumOfVnodeFetchThreads; diff --git a/include/common/tmsg.h b/include/common/tmsg.h index ab0203adf869c57730099a4e51a089ecca279d50..857a554f624c0143a28b2d501d2bd24904d3676c 100644 --- a/include/common/tmsg.h +++ b/include/common/tmsg.h @@ -106,6 +106,7 @@ typedef enum _mgmt_table { TSDB_MGMT_TABLE_CONNS, TSDB_MGMT_TABLE_QUERIES, TSDB_MGMT_TABLE_VNODES, + TSDB_MGMT_TABLE_APPS, TSDB_MGMT_TABLE_MAX, } EShowType; @@ -428,8 +429,10 @@ STSchema* tdGetSTSChemaFromSSChema(SSchema** pSchema, int32_t nCols); typedef struct { char name[TSDB_TABLE_FNAME_LEN]; int8_t igExists; - float xFilesFactor; - int32_t delay; + int64_t delay1; + int64_t delay2; + int64_t watermark1; + int64_t watermark2; int32_t ttl; int32_t numOfColumns; int32_t numOfTags; @@ -1314,8 +1317,6 @@ int32_t tSerializeSSetStandbyReq(void* buf, int32_t bufLen, SSetStandbyReq* pReq int32_t tDeserializeSSetStandbyReq(void* buf, int32_t bufLen, SSetStandbyReq* pReq); typedef struct { - int32_t connId; // todo remove - int32_t queryId; // todo remove char queryStrId[TSDB_QUERY_ID_LEN]; } SKillQueryReq; @@ -1323,7 +1324,7 @@ int32_t tSerializeSKillQueryReq(void* buf, int32_t bufLen, SKillQueryReq* pReq); int32_t tDeserializeSKillQueryReq(void* buf, int32_t bufLen, SKillQueryReq* pReq); typedef struct { - int32_t connId; + uint32_t connId; } SKillConnReq; int32_t tSerializeSKillConnReq(void* buf, int32_t bufLen, SKillConnReq* pReq); @@ -2010,7 +2011,6 @@ typedef struct { int64_t useconds; int64_t stime; // timestamp precision ms int64_t reqRid; - int32_t pid; bool stableQuery; char fqdn[TSDB_FQDN_LEN]; int32_t subPlanNum; @@ -2019,8 +2019,6 @@ typedef struct { typedef struct { uint32_t connId; - int32_t pid; - char app[TSDB_APP_NAME_LEN]; SArray* queryDesc; // SArray } SQueryHbReqBasic; @@ -2035,9 +2033,31 @@ typedef struct { SArray* pQnodeList; } SQueryHbRspBasic; +typedef struct SAppClusterSummary { + uint64_t numOfInsertsReq; + uint64_t numOfInsertRows; + uint64_t insertElapsedTime; + uint64_t insertBytes; // submit to tsdb since launched. + + uint64_t fetchBytes; + uint64_t queryElapsedTime; + uint64_t numOfSlowQueries; + uint64_t totalRequests; + uint64_t currentRequests; // the number of SRequestObj +} SAppClusterSummary; + +typedef struct { + int64_t appId; + int32_t pid; + char name[TSDB_APP_NAME_LEN]; + int64_t startTime; + SAppClusterSummary summary; +} SAppHbReq; + typedef struct { SClientHbKey connKey; int64_t clusterId; + SAppHbReq app; SQueryHbReqBasic* query; SHashObj* info; // hash } SClientHbReq; diff --git a/include/common/ttokendef.h b/include/common/ttokendef.h index 061c734484b22bb5cb0eb3ea349fcb4766e3e3ae..1edfcbabadf48b98324b94c1e15718b1e7161bd6 100644 --- a/include/common/ttokendef.h +++ b/include/common/ttokendef.h @@ -127,134 +127,133 @@ #define TK_BLOB 109 #define TK_VARBINARY 110 #define TK_DECIMAL 111 -#define TK_FILE_FACTOR 112 -#define TK_NK_FLOAT 113 +#define TK_MAX_DELAY 112 +#define TK_WATERMARK 113 #define TK_ROLLUP 114 #define TK_TTL 115 #define TK_SMA 116 -#define TK_SHOW 117 -#define TK_DATABASES 118 -#define TK_TABLES 119 -#define TK_STABLES 120 -#define TK_MNODES 121 -#define TK_MODULES 122 -#define TK_QNODES 123 -#define TK_FUNCTIONS 124 -#define TK_INDEXES 125 -#define TK_ACCOUNTS 126 -#define TK_APPS 127 -#define TK_CONNECTIONS 128 -#define TK_LICENCE 129 -#define TK_GRANTS 130 -#define TK_QUERIES 131 -#define TK_SCORES 132 -#define TK_TOPICS 133 -#define TK_VARIABLES 134 -#define TK_BNODES 135 -#define TK_SNODES 136 -#define TK_CLUSTER 137 -#define TK_TRANSACTIONS 138 -#define TK_LIKE 139 -#define TK_INDEX 140 -#define TK_FULLTEXT 141 -#define TK_FUNCTION 142 -#define TK_INTERVAL 143 -#define TK_TOPIC 144 -#define TK_AS 145 -#define TK_CONSUMER 146 -#define TK_GROUP 147 -#define TK_DESC 148 -#define TK_DESCRIBE 149 -#define TK_RESET 150 -#define TK_QUERY 151 -#define TK_CACHE 152 -#define TK_EXPLAIN 153 -#define TK_ANALYZE 154 -#define TK_VERBOSE 155 -#define TK_NK_BOOL 156 -#define TK_RATIO 157 -#define TK_COMPACT 158 -#define TK_VNODES 159 -#define TK_IN 160 -#define TK_OUTPUTTYPE 161 -#define TK_AGGREGATE 162 -#define TK_BUFSIZE 163 -#define TK_STREAM 164 -#define TK_INTO 165 -#define TK_TRIGGER 166 -#define TK_AT_ONCE 167 -#define TK_WINDOW_CLOSE 168 -#define TK_MAX_DELAY 169 -#define TK_WATERMARK 170 -#define TK_KILL 171 -#define TK_CONNECTION 172 -#define TK_TRANSACTION 173 -#define TK_BALANCE 174 -#define TK_VGROUP 175 -#define TK_MERGE 176 -#define TK_REDISTRIBUTE 177 -#define TK_SPLIT 178 -#define TK_SYNCDB 179 -#define TK_DELETE 180 -#define TK_NULL 181 -#define TK_NK_QUESTION 182 -#define TK_NK_ARROW 183 -#define TK_ROWTS 184 -#define TK_TBNAME 185 -#define TK_QSTARTTS 186 -#define TK_QENDTS 187 -#define TK_WSTARTTS 188 -#define TK_WENDTS 189 -#define TK_WDURATION 190 -#define TK_CAST 191 -#define TK_NOW 192 -#define TK_TODAY 193 -#define TK_TIMEZONE 194 -#define TK_COUNT 195 -#define TK_FIRST 196 -#define TK_LAST 197 -#define TK_LAST_ROW 198 -#define TK_BETWEEN 199 -#define TK_IS 200 -#define TK_NK_LT 201 -#define TK_NK_GT 202 -#define TK_NK_LE 203 -#define TK_NK_GE 204 -#define TK_NK_NE 205 -#define TK_MATCH 206 -#define TK_NMATCH 207 -#define TK_CONTAINS 208 -#define TK_JOIN 209 -#define TK_INNER 210 -#define TK_SELECT 211 -#define TK_DISTINCT 212 -#define TK_WHERE 213 -#define TK_PARTITION 214 -#define TK_BY 215 -#define TK_SESSION 216 -#define TK_STATE_WINDOW 217 -#define TK_SLIDING 218 -#define TK_FILL 219 -#define TK_VALUE 220 -#define TK_NONE 221 -#define TK_PREV 222 -#define TK_LINEAR 223 -#define TK_NEXT 224 -#define TK_HAVING 225 -#define TK_ORDER 226 -#define TK_SLIMIT 227 -#define TK_SOFFSET 228 -#define TK_LIMIT 229 -#define TK_OFFSET 230 -#define TK_ASC 231 -#define TK_NULLS 232 -#define TK_ID 233 -#define TK_NK_BITNOT 234 -#define TK_INSERT 235 -#define TK_VALUES 236 -#define TK_IMPORT 237 -#define TK_NK_SEMI 238 -#define TK_FILE 239 +#define TK_FIRST 117 +#define TK_LAST 118 +#define TK_SHOW 119 +#define TK_DATABASES 120 +#define TK_TABLES 121 +#define TK_STABLES 122 +#define TK_MNODES 123 +#define TK_MODULES 124 +#define TK_QNODES 125 +#define TK_FUNCTIONS 126 +#define TK_INDEXES 127 +#define TK_ACCOUNTS 128 +#define TK_APPS 129 +#define TK_CONNECTIONS 130 +#define TK_LICENCE 131 +#define TK_GRANTS 132 +#define TK_QUERIES 133 +#define TK_SCORES 134 +#define TK_TOPICS 135 +#define TK_VARIABLES 136 +#define TK_BNODES 137 +#define TK_SNODES 138 +#define TK_CLUSTER 139 +#define TK_TRANSACTIONS 140 +#define TK_LIKE 141 +#define TK_INDEX 142 +#define TK_FULLTEXT 143 +#define TK_FUNCTION 144 +#define TK_INTERVAL 145 +#define TK_TOPIC 146 +#define TK_AS 147 +#define TK_CONSUMER 148 +#define TK_GROUP 149 +#define TK_DESC 150 +#define TK_DESCRIBE 151 +#define TK_RESET 152 +#define TK_QUERY 153 +#define TK_CACHE 154 +#define TK_EXPLAIN 155 +#define TK_ANALYZE 156 +#define TK_VERBOSE 157 +#define TK_NK_BOOL 158 +#define TK_RATIO 159 +#define TK_NK_FLOAT 160 +#define TK_COMPACT 161 +#define TK_VNODES 162 +#define TK_IN 163 +#define TK_OUTPUTTYPE 164 +#define TK_AGGREGATE 165 +#define TK_BUFSIZE 166 +#define TK_STREAM 167 +#define TK_INTO 168 +#define TK_TRIGGER 169 +#define TK_AT_ONCE 170 +#define TK_WINDOW_CLOSE 171 +#define TK_KILL 172 +#define TK_CONNECTION 173 +#define TK_TRANSACTION 174 +#define TK_BALANCE 175 +#define TK_VGROUP 176 +#define TK_MERGE 177 +#define TK_REDISTRIBUTE 178 +#define TK_SPLIT 179 +#define TK_SYNCDB 180 +#define TK_DELETE 181 +#define TK_NULL 182 +#define TK_NK_QUESTION 183 +#define TK_NK_ARROW 184 +#define TK_ROWTS 185 +#define TK_TBNAME 186 +#define TK_QSTARTTS 187 +#define TK_QENDTS 188 +#define TK_WSTARTTS 189 +#define TK_WENDTS 190 +#define TK_WDURATION 191 +#define TK_CAST 192 +#define TK_NOW 193 +#define TK_TODAY 194 +#define TK_TIMEZONE 195 +#define TK_COUNT 196 +#define TK_LAST_ROW 197 +#define TK_BETWEEN 198 +#define TK_IS 199 +#define TK_NK_LT 200 +#define TK_NK_GT 201 +#define TK_NK_LE 202 +#define TK_NK_GE 203 +#define TK_NK_NE 204 +#define TK_MATCH 205 +#define TK_NMATCH 206 +#define TK_CONTAINS 207 +#define TK_JOIN 208 +#define TK_INNER 209 +#define TK_SELECT 210 +#define TK_DISTINCT 211 +#define TK_WHERE 212 +#define TK_PARTITION 213 +#define TK_BY 214 +#define TK_SESSION 215 +#define TK_STATE_WINDOW 216 +#define TK_SLIDING 217 +#define TK_FILL 218 +#define TK_VALUE 219 +#define TK_NONE 220 +#define TK_PREV 221 +#define TK_LINEAR 222 +#define TK_NEXT 223 +#define TK_HAVING 224 +#define TK_ORDER 225 +#define TK_SLIMIT 226 +#define TK_SOFFSET 227 +#define TK_LIMIT 228 +#define TK_OFFSET 229 +#define TK_ASC 230 +#define TK_NULLS 231 +#define TK_ID 232 +#define TK_NK_BITNOT 233 +#define TK_INSERT 234 +#define TK_VALUES 235 +#define TK_IMPORT 236 +#define TK_NK_SEMI 237 +#define TK_FILE 238 #define TK_NK_SPACE 300 #define TK_NK_COMMENT 301 diff --git a/include/libs/catalog/catalog.h b/include/libs/catalog/catalog.h index f3600a509e8be0946b5845634b13c9e6a4a9ed98..c0116051af2f2b3dc6753c96b167427a931980d1 100644 --- a/include/libs/catalog/catalog.h +++ b/include/libs/catalog/catalog.h @@ -292,6 +292,8 @@ int32_t catalogUpdateVgEpSet(SCatalog* pCtg, const char* dbFName, int32_t vgId, int32_t ctgdLaunchAsyncCall(SCatalog* pCtg, SRequestConnInfo* pConn, uint64_t reqId, bool forceUpdate); +int32_t catalogClearCache(void); + /** * Destroy catalog and relase all resources */ diff --git a/include/libs/executor/executor.h b/include/libs/executor/executor.h index fdedb947d7a9eabaaa6983cb8c0e54b1eb6a8027..083f6ae1b07d4db9e7f4d339b63b8b89c0a8becc 100644 --- a/include/libs/executor/executor.h +++ b/include/libs/executor/executor.h @@ -159,11 +159,14 @@ int64_t qGetQueriedTableUid(qTaskInfo_t tinfo); */ int32_t qGetQualifiedTableIdList(void* pTableList, const char* tagCond, int32_t tagCondLen, SArray* pTableIdList); - void qProcessFetchRsp(void* parent, struct SRpcMsg* pMsg, struct SEpSet* pEpSet); int32_t qGetExplainExecInfo(qTaskInfo_t tinfo, int32_t* resNum, SExplainExecInfo** pRes); +int32_t qSerializeTaskStatus(qTaskInfo_t tinfo, char** pOutput, int32_t* len); + +int32_t qDeserializeTaskStatus(qTaskInfo_t tinfo, const char* pInput, int32_t len); + #ifdef __cplusplus } #endif diff --git a/include/libs/function/function.h b/include/libs/function/function.h index 54762a0f17ba45929414adf99f1226d25cac1242..0d8ae289cc4def348556adc5549e948cff7fcda6 100644 --- a/include/libs/function/function.h +++ b/include/libs/function/function.h @@ -158,10 +158,10 @@ typedef struct tExprNode { int32_t nodeType; union { struct {// function node - char functionName[FUNCTIONS_NAME_MAX_LENGTH]; // todo refactor - int32_t functionId; - int32_t num; - struct SFunctionNode *pFunctNode; + char functionName[FUNCTIONS_NAME_MAX_LENGTH]; // todo refactor + int32_t functionId; + int32_t num; + struct SFunctionNode *pFunctNode; } _function; struct { diff --git a/include/libs/function/functionMgt.h b/include/libs/function/functionMgt.h index dcc0a2ba4b5e90762d982532525d30a4ccdeb434..8e003eea7fcca60f5e4e8448f087acce72857852 100644 --- a/include/libs/function/functionMgt.h +++ b/include/libs/function/functionMgt.h @@ -185,6 +185,8 @@ bool fmIsUserDefinedFunc(int32_t funcId); bool fmIsDistExecFunc(int32_t funcId); bool fmIsForbidFillFunc(int32_t funcId); bool fmIsForbidStreamFunc(int32_t funcId); +bool fmIsForbidWindowFunc(int32_t funcId); +bool fmIsForbidGroupByFunc(int32_t funcId); bool fmIsIntervalInterpoFunc(int32_t funcId); int32_t fmGetDistMethod(const SFunctionNode* pFunc, SFunctionNode** pPartialFunc, SFunctionNode** pMergeFunc); diff --git a/include/libs/nodes/cmdnodes.h b/include/libs/nodes/cmdnodes.h index 1282836425e220f67e904089b107bb8853f2f890..087d252ed733ca400718e8eb6b8433edcf566b8a 100644 --- a/include/libs/nodes/cmdnodes.h +++ b/include/libs/nodes/cmdnodes.h @@ -88,8 +88,12 @@ typedef struct SAlterDatabaseStmt { typedef struct STableOptions { ENodeType type; char comment[TSDB_TB_COMMENT_LEN]; - double filesFactor; - int32_t delay; + SNodeList* pMaxDelay; + int64_t maxDelay1; + int64_t maxDelay2; + SNodeList* pWatermark; + int64_t watermark1; + int64_t watermark2; SNodeList* pRollupFuncs; int32_t ttl; SNodeList* pSma; @@ -204,11 +208,18 @@ typedef struct SShowStmt { SNode* pTbNamePattern; // SValueNode } SShowStmt; -typedef struct SShowCreatStmt { +typedef struct SShowCreateDatabaseStmt { ENodeType type; char dbName[TSDB_DB_NAME_LEN]; - char tableName[TSDB_TABLE_NAME_LEN]; -} SShowCreatStmt; + void* pCfg; // SDbCfgInfo +} SShowCreateDatabaseStmt; + +typedef struct SShowCreateTableStmt { + ENodeType type; + char dbName[TSDB_DB_NAME_LEN]; + char tableName[TSDB_TABLE_NAME_LEN]; + STableMeta* pMeta; +} SShowCreateTableStmt; typedef enum EIndexType { INDEX_TYPE_SMA = 1, INDEX_TYPE_FULLTEXT } EIndexType; diff --git a/include/libs/nodes/nodes.h b/include/libs/nodes/nodes.h index 3620f454083eba87cfcf92aaf17eecd3e2242c33..08190d978eca06a0938d6935d15161cdf18e5cce 100644 --- a/include/libs/nodes/nodes.h +++ b/include/libs/nodes/nodes.h @@ -102,6 +102,7 @@ typedef enum ENodeType { QUERY_NODE_EXPLAIN_OPTIONS, QUERY_NODE_STREAM_OPTIONS, QUERY_NODE_LEFT_VALUE, + QUERY_NODE_COLUMN_REF, // Statement nodes are used in parser and planner module. QUERY_NODE_SET_OPERATOR, @@ -227,6 +228,7 @@ typedef enum ENodeType { QUERY_NODE_PHYSICAL_PLAN_FILL, QUERY_NODE_PHYSICAL_PLAN_MERGE_SESSION, QUERY_NODE_PHYSICAL_PLAN_STREAM_SESSION, + QUERY_NODE_PHYSICAL_PLAN_STREAM_SEMI_SESSION, QUERY_NODE_PHYSICAL_PLAN_STREAM_FINAL_SESSION, QUERY_NODE_PHYSICAL_PLAN_MERGE_STATE, QUERY_NODE_PHYSICAL_PLAN_STREAM_STATE, diff --git a/include/libs/nodes/plannodes.h b/include/libs/nodes/plannodes.h index 749d58b224bc261b64c7ae8fae07ec150dfcf657..a9002b5d19bf834764b2b012200b637fb3d3e856 100644 --- a/include/libs/nodes/plannodes.h +++ b/include/libs/nodes/plannodes.h @@ -130,13 +130,17 @@ typedef struct SMergeLogicNode { typedef enum EWindowType { WINDOW_TYPE_INTERVAL = 1, WINDOW_TYPE_SESSION, WINDOW_TYPE_STATE } EWindowType; -typedef enum EIntervalAlgorithm { +typedef enum EWindowAlgorithm { INTERVAL_ALGO_HASH = 1, INTERVAL_ALGO_MERGE, INTERVAL_ALGO_STREAM_FINAL, INTERVAL_ALGO_STREAM_SEMI, INTERVAL_ALGO_STREAM_SINGLE, -} EIntervalAlgorithm; + SESSION_ALGO_STREAM_SEMI, + SESSION_ALGO_STREAM_FINAL, + SESSION_ALGO_STREAM_SINGLE, + SESSION_ALGO_MERGE, +} EWindowAlgorithm; typedef struct SWindowLogicNode { SLogicNode node; @@ -153,7 +157,7 @@ typedef struct SWindowLogicNode { int8_t triggerType; int64_t watermark; double filesFactor; - EIntervalAlgorithm intervalAlgo; + EWindowAlgorithm windowAlgo; } SWindowLogicNode; typedef struct SFillLogicNode { @@ -371,6 +375,8 @@ typedef struct SSessionWinodwPhysiNode { } SSessionWinodwPhysiNode; typedef SSessionWinodwPhysiNode SStreamSessionWinodwPhysiNode; +typedef SSessionWinodwPhysiNode SStreamSemiSessionWinodwPhysiNode; +typedef SSessionWinodwPhysiNode SStreamFinalSessionWinodwPhysiNode; typedef struct SStateWinodwPhysiNode { SWinodwPhysiNode window; diff --git a/include/libs/nodes/querynodes.h b/include/libs/nodes/querynodes.h index 989dcc94929674988ae4c5240e99b2b762912e83..3a0d3178a3330e98a06f248d5e1e1bb5270d36d2 100644 --- a/include/libs/nodes/querynodes.h +++ b/include/libs/nodes/querynodes.h @@ -65,11 +65,16 @@ typedef struct SColumnNode { char tableName[TSDB_TABLE_NAME_LEN]; char tableAlias[TSDB_TABLE_NAME_LEN]; char colName[TSDB_COL_NAME_LEN]; - SNode* pProjectRef; - int16_t dataBlockId; - int16_t slotId; + // SNode* pProjectRef; + int16_t dataBlockId; + int16_t slotId; } SColumnNode; +typedef struct SColumnRefNode { + ENodeType type; + char colName[TSDB_COL_NAME_LEN]; +} SColumnRefNode; + typedef struct STargetNode { ENodeType type; int16_t dataBlockId; @@ -247,6 +252,8 @@ typedef struct SSelectStmt { bool hasIndefiniteRowsFunc; bool hasSelectFunc; bool hasSelectValFunc; + bool hasUniqueFunc; + bool hasTailFunc; } SSelectStmt; typedef enum ESetOperatorType { SET_OP_TYPE_UNION_ALL = 1, SET_OP_TYPE_UNION } ESetOperatorType; @@ -374,6 +381,9 @@ bool nodesIsComparisonOp(const SOperatorNode* pOp); bool nodesIsJsonOp(const SOperatorNode* pOp); bool nodesIsRegularOp(const SOperatorNode* pOp); +bool nodesExprHasColumn(SNode* pNode); +bool nodesExprsHasColumn(SNodeList* pList); + void* nodesGetValueFromNode(SValueNode* pNode); int32_t nodesSetValueNodeValue(SValueNode* pNode, void* value); char* nodesGetStrValueFromNode(SValueNode* pNode); diff --git a/include/libs/qcom/query.h b/include/libs/qcom/query.h index 89eb21556afaa4af30ba0d7636b3c9402d9342a1..b6a3a08f7e7f04c60908acc60be509b9d9262fa3 100644 --- a/include/libs/qcom/query.h +++ b/include/libs/qcom/query.h @@ -33,8 +33,6 @@ typedef enum { JOB_TASK_STATUS_PARTIAL_SUCCEED, JOB_TASK_STATUS_SUCCEED, JOB_TASK_STATUS_FAILED, - JOB_TASK_STATUS_CANCELLING, - JOB_TASK_STATUS_CANCELLED, JOB_TASK_STATUS_DROPPING, } EJobTaskType; diff --git a/include/libs/scheduler/scheduler.h b/include/libs/scheduler/scheduler.h index 8087a6d33cd14e818355651b28b940ede4a0ef37..ecb21335b9cf9edb566c1861042fcaffd6da13c4 100644 --- a/include/libs/scheduler/scheduler.h +++ b/include/libs/scheduler/scheduler.h @@ -73,13 +73,14 @@ typedef void (*schedulerExecCallback)(SQueryResult* pResult, void* param, int32_ typedef void (*schedulerFetchCallback)(void* pResult, void* param, int32_t code); typedef struct SSchedulerReq { - SRequestConnInfo *pConn; - SArray *pNodeList; - SQueryPlan *pDag; - const char *sql; - int64_t startTs; + bool *reqKilled; + SRequestConnInfo *pConn; + SArray *pNodeList; + SQueryPlan *pDag; + const char *sql; + int64_t startTs; schedulerExecCallback fp; - void* cbParam; + void* cbParam; } SSchedulerReq; @@ -127,7 +128,7 @@ void schedulerStopQueryHb(void *pTrans); * Free the query job * @param pJob */ -void schedulerFreeJob(int64_t job); +void schedulerFreeJob(int64_t job, int32_t errCode); void schedulerDestroy(void); diff --git a/include/libs/sync/sync.h b/include/libs/sync/sync.h index fc9d419b1f901a48edfae1aef8acc039bbadb57b..e8c4faa240bbcfc546c5a00fa58497616aaccae0 100644 --- a/include/libs/sync/sync.h +++ b/include/libs/sync/sync.h @@ -62,6 +62,7 @@ typedef struct SSyncCfg { typedef struct SFsmCbMeta { SyncIndex index; + SyncIndex lastConfigIndex; bool isWeak; int32_t code; ESyncState state; @@ -75,6 +76,7 @@ typedef struct SReConfigCbMeta { int32_t code; SyncIndex index; SyncTerm term; + SyncIndex lastConfigIndex; SyncTerm currentTerm; SSyncCfg oldCfg; SSyncCfg newCfg; @@ -192,6 +194,7 @@ bool syncEnvIsStart(); const char* syncStr(ESyncState state); bool syncIsRestoreFinish(int64_t rid); int32_t syncGetSnapshotMeta(int64_t rid, struct SSnapshotMeta* sMeta); +int32_t syncGetSnapshotMetaByIndex(int64_t rid, SyncIndex snapshotIndex, struct SSnapshotMeta* sMeta); int32_t syncReconfig(int64_t rid, const SSyncCfg* pNewCfg); diff --git a/include/libs/transport/trpc.h b/include/libs/transport/trpc.h index 6ab962e13f77ba0b99bcf16f21b0fe78f917743c..77de5f47562897815e6cd8633c09345dee82794a 100644 --- a/include/libs/transport/trpc.h +++ b/include/libs/transport/trpc.h @@ -49,7 +49,6 @@ typedef struct SRpcHandleInfo { STraceId traceId; // int64_t traceId; - SRpcConnInfo connInfo; // app info void *ahandle; // app handle set by client void *wrapper; // wrapper handle @@ -58,6 +57,9 @@ typedef struct SRpcHandleInfo { // resp info void * rsp; int32_t rspLen; + + // conn info + SRpcConnInfo conn; } SRpcHandleInfo; typedef struct SRpcMsg { @@ -66,7 +68,6 @@ typedef struct SRpcMsg { int32_t contLen; int32_t code; SRpcHandleInfo info; - SRpcConnInfo conn; } SRpcMsg; typedef void (*RpcCfp)(void *parent, SRpcMsg *, SEpSet *rf); diff --git a/include/os/osTime.h b/include/os/osTime.h index b9e407cbcf0f664236f87dc01e7a1279e2a39150..949c15ed0d7c7a4e4c317edaace21f29a9eaab36 100644 --- a/include/os/osTime.h +++ b/include/os/osTime.h @@ -23,22 +23,22 @@ extern "C" { // If the error is in a third-party library, place this header file under the third-party library header file. // When you want to use this feature, you should find or add the same function in the following section. #ifndef ALLOW_FORBID_FUNC - #define strptime STRPTIME_FUNC_TAOS_FORBID - #define gettimeofday GETTIMEOFDAY_FUNC_TAOS_FORBID - #define localtime LOCALTIME_FUNC_TAOS_FORBID - #define localtime_s LOCALTIMES_FUNC_TAOS_FORBID - #define localtime_r LOCALTIMER_FUNC_TAOS_FORBID - #define time TIME_FUNC_TAOS_FORBID - #define mktime MKTIME_FUNC_TAOS_FORBID +#define strptime STRPTIME_FUNC_TAOS_FORBID +#define gettimeofday GETTIMEOFDAY_FUNC_TAOS_FORBID +#define localtime LOCALTIME_FUNC_TAOS_FORBID +#define localtime_s LOCALTIMES_FUNC_TAOS_FORBID +#define localtime_r LOCALTIMER_FUNC_TAOS_FORBID +#define time TIME_FUNC_TAOS_FORBID +#define mktime MKTIME_FUNC_TAOS_FORBID #endif #ifdef WINDOWS - #define CLOCK_REALTIME 0 +#define CLOCK_REALTIME 0 - #define MILLISECOND_PER_SECOND (1000i64) +#define MILLISECOND_PER_SECOND (1000i64) #else - #define MILLISECOND_PER_SECOND ((int64_t)1000L) +#define MILLISECOND_PER_SECOND ((int64_t)1000L) #endif #define MILLISECOND_PER_MINUTE (MILLISECOND_PER_SECOND * 60) @@ -82,13 +82,13 @@ static FORCE_INLINE int64_t taosGetTimestampNs() { return (int64_t)systemTime.tv_sec * 1000000000L + (int64_t)systemTime.tv_nsec; } -char *taosStrpTime(const char *buf, const char *fmt, struct tm *tm); +char * taosStrpTime(const char *buf, const char *fmt, struct tm *tm); struct tm *taosLocalTime(const time_t *timep, struct tm *result); -time_t taosTime(time_t *t); -time_t taosMktime(struct tm *timep); +time_t taosTime(time_t *t); +time_t taosMktime(struct tm *timep); #ifdef __cplusplus } #endif -#endif /*_TD_OS_TIME_H_*/ +#endif /*_TD_OS_TIME_H_*/ diff --git a/include/util/taoserror.h b/include/util/taoserror.h index fcbe5cd8782a77d27afc01644967caece454d52c..280af6adb5e6475307f93b23231b67f803034e1f 100644 --- a/include/util/taoserror.h +++ b/include/util/taoserror.h @@ -127,6 +127,7 @@ int32_t* taosGetErrno(); #define TSDB_CODE_TSC_STMT_API_ERROR TAOS_DEF_ERROR_CODE(0, 0X0225) #define TSDB_CODE_TSC_STMT_TBNAME_ERROR TAOS_DEF_ERROR_CODE(0, 0X0226) #define TSDB_CODE_TSC_STMT_CLAUSE_ERROR TAOS_DEF_ERROR_CODE(0, 0X0227) +#define TSDB_CODE_TSC_QUERY_KILLED TAOS_DEF_ERROR_CODE(0, 0X0228) // mnode-common #define TSDB_CODE_MND_APP_ERROR TAOS_DEF_ERROR_CODE(0, 0x0300) @@ -429,7 +430,8 @@ int32_t* taosGetErrno(); #define TSDB_CODE_TQ_META_KEY_NOT_IN_TXN TAOS_DEF_ERROR_CODE(0, 0x0A09) #define TSDB_CODE_TQ_META_KEY_DUP_IN_TXN TAOS_DEF_ERROR_CODE(0, 0x0A0A) #define TSDB_CODE_TQ_GROUP_NOT_SET TAOS_DEF_ERROR_CODE(0, 0x0A0B) -#define TSDB_CODE_TQ_TABLE_SCHEMA_NOT_FOUND TAOS_DEF_ERROR_CODE(0, 0x0A0B) +#define TSDB_CODE_TQ_TABLE_SCHEMA_NOT_FOUND TAOS_DEF_ERROR_CODE(0, 0x0A0C) +#define TSDB_CODE_TQ_NO_COMMITTED_OFFSET TAOS_DEF_ERROR_CODE(0, 0x0A0D) // wal #define TSDB_CODE_WAL_APP_ERROR TAOS_DEF_ERROR_CODE(0, 0x1000) @@ -439,108 +441,6 @@ int32_t* taosGetErrno(); #define TSDB_CODE_WAL_OUT_OF_MEMORY TAOS_DEF_ERROR_CODE(0, 0x1004) #define TSDB_CODE_WAL_LOG_NOT_EXIST TAOS_DEF_ERROR_CODE(0, 0x1005) -// http -#define TSDB_CODE_HTTP_SERVER_OFFLINE TAOS_DEF_ERROR_CODE(0, 0x1100) //"http server is not online" -#define TSDB_CODE_HTTP_UNSUPPORT_URL TAOS_DEF_ERROR_CODE(0, 0x1101) //"url is not support" -#define TSDB_CODE_HTTP_INVALID_URL TAOS_DEF_ERROR_CODE(0, 0x1102) //invalid url format" -#define TSDB_CODE_HTTP_NO_ENOUGH_MEMORY TAOS_DEF_ERROR_CODE(0, 0x1103) //"no enough memory" -#define TSDB_CODE_HTTP_REQUSET_TOO_BIG TAOS_DEF_ERROR_CODE(0, 0x1104) //"request size is too big" -#define TSDB_CODE_HTTP_NO_AUTH_INFO TAOS_DEF_ERROR_CODE(0, 0x1105) //"no auth info input" -#define TSDB_CODE_HTTP_NO_MSG_INPUT TAOS_DEF_ERROR_CODE(0, 0x1106) //"request is empty" -#define TSDB_CODE_HTTP_NO_SQL_INPUT TAOS_DEF_ERROR_CODE(0, 0x1107) //"no sql input" -#define TSDB_CODE_HTTP_NO_EXEC_USEDB TAOS_DEF_ERROR_CODE(0, 0x1108) //"no need to execute use db cmd" -#define TSDB_CODE_HTTP_SESSION_FULL TAOS_DEF_ERROR_CODE(0, 0x1109) //"session list was full" -#define TSDB_CODE_HTTP_GEN_TAOSD_TOKEN_ERR TAOS_DEF_ERROR_CODE(0, 0x110A) //"generate taosd token error" -#define TSDB_CODE_HTTP_INVALID_MULTI_REQUEST TAOS_DEF_ERROR_CODE(0, 0x110B) //"size of multi request is 0" -#define TSDB_CODE_HTTP_CREATE_GZIP_FAILED TAOS_DEF_ERROR_CODE(0, 0x110C) //"failed to create gzip" -#define TSDB_CODE_HTTP_FINISH_GZIP_FAILED TAOS_DEF_ERROR_CODE(0, 0x110D) //"failed to finish gzip" -#define TSDB_CODE_HTTP_LOGIN_FAILED TAOS_DEF_ERROR_CODE(0, 0x110E) //"failed to login" - -#define TSDB_CODE_HTTP_INVALID_VERSION TAOS_DEF_ERROR_CODE(0, 0x1120) //"invalid http version" -#define TSDB_CODE_HTTP_INVALID_CONTENT_LENGTH TAOS_DEF_ERROR_CODE(0, 0x1121) //"invalid content length" -#define TSDB_CODE_HTTP_INVALID_AUTH_TYPE TAOS_DEF_ERROR_CODE(0, 0x1122) //"invalid type of Authorization" -#define TSDB_CODE_HTTP_INVALID_AUTH_FORMAT TAOS_DEF_ERROR_CODE(0, 0x1123) //"invalid format of Authorization" -#define TSDB_CODE_HTTP_INVALID_BASIC_AUTH TAOS_DEF_ERROR_CODE(0, 0x1124) //"invalid basic Authorization" -#define TSDB_CODE_HTTP_INVALID_TAOSD_AUTH TAOS_DEF_ERROR_CODE(0, 0x1125) //"invalid taosd Authorization" -#define TSDB_CODE_HTTP_PARSE_METHOD_FAILED TAOS_DEF_ERROR_CODE(0, 0x1126) //"failed to parse method" -#define TSDB_CODE_HTTP_PARSE_TARGET_FAILED TAOS_DEF_ERROR_CODE(0, 0x1127) //"failed to parse target" -#define TSDB_CODE_HTTP_PARSE_VERSION_FAILED TAOS_DEF_ERROR_CODE(0, 0x1128) //"failed to parse http version" -#define TSDB_CODE_HTTP_PARSE_SP_FAILED TAOS_DEF_ERROR_CODE(0, 0x1129) //"failed to parse sp" -#define TSDB_CODE_HTTP_PARSE_STATUS_FAILED TAOS_DEF_ERROR_CODE(0, 0x112A) //"failed to parse status" -#define TSDB_CODE_HTTP_PARSE_PHRASE_FAILED TAOS_DEF_ERROR_CODE(0, 0x112B) //"failed to parse phrase" -#define TSDB_CODE_HTTP_PARSE_CRLF_FAILED TAOS_DEF_ERROR_CODE(0, 0x112C) //"failed to parse crlf" -#define TSDB_CODE_HTTP_PARSE_HEADER_FAILED TAOS_DEF_ERROR_CODE(0, 0x112D) //"failed to parse header" -#define TSDB_CODE_HTTP_PARSE_HEADER_KEY_FAILED TAOS_DEF_ERROR_CODE(0, 0x112E) //"failed to parse header key" -#define TSDB_CODE_HTTP_PARSE_HEADER_VAL_FAILED TAOS_DEF_ERROR_CODE(0, 0x112F) //"failed to parse header val" -#define TSDB_CODE_HTTP_PARSE_CHUNK_SIZE_FAILED TAOS_DEF_ERROR_CODE(0, 0x1130) //"failed to parse chunk size" -#define TSDB_CODE_HTTP_PARSE_CHUNK_FAILED TAOS_DEF_ERROR_CODE(0, 0x1131) //"failed to parse chunk" -#define TSDB_CODE_HTTP_PARSE_END_FAILED TAOS_DEF_ERROR_CODE(0, 0x1132) //"failed to parse end section" -#define TSDB_CODE_HTTP_PARSE_INVALID_STATE TAOS_DEF_ERROR_CODE(0, 0x1134) //"invalid parse state" -#define TSDB_CODE_HTTP_PARSE_ERROR_STATE TAOS_DEF_ERROR_CODE(0, 0x1135) //"failed to parse error section" - -#define TSDB_CODE_HTTP_GC_QUERY_NULL TAOS_DEF_ERROR_CODE(0, 0x1150) //"query size is 0" -#define TSDB_CODE_HTTP_GC_QUERY_SIZE TAOS_DEF_ERROR_CODE(0, 0x1151) //"query size can not more than 100" -#define TSDB_CODE_HTTP_GC_REQ_PARSE_ERROR TAOS_DEF_ERROR_CODE(0, 0x1152) //"parse grafana json error" - -#define TSDB_CODE_HTTP_TG_DB_NOT_INPUT TAOS_DEF_ERROR_CODE(0, 0x1160) //"database name can not be null" -#define TSDB_CODE_HTTP_TG_DB_TOO_LONG TAOS_DEF_ERROR_CODE(0, 0x1161) //"database name too long" -#define TSDB_CODE_HTTP_TG_INVALID_JSON TAOS_DEF_ERROR_CODE(0, 0x1162) //"invalid telegraf json fromat" -#define TSDB_CODE_HTTP_TG_METRICS_NULL TAOS_DEF_ERROR_CODE(0, 0x1163) //"metrics size is 0" -#define TSDB_CODE_HTTP_TG_METRICS_SIZE TAOS_DEF_ERROR_CODE(0, 0x1164) //"metrics size can not more than 1K" -#define TSDB_CODE_HTTP_TG_METRIC_NULL TAOS_DEF_ERROR_CODE(0, 0x1165) //"metric name not find" -#define TSDB_CODE_HTTP_TG_METRIC_TYPE TAOS_DEF_ERROR_CODE(0, 0x1166) //"metric name type should be string" -#define TSDB_CODE_HTTP_TG_METRIC_NAME_NULL TAOS_DEF_ERROR_CODE(0, 0x1167) //"metric name length is 0" -#define TSDB_CODE_HTTP_TG_METRIC_NAME_LONG TAOS_DEF_ERROR_CODE(0, 0x1168) //"metric name length too long" -#define TSDB_CODE_HTTP_TG_TIMESTAMP_NULL TAOS_DEF_ERROR_CODE(0, 0x1169) //"timestamp not find" -#define TSDB_CODE_HTTP_TG_TIMESTAMP_TYPE TAOS_DEF_ERROR_CODE(0, 0x116A) //"timestamp type should be integer" -#define TSDB_CODE_HTTP_TG_TIMESTAMP_VAL_NULL TAOS_DEF_ERROR_CODE(0, 0x116B) //"timestamp value smaller than 0" -#define TSDB_CODE_HTTP_TG_TAGS_NULL TAOS_DEF_ERROR_CODE(0, 0x116C) //"tags not find" -#define TSDB_CODE_HTTP_TG_TAGS_SIZE_0 TAOS_DEF_ERROR_CODE(0, 0x116D) //"tags size is 0" -#define TSDB_CODE_HTTP_TG_TAGS_SIZE_LONG TAOS_DEF_ERROR_CODE(0, 0x116E) //"tags size too long" -#define TSDB_CODE_HTTP_TG_TAG_NULL TAOS_DEF_ERROR_CODE(0, 0x116F) //"tag is null" -#define TSDB_CODE_HTTP_TG_TAG_NAME_NULL TAOS_DEF_ERROR_CODE(0, 0x1170) //"tag name is null" -#define TSDB_CODE_HTTP_TG_TAG_NAME_SIZE TAOS_DEF_ERROR_CODE(0, 0x1171) //"tag name length too long" -#define TSDB_CODE_HTTP_TG_TAG_VALUE_TYPE TAOS_DEF_ERROR_CODE(0, 0x1172) //"tag value type should be number or string" -#define TSDB_CODE_HTTP_TG_TAG_VALUE_NULL TAOS_DEF_ERROR_CODE(0, 0x1173) //"tag value is null" -#define TSDB_CODE_HTTP_TG_TABLE_NULL TAOS_DEF_ERROR_CODE(0, 0x1174) //"table is null" -#define TSDB_CODE_HTTP_TG_TABLE_SIZE TAOS_DEF_ERROR_CODE(0, 0x1175) //"table name length too long" -#define TSDB_CODE_HTTP_TG_FIELDS_NULL TAOS_DEF_ERROR_CODE(0, 0x1176) //"fields not find" -#define TSDB_CODE_HTTP_TG_FIELDS_SIZE_0 TAOS_DEF_ERROR_CODE(0, 0x1177) //"fields size is 0" -#define TSDB_CODE_HTTP_TG_FIELDS_SIZE_LONG TAOS_DEF_ERROR_CODE(0, 0x1178) //"fields size too long" -#define TSDB_CODE_HTTP_TG_FIELD_NULL TAOS_DEF_ERROR_CODE(0, 0x1179) //"field is null" -#define TSDB_CODE_HTTP_TG_FIELD_NAME_NULL TAOS_DEF_ERROR_CODE(0, 0x117A) //"field name is null" -#define TSDB_CODE_HTTP_TG_FIELD_NAME_SIZE TAOS_DEF_ERROR_CODE(0, 0x117B) //"field name length too long" -#define TSDB_CODE_HTTP_TG_FIELD_VALUE_TYPE TAOS_DEF_ERROR_CODE(0, 0x117C) //"field value type should be number or string" -#define TSDB_CODE_HTTP_TG_FIELD_VALUE_NULL TAOS_DEF_ERROR_CODE(0, 0x117D) //"field value is null" -#define TSDB_CODE_HTTP_TG_HOST_NOT_STRING TAOS_DEF_ERROR_CODE(0, 0x117E) //"host type should be string" -#define TSDB_CODE_HTTP_TG_STABLE_NOT_EXIST TAOS_DEF_ERROR_CODE(0, 0x117F) //"stable not exist" - -#define TSDB_CODE_HTTP_OP_DB_NOT_INPUT TAOS_DEF_ERROR_CODE(0, 0x1190) //"database name can not be null" -#define TSDB_CODE_HTTP_OP_DB_TOO_LONG TAOS_DEF_ERROR_CODE(0, 0x1191) //"database name too long" -#define TSDB_CODE_HTTP_OP_INVALID_JSON TAOS_DEF_ERROR_CODE(0, 0x1192) //"invalid opentsdb json fromat" -#define TSDB_CODE_HTTP_OP_METRICS_NULL TAOS_DEF_ERROR_CODE(0, 0x1193) //"metrics size is 0" -#define TSDB_CODE_HTTP_OP_METRICS_SIZE TAOS_DEF_ERROR_CODE(0, 0x1194) //"metrics size can not more than 10K" -#define TSDB_CODE_HTTP_OP_METRIC_NULL TAOS_DEF_ERROR_CODE(0, 0x1195) //"metric name not find" -#define TSDB_CODE_HTTP_OP_METRIC_TYPE TAOS_DEF_ERROR_CODE(0, 0x1196) //"metric name type should be string" -#define TSDB_CODE_HTTP_OP_METRIC_NAME_NULL TAOS_DEF_ERROR_CODE(0, 0x1197) //"metric name length is 0" -#define TSDB_CODE_HTTP_OP_METRIC_NAME_LONG TAOS_DEF_ERROR_CODE(0, 0x1198) //"metric name length can not more than 22" -#define TSDB_CODE_HTTP_OP_TIMESTAMP_NULL TAOS_DEF_ERROR_CODE(0, 0x1199) //"timestamp not find" -#define TSDB_CODE_HTTP_OP_TIMESTAMP_TYPE TAOS_DEF_ERROR_CODE(0, 0x119A) //"timestamp type should be integer" -#define TSDB_CODE_HTTP_OP_TIMESTAMP_VAL_NULL TAOS_DEF_ERROR_CODE(0, 0x119B) //"timestamp value smaller than 0" -#define TSDB_CODE_HTTP_OP_TAGS_NULL TAOS_DEF_ERROR_CODE(0, 0x119C) //"tags not find" -#define TSDB_CODE_HTTP_OP_TAGS_SIZE_0 TAOS_DEF_ERROR_CODE(0, 0x119D) //"tags size is 0" -#define TSDB_CODE_HTTP_OP_TAGS_SIZE_LONG TAOS_DEF_ERROR_CODE(0, 0x119E) //"tags size too long" -#define TSDB_CODE_HTTP_OP_TAG_NULL TAOS_DEF_ERROR_CODE(0, 0x119F) //"tag is null" -#define TSDB_CODE_HTTP_OP_TAG_NAME_NULL TAOS_DEF_ERROR_CODE(0, 0x11A0) //"tag name is null" -#define TSDB_CODE_HTTP_OP_TAG_NAME_SIZE TAOS_DEF_ERROR_CODE(0, 0x11A1) //"tag name length too long" -#define TSDB_CODE_HTTP_OP_TAG_VALUE_TYPE TAOS_DEF_ERROR_CODE(0, 0x11A2) //"tag value type should be boolean number or string" -#define TSDB_CODE_HTTP_OP_TAG_VALUE_NULL TAOS_DEF_ERROR_CODE(0, 0x11A3) //"tag value is null" -#define TSDB_CODE_HTTP_OP_TAG_VALUE_TOO_LONG TAOS_DEF_ERROR_CODE(0, 0x11A4) //"tag value can not more than 64" -#define TSDB_CODE_HTTP_OP_VALUE_NULL TAOS_DEF_ERROR_CODE(0, 0x11A5) //"value not find" -#define TSDB_CODE_HTTP_OP_VALUE_TYPE TAOS_DEF_ERROR_CODE(0, 0x11A6) //"value type should be boolean number or string" - -#define TSDB_CODE_HTTP_REQUEST_JSON_ERROR TAOS_DEF_ERROR_CODE(0, 0x1F00) //"http request json error" - // tfs #define TSDB_CODE_FS_APP_ERROR TAOS_DEF_ERROR_CODE(0, 0x2200) #define TSDB_CODE_FS_INVLD_CFG TAOS_DEF_ERROR_CODE(0, 0x2201) @@ -570,6 +470,7 @@ int32_t* taosGetErrno(); #define TSDB_CODE_SCH_INTERNAL_ERROR TAOS_DEF_ERROR_CODE(0, 0x2502) #define TSDB_CODE_SCH_IGNORE_ERROR TAOS_DEF_ERROR_CODE(0, 0x2503) #define TSDB_CODE_SCH_TIMEOUT_ERROR TAOS_DEF_ERROR_CODE(0, 0x2504) +#define TSDB_CODE_SCH_JOB_IS_DROPPING TAOS_DEF_ERROR_CODE(0, 0x2505) #define TSDB_CODE_QW_MSG_ERROR TAOS_DEF_ERROR_CODE(0, 0x2550) //parser @@ -658,6 +559,10 @@ int32_t* taosGetErrno(); #define TSDB_CODE_PAR_INVALID_REDISTRIBUTE_VG TAOS_DEF_ERROR_CODE(0, 0x2656) #define TSDB_CODE_PAR_FILL_NOT_ALLOWED_FUNC TAOS_DEF_ERROR_CODE(0, 0x2657) #define TSDB_CODE_PAR_INVALID_WINDOW_PC TAOS_DEF_ERROR_CODE(0, 0x2658) +#define TSDB_CODE_PAR_WINDOW_NOT_ALLOWED_FUNC TAOS_DEF_ERROR_CODE(0, 0x2659) +#define TSDB_CODE_PAR_STREAM_NOT_ALLOWED_FUNC TAOS_DEF_ERROR_CODE(0, 0x265A) +#define TSDB_CODE_PAR_GROUP_BY_NOT_ALLOWED_FUNC TAOS_DEF_ERROR_CODE(0, 0x265B) +#define TSDB_CODE_PAR_INVALID_TABLE_OPTION TAOS_DEF_ERROR_CODE(0, 0x265C) //planner #define TSDB_CODE_PLAN_INTERNAL_ERROR TAOS_DEF_ERROR_CODE(0, 0x2700) @@ -706,6 +611,8 @@ int32_t* taosGetErrno(); //index #define TSDB_CODE_INDEX_REBUILDING TAOS_DEF_ERROR_CODE(0, 0x3200) +//tmq +#define TSDB_CODE_TMQ_INVALID_MSG TAOS_DEF_ERROR_CODE(0, 0x4000) #ifdef __cplusplus } diff --git a/include/util/tdef.h b/include/util/tdef.h index a48e823bade5eeb09d1324470981428f7fca437b..c5a8b95a08cc37e4d665e01f5de01eca97b16087 100644 --- a/include/util/tdef.h +++ b/include/util/tdef.h @@ -344,14 +344,13 @@ typedef enum ELogicConditionType { #define TSDB_DB_SCHEMALESS_OFF 0 #define TSDB_DEFAULT_DB_SCHEMALESS TSDB_DB_SCHEMALESS_OFF -// #define TSDB_MIN_ROLLUP_DELAY 1 -// #define TSDB_MAX_ROLLUP_DELAY 10 -// #define TSDB_DEFAULT_ROLLUP_DELAY 1 -#define TSDB_MIN_ROLLUP_FILE_FACTOR 0 -#define TSDB_MAX_ROLLUP_FILE_FACTOR 10 -#define TSDB_DEFAULT_ROLLUP_FILE_FACTOR 0.1 -#define TSDB_MIN_TABLE_TTL 0 -#define TSDB_DEFAULT_TABLE_TTL 0 +#define TSDB_MIN_ROLLUP_MAX_DELAY 1 // unit millisecond +#define TSDB_MAX_ROLLUP_MAX_DELAY (15 * 60 * 1000) +#define TSDB_MIN_ROLLUP_WATERMARK 0 // unit millisecond +#define TSDB_MAX_ROLLUP_WATERMARK (15 * 60 * 1000) +#define TSDB_DEFAULT_ROLLUP_WATERMARK 5000 +#define TSDB_MIN_TABLE_TTL 0 +#define TSDB_DEFAULT_TABLE_TTL 0 #define TSDB_MIN_EXPLAIN_RATIO 0 #define TSDB_MAX_EXPLAIN_RATIO 1 diff --git a/source/client/inc/clientInt.h b/source/client/inc/clientInt.h index 69a24e20cd7e4a6d20efc9c12f1766f06af050ae..51a5b84a1c3abdd37947092c02cc5a2fc5a474fe 100644 --- a/source/client/inc/clientInt.h +++ b/source/client/inc/clientInt.h @@ -76,10 +76,12 @@ typedef int32_t (*FHbReqHandle)(SClientHbKey* connKey, void* param, SClientHbReq typedef struct { int8_t inited; + int64_t appId; // ctl int8_t threadStop; TdThread thread; TdThreadMutex lock; // used when app init and cleanup + SHashObj *appSummary; SArray* appHbMgrs; // SArray one for each cluster FHbReqHandle reqHandle[CONN_TYPE__MAX]; FHbRspHandle rspHandle[CONN_TYPE__MAX]; @@ -92,33 +94,20 @@ typedef struct SQueryExecMetric { int64_t rsp; // receive response from server, us } SQueryExecMetric; -typedef struct SInstanceSummary { - uint64_t numOfInsertsReq; - uint64_t numOfInsertRows; - uint64_t insertElapsedTime; - uint64_t insertBytes; // submit to tsdb since launched. - - uint64_t fetchBytes; - uint64_t queryElapsedTime; - uint64_t numOfSlowQueries; - uint64_t totalRequests; - uint64_t currentRequests; // the number of SRequestObj -} SInstanceSummary; - typedef struct SHeartBeatInfo { void* pTimer; // timer, used to send request msg to mnode } SHeartBeatInfo; struct SAppInstInfo { - int64_t numOfConns; - SCorEpSet mgmtEp; - TdThreadMutex qnodeMutex; - SArray* pQnodeList; - SInstanceSummary summary; - SList* pConnList; // STscObj linked list - uint64_t clusterId; - void* pTransporter; - SAppHbMgr* pAppHbMgr; + int64_t numOfConns; + SCorEpSet mgmtEp; + TdThreadMutex qnodeMutex; + SArray* pQnodeList; + SAppClusterSummary summary; + SList* pConnList; // STscObj linked list + uint64_t clusterId; + void* pTransporter; + SAppHbMgr* pAppHbMgr; }; typedef struct SAppInfo { @@ -139,7 +128,7 @@ typedef struct STscObj { int8_t connType; int32_t acctId; uint32_t connId; - uint64_t id; // ref ID returned by taosAddRef + TAOS *id; // ref ID returned by taosAddRef TdThreadMutex mutex; // used to protect the operation on db int32_t numOfReqs; // number of sqlObj bound to this connection SAppInstInfo* pAppInfo; @@ -215,6 +204,7 @@ typedef struct SRequestObj { SRequestSendRecvBody body; bool stableQuery; + bool killed; uint32_t prevCode; //previous error code: todo refactor, add update flag for catalog uint32_t retry; } SRequestObj; diff --git a/source/client/src/clientEnv.c b/source/client/src/clientEnv.c index bb60624145f47545769b3a606a25db93268ac1ce..78454268b715d369d91872740fe8add850c2f4ac 100644 --- a/source/client/src/clientEnv.c +++ b/source/client/src/clientEnv.c @@ -38,7 +38,7 @@ static TdThreadOnce tscinit = PTHREAD_ONCE_INIT; volatile int32_t tscInitRes = 0; static void registerRequest(SRequestObj *pRequest) { - STscObj *pTscObj = acquireTscObj(pRequest->pTscObj->id); + STscObj *pTscObj = acquireTscObj(*(int64_t*)pRequest->pTscObj->id); assert(pTscObj != NULL); @@ -48,13 +48,13 @@ static void registerRequest(SRequestObj *pRequest) { int32_t num = atomic_add_fetch_32(&pTscObj->numOfReqs, 1); if (pTscObj->pAppInfo) { - SInstanceSummary *pSummary = &pTscObj->pAppInfo->summary; + SAppClusterSummary *pSummary = &pTscObj->pAppInfo->summary; int32_t total = atomic_add_fetch_64((int64_t *)&pSummary->totalRequests, 1); int32_t currentInst = atomic_add_fetch_64((int64_t *)&pSummary->currentRequests, 1); tscDebug("0x%" PRIx64 " new Request from connObj:0x%" PRIx64 ", current:%d, app current:%d, total:%d, reqId:0x%" PRIx64, - pRequest->self, pRequest->pTscObj->id, num, currentInst, total, pRequest->requestId); + pRequest->self, *(int64_t*)pRequest->pTscObj->id, num, currentInst, total, pRequest->requestId); } } @@ -62,7 +62,7 @@ static void deregisterRequest(SRequestObj *pRequest) { assert(pRequest != NULL); STscObj *pTscObj = pRequest->pTscObj; - SInstanceSummary *pActivity = &pTscObj->pAppInfo->summary; + SAppClusterSummary *pActivity = &pTscObj->pAppInfo->summary; int32_t currentInst = atomic_sub_fetch_64((int64_t *)&pActivity->currentRequests, 1); int32_t num = atomic_sub_fetch_32(&pTscObj->numOfReqs, 1); @@ -70,8 +70,8 @@ static void deregisterRequest(SRequestObj *pRequest) { int64_t duration = taosGetTimestampUs() - pRequest->metric.start; tscDebug("0x%" PRIx64 " free Request from connObj: 0x%" PRIx64 ", reqId:0x%" PRIx64 " elapsed:%" PRIu64 " ms, current:%d, app current:%d", - pRequest->self, pTscObj->id, pRequest->requestId, duration / 1000, num, currentInst); - releaseTscObj(pTscObj->id); + pRequest->self, *(int64_t*)pTscObj->id, pRequest->requestId, duration / 1000, num, currentInst); + releaseTscObj(*(int64_t*)pTscObj->id); } // todo close the transporter properly @@ -80,7 +80,7 @@ void closeTransporter(STscObj *pTscObj) { return; } - tscDebug("free transporter:%p in connObj: 0x%" PRIx64, pTscObj->pAppInfo->pTransporter, pTscObj->id); + tscDebug("free transporter:%p in connObj: 0x%" PRIx64, pTscObj->pAppInfo->pTransporter, *(int64_t*)pTscObj->id); rpcClose(pTscObj->pAppInfo->pTransporter); } @@ -128,7 +128,7 @@ void closeAllRequests(SHashObj *pRequests) { void destroyTscObj(void *pObj) { STscObj *pTscObj = pObj; - SClientHbKey connKey = {.tscRid = pTscObj->id, .connType = pTscObj->connType}; + SClientHbKey connKey = {.tscRid = *(int64_t*)pTscObj->id, .connType = pTscObj->connType}; hbDeregisterConn(pTscObj->pAppInfo->pAppHbMgr, connKey); int64_t connNum = atomic_sub_fetch_64(&pTscObj->pAppInfo->numOfConns, 1); closeAllRequests(pTscObj->pRequests); @@ -137,7 +137,7 @@ void destroyTscObj(void *pObj) { // TODO //closeTransporter(pTscObj); } - tscDebug("connObj 0x%" PRIx64 " destroyed, totalConn:%" PRId64, pTscObj->id, pTscObj->pAppInfo->numOfConns); + tscDebug("connObj 0x%" PRIx64 " destroyed, totalConn:%" PRId64, *(int64_t*)pTscObj->id, pTscObj->pAppInfo->numOfConns); taosThreadMutexDestroy(&pTscObj->mutex); taosMemoryFreeClear(pTscObj); } @@ -166,10 +166,11 @@ void *createTscObj(const char *user, const char *auth, const char *db, int32_t c } taosThreadMutexInit(&pObj->mutex, NULL); - pObj->id = taosAddRef(clientConnRefPool, pObj); + pObj->id = taosMemoryMalloc(sizeof(int64_t)); + *(int64_t*)pObj->id = taosAddRef(clientConnRefPool, pObj); pObj->schemalessType = 1; - tscDebug("connObj created, 0x%" PRIx64, pObj->id); + tscDebug("connObj created, 0x%" PRIx64, *(int64_t*)pObj->id); return pObj; } @@ -229,7 +230,7 @@ static void doDestroyRequest(void *p) { taosHashRemove(pRequest->pTscObj->pRequests, &pRequest->self, sizeof(pRequest->self)); if (pRequest->body.queryJob != 0) { - schedulerFreeJob(pRequest->body.queryJob); + schedulerFreeJob(pRequest->body.queryJob, 0); } taosMemoryFreeClear(pRequest->msgBuf); diff --git a/source/client/src/clientHb.c b/source/client/src/clientHb.c index e0b8f322bfdc446da978376348718017edb86bc9..c6d0d6a860e747a7240b4bab4f9153a07f5ae8c6 100644 --- a/source/client/src/clientHb.c +++ b/source/client/src/clientHb.c @@ -164,6 +164,7 @@ static int32_t hbQueryHbRspHandle(SAppHbMgr *pAppHbMgr, SClientHbRsp *pRsp) { pTscObj->connId = pRsp->query->connId; if (pRsp->query->killRid) { + tscDebug("request rid %" PRIx64 " need to be killed now", pRsp->query->killRid); SRequestObj *pRequest = acquireRequest(pRsp->query->killRid); if (NULL == pRequest) { tscDebug("request 0x%" PRIx64 " not exist to kill", pRsp->query->killRid); @@ -304,7 +305,7 @@ int32_t hbBuildQueryDesc(SQueryHbReqBasic *hbBasic, STscObj *pObj) { while (pIter != NULL) { int64_t *rid = pIter; SRequestObj *pRequest = acquireRequest(*rid); - if (NULL == pRequest) { + if (NULL == pRequest || pRequest->killed) { pIter = taosHashIterate(pObj->pRequests, pIter); continue; } @@ -314,7 +315,6 @@ int32_t hbBuildQueryDesc(SQueryHbReqBasic *hbBasic, STscObj *pObj) { desc.queryId = pRequest->requestId; desc.useconds = now - pRequest->metric.start; desc.reqRid = pRequest->self; - desc.pid = hbBasic->pid; desc.stableQuery = pRequest->stableQuery; taosGetFqdn(desc.fqdn); desc.subPlanNum = pRequest->body.pDag ? pRequest->body.pDag->numOfSubplans : 0; @@ -360,8 +360,6 @@ int32_t hbGetQueryBasicInfo(SClientHbKey *connKey, SClientHbReq *req) { } hbBasic->connId = pTscObj->connId; - hbBasic->pid = taosGetPId(); - taosGetAppName(hbBasic->app, NULL); int32_t numOfQueries = pTscObj->pRequests ? taosHashGetSize(pTscObj->pRequests) : 0; if (numOfQueries <= 0) { @@ -507,6 +505,21 @@ int32_t hbGetExpiredStbInfo(SClientHbKey *connKey, struct SCatalog *pCatalog, SC return TSDB_CODE_SUCCESS; } +int32_t hbGetAppInfo(int64_t clusterId, SClientHbReq *req) { + SAppHbReq* pApp = taosHashGet(clientHbMgr.appSummary, &clusterId, sizeof(clusterId)); + if (NULL != pApp) { + memcpy(&req->app, pApp, sizeof(*pApp)); + } else { + memset(&req->app.summary, 0, sizeof(req->app.summary)); + req->app.pid = taosGetPId(); + req->app.appId = clientHbMgr.appId; + taosGetAppName(req->app.name, NULL); + } + + return TSDB_CODE_SUCCESS; +} + + int32_t hbQueryHbReqHandle(SClientHbKey *connKey, void *param, SClientHbReq *req) { int64_t *clusterId = (int64_t *)param; struct SCatalog *pCatalog = NULL; @@ -517,6 +530,8 @@ int32_t hbQueryHbReqHandle(SClientHbKey *connKey, void *param, SClientHbReq *req return code; } + hbGetAppInfo(*clusterId, req); + hbGetQueryBasicInfo(connKey, req); code = hbGetExpiredUserInfo(connKey, pCatalog, req); @@ -589,6 +604,50 @@ void hbThreadFuncUnexpectedStopped(void) { atomic_store_8(&clientHbMgr.threadStop, 2); } +void hbMergeSummary(SAppClusterSummary* dst, SAppClusterSummary* src) { + dst->numOfInsertsReq += src->numOfInsertsReq; + dst->numOfInsertRows += src->numOfInsertRows; + dst->insertElapsedTime += src->insertElapsedTime; + dst->insertBytes += src->insertBytes; + dst->fetchBytes += src->fetchBytes; + dst->queryElapsedTime += src->queryElapsedTime; + dst->numOfSlowQueries += src->numOfSlowQueries; + dst->totalRequests += src->totalRequests; + dst->currentRequests += src->currentRequests; +} + +int32_t hbGatherAppInfo(void) { + SAppHbReq req = {0}; + int sz = taosArrayGetSize(clientHbMgr.appHbMgrs); + if (sz > 0) { + req.pid = taosGetPId(); + req.appId = clientHbMgr.appId; + taosGetAppName(req.name, NULL); + } + + taosHashClear(clientHbMgr.appSummary); + + for (int32_t i = 0; i < sz; ++i) { + SAppHbMgr *pAppHbMgr = taosArrayGetP(clientHbMgr.appHbMgrs, i); + uint64_t clusterId = pAppHbMgr->pAppInstInfo->clusterId; + SAppHbReq* pApp = taosHashGet(clientHbMgr.appSummary, &clusterId, sizeof(clusterId)); + if (NULL == pApp) { + memcpy(&req.summary, &pAppHbMgr->pAppInstInfo->summary, sizeof(req.summary)); + req.startTime = pAppHbMgr->startTime; + taosHashPut(clientHbMgr.appSummary, &clusterId, sizeof(clusterId), &req, sizeof(req)); + } else { + if (pAppHbMgr->startTime < pApp->startTime) { + pApp->startTime = pAppHbMgr->startTime; + } + + hbMergeSummary(&pApp->summary, &pAppHbMgr->pAppInstInfo->summary); + } + } + + return TSDB_CODE_SUCCESS; +} + + static void *hbThreadFunc(void *param) { setThreadName("hb"); #ifdef WINDOWS @@ -605,6 +664,10 @@ static void *hbThreadFunc(void *param) { taosThreadMutexLock(&clientHbMgr.lock); int sz = taosArrayGetSize(clientHbMgr.appHbMgrs); + if (sz > 0) { + hbGatherAppInfo(); + } + for (int i = 0; i < sz; i++) { SAppHbMgr *pAppHbMgr = taosArrayGetP(clientHbMgr.appHbMgrs, i); @@ -748,6 +811,10 @@ int hbMgrInit() { int8_t old = atomic_val_compare_exchange_8(&clientHbMgr.inited, 0, 1); if (old == 1) return 0; + clientHbMgr.appId = tGenIdPI64(); + tscDebug("app %" PRIx64 " initialized", clientHbMgr.appId); + + clientHbMgr.appSummary = taosHashInit(10, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BIGINT), false, HASH_NO_LOCK); clientHbMgr.appHbMgrs = taosArrayInit(0, sizeof(void *)); taosThreadMutexInit(&clientHbMgr.lock, NULL); diff --git a/source/client/src/clientImpl.c b/source/client/src/clientImpl.c index 10c216147839a36541144c65310eccf8c2d458c6..9668d2b72c202cad01354e7457b216c210bc09d1 100644 --- a/source/client/src/clientImpl.c +++ b/source/client/src/clientImpl.c @@ -263,6 +263,7 @@ void asyncExecLocalCmd(SRequestObj* pRequest, SQuery* pQuery) { int32_t asyncExecDdlQuery(SRequestObj* pRequest, SQuery* pQuery) { // drop table if exists not_exists_table if (NULL == pQuery->pCmdMsg) { + pRequest->body.queryFp(pRequest->body.param, pRequest, 0); return TSDB_CODE_SUCCESS; } @@ -418,7 +419,7 @@ int32_t scheduleAsyncQuery(SRequestObj* pRequest, SQueryPlan* pDag, SArray* pNod while (true) { if (code != TSDB_CODE_SUCCESS) { if (pRequest->body.queryJob != 0) { - schedulerFreeJob(pRequest->body.queryJob); + schedulerFreeJob(pRequest->body.queryJob, 0); } pRequest->code = code; @@ -439,7 +440,7 @@ int32_t scheduleAsyncQuery(SRequestObj* pRequest, SQueryPlan* pDag, SArray* pNod pRequest->body.resInfo.numOfRows = res.numOfRows; if (pRequest->body.queryJob != 0) { - schedulerFreeJob(pRequest->body.queryJob); + schedulerFreeJob(pRequest->body.queryJob, 0); } } @@ -461,14 +462,15 @@ int32_t scheduleQuery(SRequestObj* pRequest, SQueryPlan* pDag, SArray* pNodeList .sql = pRequest->sqlstr, .startTs = pRequest->metric.start, .fp = NULL, - .cbParam = NULL}; + .cbParam = NULL, + .reqKilled = &pRequest->killed}; int32_t code = schedulerExecJob(&req, &pRequest->body.queryJob, &res); pRequest->body.resInfo.execRes = res.res; if (code != TSDB_CODE_SUCCESS) { if (pRequest->body.queryJob != 0) { - schedulerFreeJob(pRequest->body.queryJob); + schedulerFreeJob(pRequest->body.queryJob, 0); } pRequest->code = code; @@ -481,7 +483,7 @@ int32_t scheduleQuery(SRequestObj* pRequest, SQueryPlan* pDag, SArray* pNodeList pRequest->body.resInfo.numOfRows = res.numOfRows; if (pRequest->body.queryJob != 0) { - schedulerFreeJob(pRequest->body.queryJob); + schedulerFreeJob(pRequest->body.queryJob, 0); } } @@ -608,6 +610,19 @@ void schedulerExecCb(SQueryResult* pResult, void* param, int32_t code) { SRequestObj* pRequest = (SRequestObj*)param; pRequest->code = code; + if (TDMT_VND_SUBMIT == pRequest->type || TDMT_VND_DELETE == pRequest->type || + TDMT_VND_CREATE_TABLE == pRequest->type) { + pRequest->body.resInfo.numOfRows = pResult->numOfRows; + + if (pRequest->body.queryJob != 0) { + schedulerFreeJob(pRequest->body.queryJob, 0); + pRequest->body.queryJob = 0; + } + } + + tscDebug("0x%" PRIx64 " enter scheduler exec cb, code:%d - %s, reqId:0x%" PRIx64, + pRequest->self, code, tstrerror(code), pRequest->requestId); + STscObj* pTscObj = pRequest->pTscObj; if (code != TSDB_CODE_SUCCESS && NEED_CLIENT_HANDLE_ERROR(code)) { tscDebug("0x%" PRIx64 " client retry to handle the error, code:%d - %s, tryCount:%d, reqId:0x%" PRIx64, @@ -708,7 +723,7 @@ void launchAsyncQuery(SRequestObj* pRequest, SQuery* pQuery) { code = asyncExecDdlQuery(pRequest, pQuery); break; case QUERY_EXEC_MODE_SCHEDULE: { - SArray* pNodeList = taosArrayInit(4, sizeof(struct SQueryNodeAddr)); + SArray* pNodeList = taosArrayInit(4, sizeof(SQueryNodeLoad)); pRequest->type = pQuery->msgType; @@ -721,12 +736,10 @@ void launchAsyncQuery(SRequestObj* pRequest, SQuery* pQuery) { .msgLen = ERROR_MSG_BUF_DEFAULT_SIZE}; SAppInstInfo* pAppInfo = getAppInfo(pRequest); - if (TSDB_CODE_SUCCESS == code) { - code = qCreateQueryPlan(&cxt, &pRequest->body.pDag, pNodeList); - if (code) { - tscError("0x%" PRIx64 " failed to create query plan, code:%s 0x%" PRIx64, pRequest->self, tstrerror(code), - pRequest->requestId); - } + code = qCreateQueryPlan(&cxt, &pRequest->body.pDag, pNodeList); + if (code) { + tscError("0x%" PRIx64 " failed to create query plan, code:%s 0x%" PRIx64, pRequest->self, tstrerror(code), + pRequest->requestId); } if (TSDB_CODE_SUCCESS == code) { @@ -738,7 +751,8 @@ void launchAsyncQuery(SRequestObj* pRequest, SQuery* pQuery) { .sql = pRequest->sqlstr, .startTs = pRequest->metric.start, .fp = schedulerExecCb, - .cbParam = pRequest}; + .cbParam = pRequest, + .reqKilled = &pRequest->killed}; code = schedulerAsyncExecJob(&req, &pRequest->body.queryJob); } else { tscError("0x%" PRIx64 " failed to create query plan, code:%s 0x%" PRIx64, pRequest->self, tstrerror(code), @@ -922,7 +936,7 @@ STscObj* taosConnectImpl(const char* user, const char* auth, const char* db, __t taos_close_internal(pTscObj); pTscObj = NULL; } else { - tscDebug("0x%" PRIx64 " connection is opening, connId:%u, dnodeConn:%p, reqId:0x%" PRIx64, pTscObj->id, + tscDebug("0x%" PRIx64 " connection is opening, connId:%u, dnodeConn:%p, reqId:0x%" PRIx64, *(int64_t*)pTscObj->id, pTscObj->connId, pTscObj->pAppInfo->pTransporter, pRequest->requestId); destroyRequest(pRequest); } @@ -1085,10 +1099,10 @@ TAOS* taos_connect_auth(const char* ip, const char* user, const char* auth, cons STscObj* pObj = taos_connect_internal(ip, user, NULL, auth, db, port, CONN_TYPE__QUERY); if (pObj) { - return (TAOS*)pObj->id; + return pObj->id; } - return (TAOS*)0; + return NULL; } TAOS* taos_connect_l(const char* ip, int ipLen, const char* user, int userLen, const char* pass, int passLen, diff --git a/source/client/src/clientMain.c b/source/client/src/clientMain.c index 13a5acb472dfce238f31909d37beb7ca69e59b12..51e05537d487fe5c64af5219974f383a9c936a64 100644 --- a/source/client/src/clientMain.c +++ b/source/client/src/clientMain.c @@ -99,10 +99,10 @@ TAOS *taos_connect(const char *ip, const char *user, const char *pass, const cha STscObj* pObj = taos_connect_internal(ip, user, pass, NULL, db, port, CONN_TYPE__QUERY); if (pObj) { - return (TAOS*)pObj->id; + return pObj->id; } - return (TAOS*)0; + return NULL; } void taos_close_internal(void *taos) { @@ -111,19 +111,24 @@ void taos_close_internal(void *taos) { } STscObj *pTscObj = (STscObj *)taos; - tscDebug("0x%" PRIx64 " try to close connection, numOfReq:%d", pTscObj->id, pTscObj->numOfReqs); + tscDebug("0x%" PRIx64 " try to close connection, numOfReq:%d", *(int64_t*)pTscObj->id, pTscObj->numOfReqs); - taosRemoveRef(clientConnRefPool, pTscObj->id); + taosRemoveRef(clientConnRefPool, *(int64_t*)pTscObj->id); } void taos_close(TAOS *taos) { - STscObj* pObj = acquireTscObj((int64_t)taos); + if (taos == NULL) { + return; + } + + STscObj* pObj = acquireTscObj(*(int64_t*)taos); if (NULL == pObj) { return; } taos_close_internal(pObj); - releaseTscObj((int64_t)taos); + releaseTscObj(*(int64_t*)taos); + taosMemoryFree(taos); } @@ -206,7 +211,12 @@ static void syncQueryFn(void *param, void *res, int32_t code) { } TAOS_RES *taos_query(TAOS *taos, const char *sql) { - STscObj* pTscObj = acquireTscObj((int64_t)taos); + if (NULL == taos) { + terrno = TSDB_CODE_TSC_DISCONNECTED; + return NULL; + } + + STscObj* pTscObj = acquireTscObj(*(int64_t*)taos); if (pTscObj == NULL || sql == NULL) { terrno = TSDB_CODE_TSC_DISCONNECTED; return NULL; @@ -219,13 +229,13 @@ TAOS_RES *taos_query(TAOS *taos, const char *sql) { taos_query_a(taos, sql, syncQueryFn, param); tsem_wait(¶m->sem); - releaseTscObj((int64_t)taos); + releaseTscObj(*(int64_t*)taos); return param->pRequest; #else size_t sqlLen = strlen(sql); if (sqlLen > (size_t)TSDB_MAX_ALLOWED_SQL_LEN) { - releaseTscObj((int64_t)taos); + releaseTscObj(*(int64_t*)taos); tscError("sql string exceeds max length:%d", TSDB_MAX_ALLOWED_SQL_LEN); terrno = TSDB_CODE_TSC_EXCEED_SQL_LIMIT; return NULL; @@ -233,7 +243,7 @@ TAOS_RES *taos_query(TAOS *taos, const char *sql) { TAOS_RES* pRes = execQuery(pTscObj, sql, sqlLen); - releaseTscObj((int64_t)taos); + releaseTscObj(*(int64_t*)taos); return pRes; #endif @@ -246,13 +256,14 @@ TAOS_ROW taos_fetch_row(TAOS_RES *res) { if (TD_RES_QUERY(res)) { SRequestObj *pRequest = (SRequestObj *)res; -#if SYNC_ON_TOP_OF_ASYNC - return doAsyncFetchRows(pRequest, true, true); -#else if (pRequest->type == TSDB_SQL_RETRIEVE_EMPTY_RESULT || pRequest->type == TSDB_SQL_INSERT || - pRequest->code != TSDB_CODE_SUCCESS || taos_num_fields(res) == 0) { + pRequest->code != TSDB_CODE_SUCCESS || taos_num_fields(res) == 0 || pRequest->killed) { return NULL; } + +#if SYNC_ON_TOP_OF_ASYNC + return doAsyncFetchRows(pRequest, true, true); +#else return doFetchRows(pRequest, true, true); #endif @@ -452,15 +463,15 @@ int taos_result_precision(TAOS_RES *res) { } int taos_select_db(TAOS *taos, const char *db) { - STscObj* pObj = acquireTscObj((int64_t)taos); + STscObj* pObj = acquireTscObj(*(int64_t*)taos); if (pObj == NULL) { - releaseTscObj((int64_t)taos); + releaseTscObj(*(int64_t*)taos); terrno = TSDB_CODE_TSC_DISCONNECTED; return TSDB_CODE_TSC_DISCONNECTED; } if (db == NULL || strlen(db) == 0) { - releaseTscObj((int64_t)taos); + releaseTscObj(*(int64_t*)taos); terrno = TSDB_CODE_TSC_INVALID_INPUT; return terrno; } @@ -472,7 +483,7 @@ int taos_select_db(TAOS *taos, const char *db) { int32_t code = taos_errno(pRequest); taos_free_result(pRequest); - releaseTscObj((int64_t)taos); + releaseTscObj(*(int64_t*)taos); return code; } @@ -482,14 +493,20 @@ void taos_stop_query(TAOS_RES *res) { } SRequestObj *pRequest = (SRequestObj *)res; + pRequest->killed = true; + int32_t numOfFields = taos_num_fields(pRequest); - // It is not a query, no need to stop. if (numOfFields == 0) { + tscDebug("request %" PRIx64 " no need to be killed since not query", pRequest->requestId); return; } - schedulerFreeJob(pRequest->body.queryJob); + if (pRequest->body.queryJob) { + schedulerFreeJob(pRequest->body.queryJob, TSDB_CODE_TSC_QUERY_KILLED); + } + + tscDebug("request %" PRIx64 " killed", pRequest->requestId); } bool taos_is_null(TAOS_RES *res, int32_t row, int32_t col) { @@ -619,7 +636,7 @@ int *taos_get_column_data_offset(TAOS_RES *res, int columnIndex) { int taos_validate_sql(TAOS *taos, const char *sql) { return true; } void taos_reset_current_db(TAOS *taos) { - STscObj* pTscObj = acquireTscObj((int64_t)taos); + STscObj* pTscObj = acquireTscObj(*(int64_t*)taos); if (pTscObj == NULL) { terrno = TSDB_CODE_TSC_DISCONNECTED; return; @@ -627,17 +644,17 @@ void taos_reset_current_db(TAOS *taos) { resetConnectDB(pTscObj); - releaseTscObj((int64_t)taos); + releaseTscObj(*(int64_t*)taos); } const char *taos_get_server_info(TAOS *taos) { - STscObj* pTscObj = acquireTscObj((int64_t)taos); + STscObj* pTscObj = acquireTscObj(*(int64_t*)taos); if (pTscObj == NULL) { terrno = TSDB_CODE_TSC_DISCONNECTED; return NULL; } - releaseTscObj((int64_t)taos); + releaseTscObj(*(int64_t*)taos); return pTscObj->ver; } @@ -664,6 +681,8 @@ static void destorySqlParseWrapper(SqlParseWrapper *pWrapper) { } void retrieveMetaCallback(SMetaData *pResultMeta, void *param, int32_t code) { + tscDebug("enter meta callback, code %s", tstrerror(code)); + SqlParseWrapper *pWrapper = (SqlParseWrapper *)param; SQuery *pQuery = pWrapper->pQuery; SRequestObj *pRequest = pWrapper->pRequest; @@ -704,11 +723,11 @@ void retrieveMetaCallback(SMetaData *pResultMeta, void *param, int32_t code) { } void taos_query_a(TAOS *taos, const char *sql, __taos_async_fn_t fp, void *param) { - STscObj* pTscObj = acquireTscObj((int64_t)taos); + STscObj* pTscObj = acquireTscObj(*(int64_t*)taos); if (pTscObj == NULL || sql == NULL || NULL == fp) { terrno = TSDB_CODE_INVALID_PARA; if (pTscObj) { - releaseTscObj((int64_t)taos); + releaseTscObj(*(int64_t*)taos); } else { terrno = TSDB_CODE_TSC_DISCONNECTED; } @@ -830,6 +849,9 @@ static void fetchCallback(void *pResult, void *param, int32_t code) { SReqResultInfo *pResultInfo = &pRequest->body.resInfo; + tscDebug("0x%" PRIx64 " enter scheduler fetch cb, code:%d - %s, reqId:0x%" PRIx64, + pRequest->self, code, tstrerror(code), pRequest->requestId); + pResultInfo->pData = pResult; pResultInfo->numOfRows = 0; @@ -926,7 +948,7 @@ int taos_load_table_info(TAOS *taos, const char *tableNameList) { } TAOS_STMT *taos_stmt_init(TAOS *taos) { - STscObj* pObj = acquireTscObj((int64_t)taos); + STscObj* pObj = acquireTscObj(*(int64_t*)taos); if (NULL == pObj) { tscError("invalid parameter for %s", __FUNCTION__); terrno = TSDB_CODE_TSC_DISCONNECTED; @@ -935,7 +957,7 @@ TAOS_STMT *taos_stmt_init(TAOS *taos) { TAOS_STMT* pStmt = stmtInit(pObj); - releaseTscObj((int64_t)taos); + releaseTscObj(*(int64_t*)taos); return pStmt; } diff --git a/source/client/src/clientMsgHandler.c b/source/client/src/clientMsgHandler.c index bf627ec26b7bd6527be8feac03c33bfe64751039..db8aebb32293de4e20c95b96062283d4187e867e 100644 --- a/source/client/src/clientMsgHandler.c +++ b/source/client/src/clientMsgHandler.c @@ -77,7 +77,7 @@ int32_t processConnectRsp(void* param, const SDataBuf* pMsg, int32_t code) { for (int32_t i = 0; i < connectRsp.epSet.numOfEps; ++i) { tscDebug("0x%" PRIx64 " epSet.fqdn[%d]:%s port:%d, connObj:0x%" PRIx64, pRequest->requestId, i, - connectRsp.epSet.eps[i].fqdn, connectRsp.epSet.eps[i].port, pTscObj->id); + connectRsp.epSet.eps[i].fqdn, connectRsp.epSet.eps[i].port, *(int64_t*)pTscObj->id); } pTscObj->connId = connectRsp.connId; @@ -90,7 +90,7 @@ int32_t processConnectRsp(void* param, const SDataBuf* pMsg, int32_t code) { pTscObj->connType = connectRsp.connType; - hbRegisterConn(pTscObj->pAppInfo->pAppHbMgr, pTscObj->id, connectRsp.clusterId, connectRsp.connType); + hbRegisterConn(pTscObj->pAppInfo->pAppHbMgr, *(int64_t*)pTscObj->id, connectRsp.clusterId, connectRsp.connType); // pRequest->body.resInfo.pRspMsg = pMsg->pData; tscDebug("0x%" PRIx64 " clusterId:%" PRId64 ", totalConn:%" PRId64, pRequest->requestId, connectRsp.clusterId, diff --git a/source/client/src/clientSml.c b/source/client/src/clientSml.c index 25018e0d15a2064b02d6b10dc44e31bc5056d7c6..580aafa010f6faeaa941952599d8c2aa1a7d3dd9 100644 --- a/source/client/src/clientSml.c +++ b/source/client/src/clientSml.c @@ -309,7 +309,7 @@ static int32_t smlApplySchemaAction(SSmlHandle *info, SSchemaAction *action) { case SCHEMA_ACTION_ADD_COLUMN: { int n = sprintf(result, "alter stable `%s` add column ", action->alterSTable.sTableName); smlBuildColumnDescription(action->alterSTable.field, result + n, capacity - n, &outBytes); - TAOS_RES *res = taos_query((TAOS*)info->taos->id, result); // TODO async doAsyncQuery + TAOS_RES *res = taos_query(info->taos->id, result); // TODO async doAsyncQuery code = taos_errno(res); const char *errStr = taos_errstr(res); if (code != TSDB_CODE_SUCCESS) { @@ -323,7 +323,7 @@ static int32_t smlApplySchemaAction(SSmlHandle *info, SSchemaAction *action) { case SCHEMA_ACTION_ADD_TAG: { int n = sprintf(result, "alter stable `%s` add tag ", action->alterSTable.sTableName); smlBuildColumnDescription(action->alterSTable.field, result + n, capacity - n, &outBytes); - TAOS_RES *res = taos_query((TAOS*)info->taos->id, result); // TODO async doAsyncQuery + TAOS_RES *res = taos_query(info->taos->id, result); // TODO async doAsyncQuery code = taos_errno(res); const char *errStr = taos_errstr(res); if (code != TSDB_CODE_SUCCESS) { @@ -337,7 +337,7 @@ static int32_t smlApplySchemaAction(SSmlHandle *info, SSchemaAction *action) { case SCHEMA_ACTION_CHANGE_COLUMN_SIZE: { int n = sprintf(result, "alter stable `%s` modify column ", action->alterSTable.sTableName); smlBuildColumnDescription(action->alterSTable.field, result + n, capacity - n, &outBytes); - TAOS_RES *res = taos_query((TAOS*)info->taos->id, result); // TODO async doAsyncQuery + TAOS_RES *res = taos_query(info->taos->id, result); // TODO async doAsyncQuery code = taos_errno(res); if (code != TSDB_CODE_SUCCESS) { uError("SML:0x%" PRIx64 " apply schema action. error : %s", info->id, taos_errstr(res)); @@ -350,7 +350,7 @@ static int32_t smlApplySchemaAction(SSmlHandle *info, SSchemaAction *action) { case SCHEMA_ACTION_CHANGE_TAG_SIZE: { int n = sprintf(result, "alter stable `%s` modify tag ", action->alterSTable.sTableName); smlBuildColumnDescription(action->alterSTable.field, result + n, capacity - n, &outBytes); - TAOS_RES *res = taos_query((TAOS*)info->taos->id, result); // TODO async doAsyncQuery + TAOS_RES *res = taos_query(info->taos->id, result); // TODO async doAsyncQuery code = taos_errno(res); if (code != TSDB_CODE_SUCCESS) { uError("SML:0x%" PRIx64 " apply schema action. error : %s", info->id, taos_errstr(res)); @@ -405,7 +405,7 @@ static int32_t smlApplySchemaAction(SSmlHandle *info, SSchemaAction *action) { pos--; ++freeBytes; outBytes = snprintf(pos, freeBytes, ")"); - TAOS_RES *res = taos_query((TAOS*)info->taos->id, result); + TAOS_RES *res = taos_query(info->taos->id, result); code = taos_errno(res); if (code != TSDB_CODE_SUCCESS) { uError("SML:0x%" PRIx64 " apply schema action. error : %s", info->id, taos_errstr(res)); @@ -2434,7 +2434,7 @@ static void smlInsertCallback(void *param, void *res, int32_t code) { */ TAOS_RES* taos_schemaless_insert(TAOS* taos, char* lines[], int numLines, int protocol, int precision) { - STscObj* pTscObj = acquireTscObj((int64_t)taos); + STscObj* pTscObj = acquireTscObj(*(int64_t*)taos); if (NULL == pTscObj) { terrno = TSDB_CODE_TSC_DISCONNECTED; uError("SML:taos_schemaless_insert invalid taos"); @@ -2443,7 +2443,7 @@ TAOS_RES* taos_schemaless_insert(TAOS* taos, char* lines[], int numLines, int pr SRequestObj* request = (SRequestObj*)createRequest(pTscObj, TSDB_SQL_INSERT); if(!request){ - releaseTscObj((int64_t)taos); + releaseTscObj(*(int64_t*)taos); uError("SML:taos_schemaless_insert error request is null"); return NULL; } @@ -2531,6 +2531,6 @@ end: // ((STscObj *)taos)->schemalessType = 0; pTscObj->schemalessType = 1; uDebug("resultend:%s", request->msgBuf); - releaseTscObj((int64_t)taos); + releaseTscObj(*(int64_t*)taos); return (TAOS_RES*)request; } diff --git a/source/client/src/tmq.c b/source/client/src/tmq.c index dd900e6045294c8d6d6eedee733a682758a24cb9..638b4f1ea559fed1d76009f1f1141aa1c9e36e07 100644 --- a/source/client/src/tmq.c +++ b/source/client/src/tmq.c @@ -48,14 +48,6 @@ struct tmq_list_t { SArray container; }; -struct tmq_topic_vgroup_t { - SMqOffset offset; -}; - -struct tmq_topic_vgroup_list_t { - SArray container; // SArray -}; - struct tmq_conf_t { char clientId[256]; char groupId[TSDB_CGROUP_LEN]; @@ -161,9 +153,9 @@ typedef struct { } SMqPollRspWrapper; typedef struct { - tmq_t* tmq; - tsem_t rspSem; - tmq_resp_err_t rspErr; + tmq_t* tmq; + tsem_t rspSem; + int32_t rspErr; } SMqSubscribeCbParam; typedef struct { @@ -189,7 +181,7 @@ typedef struct { int8_t freeOffsets; tmq_commit_cb* userCb; tsem_t rspSem; - tmq_resp_err_t rspErr; + int32_t rspErr; SArray* offsets; void* userParam; } SMqCommitCbParam; @@ -201,12 +193,12 @@ typedef struct { int8_t freeOffsets; int32_t waitingRspNum; int32_t totalRspNum; - tmq_resp_err_t rspErr; + int32_t rspErr; tmq_commit_cb* userCb; - SArray* successfulOffsets; - SArray* failedOffsets; - void* userParam; - tsem_t rspSem; + /*SArray* successfulOffsets;*/ + /*SArray* failedOffsets;*/ + void* userParam; + tsem_t rspSem; } SMqCommitCbParamSet; typedef struct { @@ -347,10 +339,9 @@ int32_t tmqCommitCb(void* param, const SDataBuf* pMsg, int32_t code) { pParam->rspErr = code; if (pParam->async) { if (pParam->automatic && pParam->tmq->commitCb) { - pParam->tmq->commitCb(pParam->tmq, pParam->rspErr, (tmq_topic_vgroup_list_t*)pParam->offsets, - pParam->tmq->commitCbUserParam); + pParam->tmq->commitCb(pParam->tmq, pParam->rspErr, pParam->tmq->commitCbUserParam); } else if (!pParam->automatic && pParam->userCb) { - pParam->userCb(pParam->tmq, pParam->rspErr, (tmq_topic_vgroup_list_t*)pParam->offsets, pParam->userParam); + pParam->userCb(pParam->tmq, pParam->rspErr, pParam->userParam); } if (pParam->freeOffsets) { @@ -368,11 +359,16 @@ int32_t tmqCommitCb2(void* param, const SDataBuf* pBuf, int32_t code) { SMqCommitCbParam2* pParam = (SMqCommitCbParam2*)param; SMqCommitCbParamSet* pParamSet = (SMqCommitCbParamSet*)pParam->params; // push into array +#if 0 if (code == 0) { taosArrayPush(pParamSet->failedOffsets, &pParam->pOffset); } else { taosArrayPush(pParamSet->successfulOffsets, &pParam->pOffset); } +#endif + + /*tscDebug("receive offset commit cb of %s on vg %d, offset is %ld", pParam->pOffset->subKey, pParam->->vgId, + * pOffset->version);*/ // count down waiting rsp int32_t waitingRspNum = atomic_sub_fetch_32(&pParamSet->waitingRspNum, 1); @@ -383,23 +379,149 @@ int32_t tmqCommitCb2(void* param, const SDataBuf* pBuf, int32_t code) { if (pParamSet->async) { // call async cb func if (pParamSet->automatic && pParamSet->tmq->commitCb) { - pParamSet->tmq->commitCb(pParamSet->tmq, pParamSet->rspErr, NULL, pParamSet->tmq->commitCbUserParam); + pParamSet->tmq->commitCb(pParamSet->tmq, pParamSet->rspErr, pParamSet->tmq->commitCbUserParam); } else if (!pParamSet->automatic && pParamSet->userCb) { // sem post - pParamSet->userCb(pParamSet->tmq, pParamSet->rspErr, NULL, pParamSet->userParam); + pParamSet->userCb(pParamSet->tmq, pParamSet->rspErr, pParamSet->userParam); } + } else { + tsem_post(&pParamSet->rspSem); } +#if 0 taosArrayDestroyP(pParamSet->successfulOffsets, taosMemoryFree); taosArrayDestroyP(pParamSet->failedOffsets, taosMemoryFree); +#endif } return 0; } -int32_t tmqCommitInner2(tmq_t* tmq, const tmq_topic_vgroup_list_t* offsets, int8_t automatic, int8_t async, - tmq_commit_cb* userCb, void* userParam) { +int32_t tmqCommitInner2(tmq_t* tmq, const TAOS_RES* msg, int8_t automatic, int8_t async, tmq_commit_cb* userCb, + void* userParam) { int32_t code = -1; + if (msg != NULL) { + SMqRspObj* pRspObj = (SMqRspObj*)msg; + if (!TD_RES_TMQ(pRspObj)) { + return TSDB_CODE_TMQ_INVALID_MSG; + } + + SMqCommitCbParamSet* pParamSet = taosMemoryCalloc(1, sizeof(SMqCommitCbParamSet)); + if (pParamSet == NULL) { + terrno = TSDB_CODE_OUT_OF_MEMORY; + return -1; + } + pParamSet->tmq = tmq; + pParamSet->automatic = automatic; + pParamSet->async = async; + pParamSet->freeOffsets = 1; + pParamSet->userCb = userCb; + pParamSet->userParam = userParam; + tsem_init(&pParamSet->rspSem, 0, 0); + + for (int32_t i = 0; i < taosArrayGetSize(tmq->clientTopics); i++) { + SMqClientTopic* pTopic = taosArrayGet(tmq->clientTopics, i); + if (strcmp(pTopic->topicName, pRspObj->topic) == 0) { + for (int32_t j = 0; j < taosArrayGetSize(pTopic->vgs); j++) { + SMqClientVg* pVg = taosArrayGet(pTopic->vgs, j); + if (pVg->vgId == pRspObj->vgId) { + if (pVg->currentOffset < 0 || pVg->committedOffset == pVg->currentOffset) { + tscDebug("consumer %ld skip commit for topic %s vg %d, current offset is %ld, committed offset is %ld", + tmq->consumerId, pTopic->topicName, pVg->vgId, pVg->currentOffset, pVg->committedOffset); + + return 0; + } + + STqOffset* pOffset = taosMemoryCalloc(1, sizeof(STqOffset)); + if (pOffset == NULL) { + terrno = TSDB_CODE_OUT_OF_MEMORY; + return -1; + } + pOffset->type = TMQ_OFFSET__LOG; + pOffset->version = pVg->currentOffset; + + int32_t groupLen = strlen(tmq->groupId); + memcpy(pOffset->subKey, tmq->groupId, groupLen); + pOffset->subKey[groupLen] = TMQ_SEPARATOR; + strcpy(pOffset->subKey + groupLen + 1, pTopic->topicName); + + int32_t len; + int32_t code; + tEncodeSize(tEncodeSTqOffset, pOffset, len, code); + if (code < 0) { + ASSERT(0); + } + void* buf = taosMemoryCalloc(1, sizeof(SMsgHead) + len); + ((SMsgHead*)buf)->vgId = htonl(pVg->vgId); + + void* abuf = POINTER_SHIFT(buf, sizeof(SMsgHead)); + + SEncoder encoder; + tEncoderInit(&encoder, abuf, len); + tEncodeSTqOffset(&encoder, pOffset); + + // build param + SMqCommitCbParam2* pParam = taosMemoryCalloc(1, sizeof(SMqCommitCbParam2)); + pParam->params = pParamSet; + pParam->pOffset = pOffset; + + // build send info + SMsgSendInfo* pMsgSendInfo = taosMemoryCalloc(1, sizeof(SMsgSendInfo)); + if (pMsgSendInfo == NULL) { + // TODO + continue; + } + pMsgSendInfo->msgInfo = (SDataBuf){ + .pData = buf, + .len = sizeof(SMsgHead) + len, + .handle = NULL, + }; + + tscDebug("consumer %ld commit offset of %s on vg %d, offset is %ld", tmq->consumerId, pOffset->subKey, + pVg->vgId, pOffset->version); + + // TODO: put into cb + pVg->committedOffset = pVg->currentOffset; + + pMsgSendInfo->requestId = generateRequestId(); + pMsgSendInfo->requestObjRefId = 0; + pMsgSendInfo->param = pParam; + pMsgSendInfo->fp = tmqCommitCb2; + pMsgSendInfo->msgType = TDMT_VND_MQ_COMMIT_OFFSET; + // send msg + + int64_t transporterId = 0; + asyncSendMsgToServer(tmq->pTscObj->pAppInfo->pTransporter, &pVg->epSet, &transporterId, pMsgSendInfo); + pParamSet->waitingRspNum++; + pParamSet->totalRspNum++; + } + } + } + } + if (pParamSet->totalRspNum == 0) { + tsem_destroy(&pParamSet->rspSem); + taosMemoryFree(pParamSet); + return 0; + } + + if (!async) { + tsem_wait(&pParamSet->rspSem); + code = pParamSet->rspErr; + tsem_destroy(&pParamSet->rspSem); + } else { + code = 0; + } + + if (code != 0 && async) { + if (automatic) { + tmq->commitCb(tmq, code, tmq->commitCbUserParam); + } else { + userCb(tmq, code, userParam); + } + } + return 0; + } + SMqCommitCbParamSet* pParamSet = taosMemoryCalloc(1, sizeof(SMqCommitCbParamSet)); if (pParamSet == NULL) { terrno = TSDB_CODE_OUT_OF_MEMORY; @@ -415,17 +537,35 @@ int32_t tmqCommitInner2(tmq_t* tmq, const tmq_topic_vgroup_list_t* offsets, int8 for (int32_t i = 0; i < taosArrayGetSize(tmq->clientTopics); i++) { SMqClientTopic* pTopic = taosArrayGet(tmq->clientTopics, i); + + tscDebug("consumer %ld begin commit for topic %s, vgNum %d", tmq->consumerId, pTopic->topicName, + (int32_t)taosArrayGetSize(pTopic->vgs)); + for (int32_t j = 0; j < taosArrayGetSize(pTopic->vgs); j++) { - SMqClientVg* pVg = taosArrayGet(pTopic->vgs, i); - STqOffset* pOffset = taosMemoryCalloc(1, sizeof(STqOffset)); + SMqClientVg* pVg = taosArrayGet(pTopic->vgs, j); + + tscDebug("consumer %ld begin commit for topic %s, vgId %d", tmq->consumerId, pTopic->topicName, pVg->vgId); + + /*if (pVg->currentOffset < 0) {*/ + if (pVg->currentOffset < 0 || pVg->committedOffset == pVg->currentOffset) { + tscDebug("consumer %ld skip commit for topic %s vg %d, current offset is %ld, committed offset is %ld", + tmq->consumerId, pTopic->topicName, pVg->vgId, pVg->currentOffset, pVg->committedOffset); + + continue; + } + STqOffset* pOffset = taosMemoryCalloc(1, sizeof(STqOffset)); if (pOffset == NULL) { terrno = TSDB_CODE_OUT_OF_MEMORY; return -1; } - int32_t tlen = strlen(tmq->groupId); - memcpy(pOffset->subKey, tmq->groupId, tlen); - pOffset->subKey[tlen] = TMQ_SEPARATOR; - strcpy(pOffset->subKey + tlen + 1, pTopic->topicName); + pOffset->type = TMQ_OFFSET__LOG; + pOffset->version = pVg->currentOffset; + + int32_t groupLen = strlen(tmq->groupId); + memcpy(pOffset->subKey, tmq->groupId, groupLen); + pOffset->subKey[groupLen] = TMQ_SEPARATOR; + strcpy(pOffset->subKey + groupLen + 1, pTopic->topicName); + int32_t len; int32_t code; tEncodeSize(tEncodeSTqOffset, pOffset, len, code); @@ -454,25 +594,36 @@ int32_t tmqCommitInner2(tmq_t* tmq, const tmq_topic_vgroup_list_t* offsets, int8 } pMsgSendInfo->msgInfo = (SDataBuf){ .pData = buf, - .len = len, + .len = sizeof(SMsgHead) + len, .handle = NULL, }; + tscDebug("consumer %ld commit offset of %s on vg %d, offset is %ld", tmq->consumerId, pOffset->subKey, pVg->vgId, + pOffset->version); + + // TODO: put into cb + pVg->committedOffset = pVg->currentOffset; + pMsgSendInfo->requestId = generateRequestId(); pMsgSendInfo->requestObjRefId = 0; pMsgSendInfo->param = pParam; pMsgSendInfo->fp = tmqCommitCb2; - pMsgSendInfo->msgType = TDMT_MND_MQ_COMMIT_OFFSET; + pMsgSendInfo->msgType = TDMT_VND_MQ_COMMIT_OFFSET; // send msg - SEpSet epSet = getEpSet_s(&tmq->pTscObj->pAppInfo->mgmtEp); int64_t transporterId = 0; - asyncSendMsgToServer(tmq->pTscObj->pAppInfo->pTransporter, &epSet, &transporterId, pMsgSendInfo); + asyncSendMsgToServer(tmq->pTscObj->pAppInfo->pTransporter, &pVg->epSet, &transporterId, pMsgSendInfo); pParamSet->waitingRspNum++; pParamSet->totalRspNum++; } } + if (pParamSet->totalRspNum == 0) { + tsem_destroy(&pParamSet->rspSem); + taosMemoryFree(pParamSet); + return 0; + } + if (!async) { tsem_wait(&pParamSet->rspSem); code = pParamSet->rspErr; @@ -483,21 +634,24 @@ int32_t tmqCommitInner2(tmq_t* tmq, const tmq_topic_vgroup_list_t* offsets, int8 if (code != 0 && async) { if (automatic) { - tmq->commitCb(tmq, code, NULL, tmq->commitCbUserParam); + tmq->commitCb(tmq, code, tmq->commitCbUserParam); } else { - userCb(tmq, code, NULL, userParam); + userCb(tmq, code, userParam); } } +#if 0 if (!async) { taosArrayDestroyP(pParamSet->successfulOffsets, taosMemoryFree); taosArrayDestroyP(pParamSet->failedOffsets, taosMemoryFree); } +#endif return 0; } -int32_t tmqCommitInner(tmq_t* tmq, const tmq_topic_vgroup_list_t* offsets, int8_t automatic, int8_t async, +#if 0 +int32_t tmqCommitInner(tmq_t* tmq, const TAOS_RES* msg, int8_t automatic, int8_t async, tmq_commit_cb* userCb, void* userParam) { SMqCMCommitOffsetReq req; SArray* pOffsets = NULL; @@ -507,7 +661,7 @@ int32_t tmqCommitInner(tmq_t* tmq, const tmq_topic_vgroup_list_t* offsets, int8_ int8_t freeOffsets; int32_t code = -1; - if (offsets == NULL) { + if (msg == NULL) { freeOffsets = 1; pOffsets = taosArrayInit(0, sizeof(SMqOffset)); for (int32_t i = 0; i < taosArrayGetSize(tmq->clientTopics); i++) { @@ -524,7 +678,7 @@ int32_t tmqCommitInner(tmq_t* tmq, const tmq_topic_vgroup_list_t* offsets, int8_ } } else { freeOffsets = 0; - pOffsets = (SArray*)&offsets->container; + pOffsets = (SArray*)&msg->container; } req.num = (int32_t)pOffsets->size; @@ -611,6 +765,7 @@ END: } return code; } +#endif void tmqAssignDelayedHbTask(void* param, void* tmrId) { tmq_t* tmq = (tmq_t*)param; @@ -648,7 +803,7 @@ int32_t tmqHandleAllDelayedTask(tmq_t* tmq) { tmqAskEp(tmq, true); taosTmrReset(tmqAssignDelayedHbTask, 1000, tmq, tmqMgmt.timer, &tmq->hbTimer); } else if (*pTaskType == TMQ_DELAYED_TASK__COMMIT) { - tmqCommitInner(tmq, NULL, 1, 1, tmq->commitCb, tmq->commitCbUserParam); + tmqCommitInner2(tmq, NULL, 1, 1, tmq->commitCb, tmq->commitCbUserParam); taosTmrReset(tmqAssignDelayedCommitTask, tmq->autoCommitInterval, tmq, tmqMgmt.timer, &tmq->commitTimer); } else if (*pTaskType == TMQ_DELAYED_TASK__REPORT) { } else { @@ -689,7 +844,7 @@ int32_t tmqSubscribeCb(void* param, const SDataBuf* pMsg, int32_t code) { return 0; } -tmq_resp_err_t tmq_subscription(tmq_t* tmq, tmq_list_t** topics) { +int32_t tmq_subscription(tmq_t* tmq, tmq_list_t** topics) { if (*topics == NULL) { *topics = tmq_list_new(); } @@ -697,12 +852,12 @@ tmq_resp_err_t tmq_subscription(tmq_t* tmq, tmq_list_t** topics) { SMqClientTopic* topic = taosArrayGet(tmq->clientTopics, i); tmq_list_append(*topics, strchr(topic->topicName, '.') + 1); } - return TMQ_RESP_ERR__SUCCESS; + return 0; } -tmq_resp_err_t tmq_unsubscribe(tmq_t* tmq) { - tmq_list_t* lst = tmq_list_new(); - tmq_resp_err_t rsp = tmq_subscribe(tmq, lst); +int32_t tmq_unsubscribe(tmq_t* tmq) { + tmq_list_t* lst = tmq_list_new(); + int32_t rsp = tmq_subscribe(tmq, lst); tmq_list_destroy(lst); return rsp; } @@ -818,11 +973,13 @@ FAIL: return NULL; } -tmq_resp_err_t tmq_commit(tmq_t* tmq, const tmq_topic_vgroup_list_t* offsets, int32_t async) { - return tmqCommitInner(tmq, offsets, 0, async, tmq->commitCb, tmq->commitCbUserParam); +#if 0 +int32_t tmq_commit(tmq_t* tmq, const tmq_topic_vgroup_list_t* offsets, int32_t async) { + return tmqCommitInner2(tmq, offsets, 0, async, tmq->commitCb, tmq->commitCbUserParam); } +#endif -tmq_resp_err_t tmq_subscribe(tmq_t* tmq, const tmq_list_t* topic_list) { +int32_t tmq_subscribe(tmq_t* tmq, const tmq_list_t* topic_list) { const SArray* container = &topic_list->container; int32_t sz = taosArrayGetSize(container); void* buf = NULL; @@ -862,7 +1019,7 @@ tmq_resp_err_t tmq_subscribe(tmq_t* tmq, const tmq_list_t* topic_list) { if (sendInfo == NULL) goto FAIL; SMqSubscribeCbParam param = { - .rspErr = TMQ_RESP_ERR__SUCCESS, + .rspErr = 0, .tmq = tmq, }; @@ -943,6 +1100,19 @@ int32_t tmqPollCb(void* param, const SDataBuf* pMsg, int32_t code) { if (code != 0) { tscWarn("msg discard from vg %d, epoch %d, code:%x", vgId, epoch, code); if (pMsg->pData) taosMemoryFree(pMsg->pData); + if (code == TSDB_CODE_TQ_NO_COMMITTED_OFFSET) { + SMqPollRspWrapper* pRspWrapper = taosAllocateQitem(sizeof(SMqPollRspWrapper), DEF_QITEM); + if (pRspWrapper == NULL) { + taosMemoryFree(pMsg->pData); + tscWarn("msg discard from vg %d, epoch %d since out of memory", vgId, epoch); + goto CREATE_MSG_FAIL; + } + pRspWrapper->tmqRspType = TMQ_MSG_TYPE__END_RSP; + /*pRspWrapper->vgHandle = pVg;*/ + /*pRspWrapper->topicHandle = pTopic;*/ + taosWriteQitem(tmq->mqueue, pRspWrapper); + tsem_post(&tmq->rspSem); + } goto CREATE_MSG_FAIL; } @@ -1017,14 +1187,13 @@ bool tmqUpdateEp2(tmq_t* tmq, int32_t epoch, SMqAskEpRsp* pRsp) { if (pTopicCur->vgs) { int32_t vgNumCur = taosArrayGetSize(pTopicCur->vgs); tscDebug("consumer %ld new vg num: %d", tmq->consumerId, vgNumCur); - if (vgNumCur == 0) break; for (int32_t j = 0; j < vgNumCur; j++) { SMqClientVg* pVgCur = taosArrayGet(pTopicCur->vgs, j); sprintf(vgKey, "%s:%d", pTopicCur->topicName, pVgCur->vgId); - tscDebug("consumer %ld epoch %d vg %d build %s", tmq->consumerId, epoch, pVgCur->vgId, vgKey); + tscDebug("consumer %ld epoch %d vg %d vgKey is %s, offset is %ld", tmq->consumerId, epoch, pVgCur->vgId, vgKey, + pVgCur->currentOffset); taosHashPut(pHash, vgKey, strlen(vgKey), &pVgCur->currentOffset, sizeof(int64_t)); } - break; } } @@ -1032,7 +1201,6 @@ bool tmqUpdateEp2(tmq_t* tmq, int32_t epoch, SMqAskEpRsp* pRsp) { SMqClientTopic topic = {0}; SMqSubTopicEp* pTopicEp = taosArrayGet(pRsp->topics, i); topic.schema = pTopicEp->schema; - taosHashClear(pHash); topic.topicName = strdup(pTopicEp->topic); tstrncpy(topic.db, pTopicEp->db, TSDB_DB_FNAME_LEN); @@ -1049,7 +1217,8 @@ bool tmqUpdateEp2(tmq_t* tmq, int32_t epoch, SMqAskEpRsp* pRsp) { offset = *pOffset; } - tscDebug("consumer %ld(epoch %d) offset of vg %d updated to %ld", tmq->consumerId, epoch, pVgEp->vgId, offset); + tscDebug("consumer %ld(epoch %d) offset of vg %d updated to %ld, vgKey is %s", tmq->consumerId, epoch, + pVgEp->vgId, offset, vgKey); SMqClientVg clientVg = { .pollCnt = 0, .currentOffset = offset, @@ -1076,6 +1245,7 @@ bool tmqUpdateEp2(tmq_t* tmq, int32_t epoch, SMqAskEpRsp* pRsp) { return set; } +#if 1 bool tmqUpdateEp(tmq_t* tmq, int32_t epoch, SMqAskEpRsp* pRsp) { /*printf("call update ep %d\n", epoch);*/ bool set = false; @@ -1160,6 +1330,7 @@ bool tmqUpdateEp(tmq_t* tmq, int32_t epoch, SMqAskEpRsp* pRsp) { atomic_store_32(&tmq->epoch, epoch); return set; } +#endif int32_t tmqAskEpCb(void* param, const SDataBuf* pMsg, int32_t code) { SMqAskEpCbParam* pParam = (SMqAskEpCbParam*)param; @@ -1186,7 +1357,7 @@ int32_t tmqAskEpCb(void* param, const SDataBuf* pMsg, int32_t code) { tDecodeSMqAskEpRsp(POINTER_SHIFT(pMsg->pData, sizeof(SMqRspHead)), &rsp); /*printf("rsp epoch %ld sz %ld\n", rsp.epoch, rsp.topics->size);*/ /*printf("tmq epoch %ld sz %ld\n", tmq->epoch, tmq->clientTopics->size);*/ - tmqUpdateEp(tmq, head->epoch, &rsp); + tmqUpdateEp2(tmq, head->epoch, &rsp); tDeleteSMqAskEpRsp(&rsp); } else { SMqAskEpRspWrapper* pWrapper = taosAllocateQitem(sizeof(SMqAskEpRspWrapper), DEF_QITEM); @@ -1283,7 +1454,8 @@ int32_t tmqAskEp(tmq_t* tmq, bool async) { return code; } -tmq_resp_err_t tmq_seek(tmq_t* tmq, const tmq_topic_vgroup_t* offset) { +#if 0 +int32_t tmq_seek(tmq_t* tmq, const tmq_topic_vgroup_t* offset) { const SMqOffset* pOffset = &offset->offset; if (strcmp(pOffset->cgroup, tmq->groupId) != 0) { return TMQ_RESP_ERR__FAIL; @@ -1305,16 +1477,17 @@ tmq_resp_err_t tmq_seek(tmq_t* tmq, const tmq_topic_vgroup_t* offset) { } return TMQ_RESP_ERR__FAIL; } +#endif SMqPollReq* tmqBuildConsumeReqImpl(tmq_t* tmq, int64_t timeout, SMqClientTopic* pTopic, SMqClientVg* pVg) { int64_t reqOffset; if (pVg->currentOffset >= 0) { reqOffset = pVg->currentOffset; } else { - if (tmq->resetOffsetCfg == TMQ_CONF__RESET_OFFSET__NONE) { - tscError("unable to poll since no committed offset but reset offset is set to none"); - return NULL; - } + /*if (tmq->resetOffsetCfg == TMQ_CONF__RESET_OFFSET__NONE) {*/ + /*tscError("unable to poll since no committed offset but reset offset is set to none");*/ + /*return NULL;*/ + /*}*/ reqOffset = tmq->resetOffsetCfg; } @@ -1326,10 +1499,10 @@ SMqPollReq* tmqBuildConsumeReqImpl(tmq_t* tmq, int64_t timeout, SMqClientTopic* /*strcpy(pReq->topic, pTopic->topicName);*/ /*strcpy(pReq->cgroup, tmq->groupId);*/ - int32_t tlen = strlen(tmq->groupId); - memcpy(pReq->subKey, tmq->groupId, tlen); - pReq->subKey[tlen] = TMQ_SEPARATOR; - strcpy(pReq->subKey + tlen + 1, pTopic->topicName); + int32_t groupLen = strlen(tmq->groupId); + memcpy(pReq->subKey, tmq->groupId, groupLen); + pReq->subKey[groupLen] = TMQ_SEPARATOR; + strcpy(pReq->subKey + groupLen + 1, pTopic->topicName); pReq->withTbName = tmq->withTbName; pReq->timeout = timeout; @@ -1440,7 +1613,7 @@ int32_t tmqHandleNoPollRsp(tmq_t* tmq, SMqRspWrapper* rspWrapper, bool* pReset) if (rspWrapper->epoch > atomic_load_32(&tmq->epoch)) { SMqAskEpRspWrapper* pEpRspWrapper = (SMqAskEpRspWrapper*)rspWrapper; SMqAskEpRsp* rspMsg = &pEpRspWrapper->msg; - tmqUpdateEp(tmq, rspWrapper->epoch, rspMsg); + tmqUpdateEp2(tmq, rspWrapper->epoch, rspMsg); /*tmqClearUnhandleMsg(tmq);*/ *pReset = true; } else { @@ -1462,7 +1635,11 @@ SMqRspObj* tmqHandleAllRsp(tmq_t* tmq, int64_t timeout, bool pollIfReset) { if (rspWrapper == NULL) return NULL; } - if (rspWrapper->tmqRspType == TMQ_MSG_TYPE__POLL_RSP) { + if (rspWrapper->tmqRspType == TMQ_MSG_TYPE__END_RSP) { + taosFreeQitem(rspWrapper); + terrno = TSDB_CODE_TQ_NO_COMMITTED_OFFSET; + return NULL; + } else if (rspWrapper->tmqRspType == TMQ_MSG_TYPE__POLL_RSP) { SMqPollRspWrapper* pollRspWrapper = (SMqPollRspWrapper*)rspWrapper; /*atomic_sub_fetch_32(&tmq->readyRequest, 1);*/ int32_t consumerEpoch = atomic_load_32(&tmq->epoch); @@ -1523,6 +1700,8 @@ TAOS_RES* tmq_consumer_poll(tmq_t* tmq, int64_t timeout) { rspObj = tmqHandleAllRsp(tmq, timeout, false); if (rspObj) { return (TAOS_RES*)rspObj; + } else if (terrno == TSDB_CODE_TQ_NO_COMMITTED_OFFSET) { + return NULL; } if (timeout != -1) { int64_t endTime = taosGetTimestampMs(); @@ -1539,10 +1718,10 @@ TAOS_RES* tmq_consumer_poll(tmq_t* tmq, int64_t timeout) { } } -tmq_resp_err_t tmq_consumer_close(tmq_t* tmq) { +int32_t tmq_consumer_close(tmq_t* tmq) { if (tmq->status == TMQ_CONSUMER_STATUS__READY) { - tmq_resp_err_t rsp = tmq_commit_sync(tmq, NULL); - if (rsp != TMQ_RESP_ERR__SUCCESS) { + int32_t rsp = tmq_commit_sync(tmq, NULL); + if (rsp != 0) { return rsp; } @@ -1550,18 +1729,18 @@ tmq_resp_err_t tmq_consumer_close(tmq_t* tmq) { rsp = tmq_subscribe(tmq, lst); tmq_list_destroy(lst); - if (rsp != TMQ_RESP_ERR__SUCCESS) { + if (rsp != 0) { return rsp; } } // TODO: free resources - return TMQ_RESP_ERR__SUCCESS; + return 0; } -const char* tmq_err2str(tmq_resp_err_t err) { - if (err == TMQ_RESP_ERR__SUCCESS) { +const char* tmq_err2str(int32_t err) { + if (err == 0) { return "success"; - } else if (err == TMQ_RESP_ERR__FAIL) { + } else if (err == -1) { return "fail"; } else { return tstrerror(err); @@ -1607,10 +1786,8 @@ const char* tmq_get_table_name(TAOS_RES* res) { return NULL; } -void tmq_commit_async(tmq_t* tmq, const tmq_topic_vgroup_list_t* offsets, tmq_commit_cb* cb, void* param) { - tmqCommitInner(tmq, offsets, 0, 1, cb, param); +void tmq_commit_async(tmq_t* tmq, const TAOS_RES* msg, tmq_commit_cb* cb, void* param) { + tmqCommitInner2(tmq, msg, 0, 1, cb, param); } -tmq_resp_err_t tmq_commit_sync(tmq_t* tmq, const tmq_topic_vgroup_list_t* offsets) { - return tmqCommitInner(tmq, offsets, 0, 0, NULL, NULL); -} +int32_t tmq_commit_sync(tmq_t* tmq, const TAOS_RES* msg) { return tmqCommitInner2(tmq, msg, 0, 0, NULL, NULL); } diff --git a/source/common/src/systable.c b/source/common/src/systable.c index bcc248d122172861daaf0bad5165340b2a22082d..a1b64b363c7c2c6d9dd62cc7c7979534b35ac92d 100644 --- a/source/common/src/systable.c +++ b/source/common/src/systable.c @@ -77,7 +77,7 @@ static const SSysDbTableSchema userDBSchema[] = { {.name = "ntables", .bytes = 8, .type = TSDB_DATA_TYPE_BIGINT}, {.name = "replica", .bytes = 1, .type = TSDB_DATA_TYPE_TINYINT}, {.name = "strict", .bytes = 9 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "duration", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, + {.name = "duration", .bytes = 10 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, {.name = "keep", .bytes = 32 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, {.name = "buffer", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, {.name = "pagesize", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, @@ -220,7 +220,8 @@ static const SSysDbTableSchema transSchema[] = { {.name = "id", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, {.name = "create_time", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP}, {.name = "stage", .bytes = TSDB_TRANS_STAGE_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, - {.name = "db", .bytes = SYSTABLE_SCH_DB_NAME_LEN, .type = TSDB_DATA_TYPE_VARCHAR}, + {.name = "db1", .bytes = SYSTABLE_SCH_DB_NAME_LEN, .type = TSDB_DATA_TYPE_VARCHAR}, + {.name = "db2", .bytes = SYSTABLE_SCH_DB_NAME_LEN, .type = TSDB_DATA_TYPE_VARCHAR}, {.name = "failed_times", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, {.name = "last_exec_time", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP}, {.name = "last_action_info", @@ -302,7 +303,7 @@ static const SSysDbTableSchema offsetSchema[] = { }; static const SSysDbTableSchema querySchema[] = { - {.name = "query_id", .bytes = 26 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, + {.name = "query_id", .bytes = TSDB_QUERY_ID_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, {.name = "req_id", .bytes = 8, .type = TSDB_DATA_TYPE_UBIGINT}, {.name = "connId", .bytes = 4, .type = TSDB_DATA_TYPE_UINT}, {.name = "app", .bytes = TSDB_APP_NAME_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, @@ -317,6 +318,24 @@ static const SSysDbTableSchema querySchema[] = { {.name = "sql", .bytes = TSDB_SHOW_SQL_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, }; +static const SSysDbTableSchema appSchema[] = { + {.name = "app_id", .bytes = 8, .type = TSDB_DATA_TYPE_UBIGINT}, + {.name = "ip", .bytes = TSDB_IPv4ADDR_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, + {.name = "pid", .bytes = 4, .type = TSDB_DATA_TYPE_INT}, + {.name = "name", .bytes = TSDB_APP_NAME_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR}, + {.name = "start_time", .bytes = 8 , .type = TSDB_DATA_TYPE_TIMESTAMP}, + {.name = "insert_req", .bytes = 8 , .type = TSDB_DATA_TYPE_UBIGINT}, + {.name = "insert_row", .bytes = 8 , .type = TSDB_DATA_TYPE_UBIGINT}, + {.name = "insert_time", .bytes = 8 , .type = TSDB_DATA_TYPE_UBIGINT}, + {.name = "insert_bytes", .bytes = 8 , .type = TSDB_DATA_TYPE_UBIGINT}, + {.name = "fetch_bytes", .bytes = 8 , .type = TSDB_DATA_TYPE_UBIGINT}, + {.name = "query_time", .bytes = 8 , .type = TSDB_DATA_TYPE_UBIGINT}, + {.name = "show_query", .bytes = 8 , .type = TSDB_DATA_TYPE_UBIGINT}, + {.name = "total_req", .bytes = 8 , .type = TSDB_DATA_TYPE_UBIGINT}, + {.name = "current_req", .bytes = 8 , .type = TSDB_DATA_TYPE_UBIGINT}, + {.name = "last_access", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP}, +}; + static const SSysTableMeta perfsMeta[] = { {TSDB_PERFS_TABLE_CONNECTIONS, connectionsSchema, tListLen(connectionsSchema)}, {TSDB_PERFS_TABLE_QUERIES, querySchema, tListLen(querySchema)}, @@ -327,6 +346,7 @@ static const SSysTableMeta perfsMeta[] = { {TSDB_PERFS_TABLE_TRANS, transSchema, tListLen(transSchema)}, {TSDB_PERFS_TABLE_SMAS, smaSchema, tListLen(smaSchema)}, {TSDB_PERFS_TABLE_STREAMS, streamSchema, tListLen(streamSchema)}, + {TSDB_PERFS_TABLE_APPS, appSchema, tListLen(appSchema)} }; void getInfosDbMeta(const SSysTableMeta** pInfosTableMeta, size_t* size) { diff --git a/source/common/src/tdatablock.c b/source/common/src/tdatablock.c index 712b4fcf42a63da1aaf6ce14200efab8864177e4..3c3d3e953dc567f7f9edb83d56d6ff48dca9edf6 100644 --- a/source/common/src/tdatablock.c +++ b/source/common/src/tdatablock.c @@ -1713,6 +1713,7 @@ int32_t buildSubmitReqFromDataBlock(SSubmitReq** pReq, const SArray* pDataBlocks } char* buildCtbNameByGroupId(const char* stbName, uint64_t groupId) { + ASSERT(stbName[0] != 0); SArray* tags = taosArrayInit(0, sizeof(void*)); SSmlKv* pTag = taosMemoryCalloc(1, sizeof(SSmlKv)); pTag->key = "group_id"; diff --git a/source/common/src/tglobal.c b/source/common/src/tglobal.c index fbb4f784254b87c7d3975325e24da210cefd09ed..7457fe7eb69434bcc2c766cba168869ee4174d1a 100644 --- a/source/common/src/tglobal.c +++ b/source/common/src/tglobal.c @@ -51,15 +51,16 @@ int32_t tsNumOfShmThreads = 1; int32_t tsNumOfRpcThreads = 1; int32_t tsNumOfCommitThreads = 2; int32_t tsNumOfTaskQueueThreads = 1; -int32_t tsNumOfMnodeQueryThreads = 1; +int32_t tsNumOfMnodeQueryThreads = 2; +int32_t tsNumOfMnodeFetchThreads = 1; int32_t tsNumOfMnodeReadThreads = 1; int32_t tsNumOfVnodeQueryThreads = 2; -int32_t tsNumOfVnodeFetchThreads = 2; +int32_t tsNumOfVnodeFetchThreads = 1; int32_t tsNumOfVnodeWriteThreads = 2; int32_t tsNumOfVnodeSyncThreads = 2; int32_t tsNumOfVnodeMergeThreads = 2; int32_t tsNumOfQnodeQueryThreads = 2; -int32_t tsNumOfQnodeFetchThreads = 2; +int32_t tsNumOfQnodeFetchThreads = 1; int32_t tsNumOfSnodeSharedThreads = 2; int32_t tsNumOfSnodeUniqueThreads = 2; @@ -417,8 +418,7 @@ static int32_t taosAddServerCfg(SConfig *pCfg) { tsNumOfVnodeQueryThreads = TMAX(tsNumOfVnodeQueryThreads, 1); if (cfgAddInt32(pCfg, "numOfVnodeQueryThreads", tsNumOfVnodeQueryThreads, 1, 1024, 0) != 0) return -1; - tsNumOfVnodeFetchThreads = tsNumOfCores / 2; - tsNumOfVnodeFetchThreads = TRANGE(tsNumOfVnodeFetchThreads, 2, 4); + tsNumOfVnodeFetchThreads = TRANGE(tsNumOfVnodeFetchThreads, 1, 1); if (cfgAddInt32(pCfg, "numOfVnodeFetchThreads", tsNumOfVnodeFetchThreads, 1, 1024, 0) != 0) return -1; tsNumOfVnodeWriteThreads = tsNumOfCores; @@ -437,8 +437,7 @@ static int32_t taosAddServerCfg(SConfig *pCfg) { tsNumOfQnodeQueryThreads = TMAX(tsNumOfQnodeQueryThreads, 1); if (cfgAddInt32(pCfg, "numOfQnodeQueryThreads", tsNumOfQnodeQueryThreads, 1, 1024, 0) != 0) return -1; - tsNumOfQnodeFetchThreads = tsNumOfCores / 2; - tsNumOfQnodeFetchThreads = TRANGE(tsNumOfQnodeFetchThreads, 2, 4); + tsNumOfQnodeFetchThreads = TRANGE(tsNumOfQnodeFetchThreads, 1, 1); if (cfgAddInt32(pCfg, "numOfQnodeFetchThreads", tsNumOfQnodeFetchThreads, 1, 1024, 0) != 0) return -1; tsNumOfSnodeSharedThreads = tsNumOfCores / 4; diff --git a/source/common/src/tmsg.c b/source/common/src/tmsg.c index 4749c86e4a53b6ba1048c4dd9e8d93bfca625e5c..2443ffbb7f24fb3a405fb5445cd14d95f662351d 100644 --- a/source/common/src/tmsg.c +++ b/source/common/src/tmsg.c @@ -191,13 +191,25 @@ static int32_t tSerializeSClientHbReq(SEncoder *pEncoder, const SClientHbReq *pR if (tEncodeSClientHbKey(pEncoder, &pReq->connKey) < 0) return -1; if (pReq->connKey.connType == CONN_TYPE__QUERY) { + if (tEncodeI64(pEncoder, pReq->app.appId) < 0) return -1; + if (tEncodeI32(pEncoder, pReq->app.pid) < 0) return -1; + if (tEncodeCStr(pEncoder, pReq->app.name) < 0) return -1; + if (tEncodeI64(pEncoder, pReq->app.startTime) < 0) return -1; + if (tEncodeU64(pEncoder, pReq->app.summary.numOfInsertsReq) < 0) return -1; + if (tEncodeU64(pEncoder, pReq->app.summary.numOfInsertRows) < 0) return -1; + if (tEncodeU64(pEncoder, pReq->app.summary.insertElapsedTime) < 0) return -1; + if (tEncodeU64(pEncoder, pReq->app.summary.insertBytes) < 0) return -1; + if (tEncodeU64(pEncoder, pReq->app.summary.fetchBytes) < 0) return -1; + if (tEncodeU64(pEncoder, pReq->app.summary.queryElapsedTime) < 0) return -1; + if (tEncodeU64(pEncoder, pReq->app.summary.numOfSlowQueries) < 0) return -1; + if (tEncodeU64(pEncoder, pReq->app.summary.totalRequests) < 0) return -1; + if (tEncodeU64(pEncoder, pReq->app.summary.currentRequests) < 0) return -1; + int32_t queryNum = 0; if (pReq->query) { queryNum = 1; if (tEncodeI32(pEncoder, queryNum) < 0) return -1; if (tEncodeU32(pEncoder, pReq->query->connId) < 0) return -1; - if (tEncodeI32(pEncoder, pReq->query->pid) < 0) return -1; - if (tEncodeCStr(pEncoder, pReq->query->app) < 0) return -1; int32_t num = taosArrayGetSize(pReq->query->queryDesc); if (tEncodeI32(pEncoder, num) < 0) return -1; @@ -209,7 +221,6 @@ static int32_t tSerializeSClientHbReq(SEncoder *pEncoder, const SClientHbReq *pR if (tEncodeI64(pEncoder, desc->useconds) < 0) return -1; if (tEncodeI64(pEncoder, desc->stime) < 0) return -1; if (tEncodeI64(pEncoder, desc->reqRid) < 0) return -1; - if (tEncodeI32(pEncoder, desc->pid) < 0) return -1; if (tEncodeI8(pEncoder, desc->stableQuery) < 0) return -1; if (tEncodeCStr(pEncoder, desc->fqdn) < 0) return -1; if (tEncodeI32(pEncoder, desc->subPlanNum) < 0) return -1; @@ -243,14 +254,26 @@ static int32_t tDeserializeSClientHbReq(SDecoder *pDecoder, SClientHbReq *pReq) if (tDecodeSClientHbKey(pDecoder, &pReq->connKey) < 0) return -1; if (pReq->connKey.connType == CONN_TYPE__QUERY) { + if (tDecodeI64(pDecoder, &pReq->app.appId) < 0) return -1; + if (tDecodeI32(pDecoder, &pReq->app.pid) < 0) return -1; + if (tDecodeCStrTo(pDecoder, pReq->app.name) < 0) return -1; + if (tDecodeI64(pDecoder, &pReq->app.startTime) < 0) return -1; + if (tDecodeU64(pDecoder, &pReq->app.summary.numOfInsertsReq) < 0) return -1; + if (tDecodeU64(pDecoder, &pReq->app.summary.numOfInsertRows) < 0) return -1; + if (tDecodeU64(pDecoder, &pReq->app.summary.insertElapsedTime) < 0) return -1; + if (tDecodeU64(pDecoder, &pReq->app.summary.insertBytes) < 0) return -1; + if (tDecodeU64(pDecoder, &pReq->app.summary.fetchBytes) < 0) return -1; + if (tDecodeU64(pDecoder, &pReq->app.summary.queryElapsedTime) < 0) return -1; + if (tDecodeU64(pDecoder, &pReq->app.summary.numOfSlowQueries) < 0) return -1; + if (tDecodeU64(pDecoder, &pReq->app.summary.totalRequests) < 0) return -1; + if (tDecodeU64(pDecoder, &pReq->app.summary.currentRequests) < 0) return -1; + int32_t queryNum = 0; if (tDecodeI32(pDecoder, &queryNum) < 0) return -1; if (queryNum) { pReq->query = taosMemoryCalloc(1, sizeof(*pReq->query)); if (NULL == pReq->query) return -1; if (tDecodeU32(pDecoder, &pReq->query->connId) < 0) return -1; - if (tDecodeI32(pDecoder, &pReq->query->pid) < 0) return -1; - if (tDecodeCStrTo(pDecoder, pReq->query->app) < 0) return -1; int32_t num = 0; if (tDecodeI32(pDecoder, &num) < 0) return -1; @@ -265,8 +288,7 @@ static int32_t tDeserializeSClientHbReq(SDecoder *pDecoder, SClientHbReq *pReq) if (tDecodeI64(pDecoder, &desc.useconds) < 0) return -1; if (tDecodeI64(pDecoder, &desc.stime) < 0) return -1; if (tDecodeI64(pDecoder, &desc.reqRid) < 0) return -1; - if (tDecodeI32(pDecoder, &desc.pid) < 0) return -1; - if (tDecodeI8(pDecoder, (int8_t*)&desc.stableQuery) < 0) return -1; + if (tDecodeI8(pDecoder, (int8_t *)&desc.stableQuery) < 0) return -1; if (tDecodeCStrTo(pDecoder, desc.fqdn) < 0) return -1; if (tDecodeI32(pDecoder, &desc.subPlanNum) < 0) return -1; @@ -474,8 +496,10 @@ int32_t tSerializeSMCreateStbReq(void *buf, int32_t bufLen, SMCreateStbReq *pReq if (tStartEncode(&encoder) < 0) return -1; if (tEncodeCStr(&encoder, pReq->name) < 0) return -1; if (tEncodeI8(&encoder, pReq->igExists) < 0) return -1; - if (tEncodeFloat(&encoder, pReq->xFilesFactor) < 0) return -1; - if (tEncodeI32(&encoder, pReq->delay) < 0) return -1; + if (tEncodeI64(&encoder, pReq->delay1) < 0) return -1; + if (tEncodeI64(&encoder, pReq->delay2) < 0) return -1; + if (tEncodeI64(&encoder, pReq->watermark1) < 0) return -1; + if (tEncodeI64(&encoder, pReq->watermark2) < 0) return -1; if (tEncodeI32(&encoder, pReq->ttl) < 0) return -1; if (tEncodeI32(&encoder, pReq->numOfColumns) < 0) return -1; if (tEncodeI32(&encoder, pReq->numOfTags) < 0) return -1; @@ -522,8 +546,10 @@ int32_t tDeserializeSMCreateStbReq(void *buf, int32_t bufLen, SMCreateStbReq *pR if (tStartDecode(&decoder) < 0) return -1; if (tDecodeCStrTo(&decoder, pReq->name) < 0) return -1; if (tDecodeI8(&decoder, &pReq->igExists) < 0) return -1; - if (tDecodeFloat(&decoder, &pReq->xFilesFactor) < 0) return -1; - if (tDecodeI32(&decoder, &pReq->delay) < 0) return -1; + if (tDecodeI64(&decoder, &pReq->delay1) < 0) return -1; + if (tDecodeI64(&decoder, &pReq->delay2) < 0) return -1; + if (tDecodeI64(&decoder, &pReq->watermark1) < 0) return -1; + if (tDecodeI64(&decoder, &pReq->watermark2) < 0) return -1; if (tDecodeI32(&decoder, &pReq->ttl) < 0) return -1; if (tDecodeI32(&decoder, &pReq->numOfColumns) < 0) return -1; if (tDecodeI32(&decoder, &pReq->numOfTags) < 0) return -1; @@ -3410,7 +3436,7 @@ int32_t tSerializeSKillConnReq(void *buf, int32_t bufLen, SKillConnReq *pReq) { tEncoderInit(&encoder, buf, bufLen); if (tStartEncode(&encoder) < 0) return -1; - if (tEncodeI32(&encoder, pReq->connId) < 0) return -1; + if (tEncodeU32(&encoder, pReq->connId) < 0) return -1; tEndEncode(&encoder); int32_t tlen = encoder.pos; @@ -3423,7 +3449,7 @@ int32_t tDeserializeSKillConnReq(void *buf, int32_t bufLen, SKillConnReq *pReq) tDecoderInit(&decoder, buf, bufLen); if (tStartDecode(&decoder) < 0) return -1; - if (tDecodeI32(&decoder, &pReq->connId) < 0) return -1; + if (tDecodeU32(&decoder, &pReq->connId) < 0) return -1; tEndDecode(&decoder); tDecoderClear(&decoder); diff --git a/source/dnode/mgmt/mgmt_mnode/inc/mmInt.h b/source/dnode/mgmt/mgmt_mnode/inc/mmInt.h index c5c3d76f1e6518fecbb98696f3b53d0ed566bf53..a4c37dd334984ff5e834463ec802b937ff557460 100644 --- a/source/dnode/mgmt/mgmt_mnode/inc/mmInt.h +++ b/source/dnode/mgmt/mgmt_mnode/inc/mmInt.h @@ -30,6 +30,7 @@ typedef struct SMnodeMgmt { const char *path; const char *name; SSingleWorker queryWorker; + SSingleWorker fetchWorker; SSingleWorker readWorker; SSingleWorker writeWorker; SSingleWorker syncWorker; @@ -57,6 +58,7 @@ int32_t mmPutMsgToWriteQueue(SMnodeMgmt *pMgmt, SRpcMsg *pMsg); int32_t mmPutMsgToSyncQueue(SMnodeMgmt *pMgmt, SRpcMsg *pMsg); int32_t mmPutMsgToReadQueue(SMnodeMgmt *pMgmt, SRpcMsg *pMsg); int32_t mmPutMsgToQueryQueue(SMnodeMgmt *pMgmt, SRpcMsg *pMsg); +int32_t mmPutMsgToFetchQueue(SMnodeMgmt *pMgmt, SRpcMsg *pMsg); int32_t mmPutMsgToMonitorQueue(SMnodeMgmt *pMgmt, SRpcMsg *pMsg); int32_t mmPutMsgToQueue(SMnodeMgmt *pMgmt, EQueueType qtype, SRpcMsg *pRpc); diff --git a/source/dnode/mgmt/mgmt_mnode/src/mmHandle.c b/source/dnode/mgmt/mgmt_mnode/src/mmHandle.c index d895b73fb0425955df6f6b1565d9a6369af6fa40..a845ae7b39d2582e52a6a01cfb222d848d3abc25 100644 --- a/source/dnode/mgmt/mgmt_mnode/src/mmHandle.c +++ b/source/dnode/mgmt/mgmt_mnode/src/mmHandle.c @@ -204,8 +204,8 @@ SArray *mmGetMsgHandles() { if (dmSetMgmtHandle(pArray, TDMT_VND_QUERY, mmPutMsgToQueryQueue, 1) == NULL) goto _OVER; if (dmSetMgmtHandle(pArray, TDMT_VND_QUERY_CONTINUE, mmPutMsgToQueryQueue, 1) == NULL) goto _OVER; - if (dmSetMgmtHandle(pArray, TDMT_VND_QUERY_HEARTBEAT, mmPutMsgToQueryQueue, 1) == NULL) goto _OVER; - if (dmSetMgmtHandle(pArray, TDMT_VND_FETCH, mmPutMsgToQueryQueue, 1) == NULL) goto _OVER; + if (dmSetMgmtHandle(pArray, TDMT_VND_QUERY_HEARTBEAT, mmPutMsgToFetchQueue, 1) == NULL) goto _OVER; + if (dmSetMgmtHandle(pArray, TDMT_VND_FETCH, mmPutMsgToFetchQueue, 1) == NULL) goto _OVER; if (dmSetMgmtHandle(pArray, TDMT_VND_CREATE_STB_RSP, mmPutMsgToWriteQueue, 0) == NULL) goto _OVER; if (dmSetMgmtHandle(pArray, TDMT_VND_ALTER_STB_RSP, mmPutMsgToWriteQueue, 0) == NULL) goto _OVER; if (dmSetMgmtHandle(pArray, TDMT_VND_DROP_STB_RSP, mmPutMsgToWriteQueue, 0) == NULL) goto _OVER; @@ -213,7 +213,7 @@ SArray *mmGetMsgHandles() { if (dmSetMgmtHandle(pArray, TDMT_VND_DROP_SMA_RSP, mmPutMsgToWriteQueue, 0) == NULL) goto _OVER; if (dmSetMgmtHandle(pArray, TDMT_VND_MQ_VG_CHANGE_RSP, mmPutMsgToWriteQueue, 0) == NULL) goto _OVER; if (dmSetMgmtHandle(pArray, TDMT_VND_MQ_VG_DELETE_RSP, mmPutMsgToWriteQueue, 0) == NULL) goto _OVER; - if (dmSetMgmtHandle(pArray, TDMT_VND_DROP_TASK, mmPutMsgToQueryQueue, 1) == NULL) goto _OVER; + if (dmSetMgmtHandle(pArray, TDMT_VND_DROP_TASK, mmPutMsgToFetchQueue, 1) == NULL) goto _OVER; if (dmSetMgmtHandle(pArray, TDMT_STREAM_TASK_DEPLOY_RSP, mmPutMsgToWriteQueue, 0) == NULL) goto _OVER; if (dmSetMgmtHandle(pArray, TDMT_VND_STREAM_TASK_DROP_RSP, mmPutMsgToWriteQueue, 0) == NULL) goto _OVER; if (dmSetMgmtHandle(pArray, TDMT_VND_ALTER_CONFIG_RSP, mmPutMsgToWriteQueue, 0) == NULL) goto _OVER; diff --git a/source/dnode/mgmt/mgmt_mnode/src/mmInt.c b/source/dnode/mgmt/mgmt_mnode/src/mmInt.c index b7124dfaa51a247f0af296e48bed300a38e20ba3..012e61d23964300df42b79fd52ed48c3af6ee4ce 100644 --- a/source/dnode/mgmt/mgmt_mnode/src/mmInt.c +++ b/source/dnode/mgmt/mgmt_mnode/src/mmInt.c @@ -148,6 +148,10 @@ static int32_t mmStart(SMnodeMgmt *pMgmt) { static void mmStop(SMnodeMgmt *pMgmt) { dDebug("mnode-mgmt start to stop"); + taosThreadRwlockWrlock(&pMgmt->lock); + pMgmt->stopped = 1; + taosThreadRwlockUnlock(&pMgmt->lock); + mndStop(pMgmt->pMnode); } diff --git a/source/dnode/mgmt/mgmt_mnode/src/mmWorker.c b/source/dnode/mgmt/mgmt_mnode/src/mmWorker.c index d0b9f3db8dab7b6f6ce79a6a8c299b47d268eef8..60c42d31f5c79363d6bde886e843e4df182f066d 100644 --- a/source/dnode/mgmt/mgmt_mnode/src/mmWorker.c +++ b/source/dnode/mgmt/mgmt_mnode/src/mmWorker.c @@ -123,6 +123,13 @@ int32_t mmPutMsgToQueryQueue(SMnodeMgmt *pMgmt, SRpcMsg *pMsg) { return mmPutMsgToWorker(pMgmt, &pMgmt->queryWorker, pMsg); } +int32_t mmPutMsgToFetchQueue(SMnodeMgmt *pMgmt, SRpcMsg *pMsg) { + pMsg->info.node = pMgmt->pMnode; + + return mmPutMsgToWorker(pMgmt, &pMgmt->fetchWorker, pMsg); +} + + int32_t mmPutMsgToMonitorQueue(SMnodeMgmt *pMgmt, SRpcMsg *pMsg) { return mmPutMsgToWorker(pMgmt, &pMgmt->monitorWorker, pMsg); } @@ -136,6 +143,9 @@ int32_t mmPutMsgToQueue(SMnodeMgmt *pMgmt, EQueueType qtype, SRpcMsg *pRpc) { case QUERY_QUEUE: pWorker = &pMgmt->queryWorker; break; + case FETCH_QUEUE: + pWorker = &pMgmt->fetchWorker; + break; case READ_QUEUE: pWorker = &pMgmt->readWorker; break; @@ -168,6 +178,18 @@ int32_t mmStartWorker(SMnodeMgmt *pMgmt) { return -1; } + SSingleWorkerCfg fCfg = { + .min = tsNumOfMnodeFetchThreads, + .max = tsNumOfMnodeFetchThreads, + .name = "mnode-fetch", + .fp = (FItem)mmProcessRpcMsg, + .param = pMgmt, + }; + if (tSingleWorkerInit(&pMgmt->fetchWorker, &fCfg) != 0) { + dError("failed to start mnode-fetch worker since %s", terrstr()); + return -1; + } + SSingleWorkerCfg rCfg = { .min = tsNumOfMnodeReadThreads, .max = tsNumOfMnodeReadThreads, @@ -221,13 +243,11 @@ int32_t mmStartWorker(SMnodeMgmt *pMgmt) { } void mmStopWorker(SMnodeMgmt *pMgmt) { - taosThreadRwlockWrlock(&pMgmt->lock); - pMgmt->stopped = 1; - taosThreadRwlockUnlock(&pMgmt->lock); while (pMgmt->refCount > 0) taosMsleep(10); tSingleWorkerCleanup(&pMgmt->monitorWorker); tSingleWorkerCleanup(&pMgmt->queryWorker); + tSingleWorkerCleanup(&pMgmt->fetchWorker); tSingleWorkerCleanup(&pMgmt->readWorker); tSingleWorkerCleanup(&pMgmt->writeWorker); tSingleWorkerCleanup(&pMgmt->syncWorker); diff --git a/source/dnode/mgmt/mgmt_vnode/src/vmHandle.c b/source/dnode/mgmt/mgmt_vnode/src/vmHandle.c index 9946185da6d9a65642dbf725a9456dfb03acbacc..f112727c0863849119468708af1ce2d2ffbf605c 100644 --- a/source/dnode/mgmt/mgmt_vnode/src/vmHandle.c +++ b/source/dnode/mgmt/mgmt_vnode/src/vmHandle.c @@ -228,7 +228,7 @@ int32_t vmProcessCreateVnodeReq(SVnodeMgmt *pMgmt, SRpcMsg *pMsg) { vmReleaseVnode(pMgmt, pVnode); terrno = TSDB_CODE_NODE_ALREADY_DEPLOYED; code = terrno; - goto _OVER; + return 0; } snprintf(path, TSDB_FILENAME_LEN, "vnode%svnode%d", TD_DIRSEP, vnodeCfg.vgId); diff --git a/source/dnode/mgmt/node_mgmt/src/dmTransport.c b/source/dnode/mgmt/node_mgmt/src/dmTransport.c index d416c4a557e7f6693dd82f14ea2faa01708614d0..660f512fc520c5657d12c0db11aae58085defc9f 100644 --- a/source/dnode/mgmt/node_mgmt/src/dmTransport.c +++ b/source/dnode/mgmt/node_mgmt/src/dmTransport.c @@ -21,21 +21,6 @@ static void dmSendRedirectRsp(SRpcMsg *pMsg, const SEpSet *pNewEpSet); static void dmSendRsp(SRpcMsg *pMsg); static void dmBuildMnodeRedirectRsp(SDnode *pDnode, SRpcMsg *pMsg); -static inline int32_t dmBuildNodeMsg(SRpcMsg *pMsg, SRpcMsg *pRpc) { - SRpcConnInfo *pConnInfo = &(pRpc->info.connInfo); - // if (IsReq(pRpc)) { - // terrno = TSDB_CODE_MND_NO_USER_FROM_CONN; - // dError("failed to build msg since %s, app:%p handle:%p", terrstr(), pRpc->info.ahandle, pRpc->info.handle); - // return -1; - //} - - memcpy(pMsg, pRpc, sizeof(SRpcMsg)); - memcpy(pMsg->conn.user, pConnInfo->user, TSDB_USER_LEN); - pMsg->conn.clientIp = pConnInfo->clientIp; - pMsg->conn.clientPort = pConnInfo->clientPort; - return 0; -} - int32_t dmProcessNodeMsg(SMgmtWrapper *pWrapper, SRpcMsg *pMsg) { NodeMsgFp msgFp = pWrapper->msgFps[TMSG_INDEX(pMsg->msgType)]; if (msgFp == NULL) { @@ -117,14 +102,10 @@ static void dmProcessRpcMsg(SDnode *pDnode, SRpcMsg *pRpc, SEpSet *pEpSet) { } pMsg = taosAllocateQitem(sizeof(SRpcMsg), RPC_QITEM); - if (pMsg == NULL) { - goto _OVER; - } - dTrace("msg:%p, is created, type:%s", pMsg, TMSG_INFO(pRpc->msgType)); + if (pMsg == NULL) goto _OVER; + memcpy(pMsg, pRpc, sizeof(SRpcMsg)); - if (dmBuildNodeMsg(pMsg, pRpc) != 0) { - goto _OVER; - } + dTrace("msg:%p, is created, type:%s handle:%p", pMsg, TMSG_INFO(pRpc->msgType), pMsg->info.handle); if (InParentProc(pWrapper)) { code = dmPutToProcCQueue(&pWrapper->proc, pMsg, DND_FUNC_REQ); diff --git a/source/dnode/mnode/impl/inc/mndDef.h b/source/dnode/mnode/impl/inc/mndDef.h index 4daeeaa9bf52dedbfc7c57d8a024aefca86e1eec..8963f6be39f14a1c3030edad555f35d3b0b4f0cf 100644 --- a/source/dnode/mnode/impl/inc/mndDef.h +++ b/source/dnode/mnode/impl/inc/mndDef.h @@ -124,7 +124,8 @@ typedef struct { int32_t lastErrorNo; tmsg_t lastMsgType; SEpSet lastEpset; - char dbname[TSDB_DB_FNAME_LEN]; + char dbname1[TSDB_DB_FNAME_LEN]; + char dbname2[TSDB_DB_FNAME_LEN]; int32_t startFunc; int32_t stopFunc; int32_t paramLen; diff --git a/source/dnode/mnode/impl/inc/mndInt.h b/source/dnode/mnode/impl/inc/mndInt.h index 45a6a04156645582b6a677c5c7065c71c3a73171..c810a0cbc75474154c9f19b5872656bb7fc2dfbe 100644 --- a/source/dnode/mnode/impl/inc/mndInt.h +++ b/source/dnode/mnode/impl/inc/mndInt.h @@ -69,7 +69,8 @@ typedef struct { } SShowMgmt; typedef struct { - SCacheObj *cache; + SCacheObj *connCache; + SCacheObj *appCache; } SProfileMgmt; typedef struct { diff --git a/source/dnode/mnode/impl/inc/mndTrans.h b/source/dnode/mnode/impl/inc/mndTrans.h index 0175e29a770d1d8521154ccb0778494529993b21..bc2d5c82b1410af66f758408b442912d02b2fd79 100644 --- a/source/dnode/mnode/impl/inc/mndTrans.h +++ b/source/dnode/mnode/impl/inc/mndTrans.h @@ -68,7 +68,7 @@ int32_t mndTransAppendRedoAction(STrans *pTrans, STransAction *pAction); int32_t mndTransAppendUndoAction(STrans *pTrans, STransAction *pAction); void mndTransSetRpcRsp(STrans *pTrans, void *pCont, int32_t contLen); void mndTransSetCb(STrans *pTrans, ETrnFunc startFunc, ETrnFunc stopFunc, void *param, int32_t paramLen); -void mndTransSetDbName(STrans *pTrans, const char *dbname); +void mndTransSetDbName(STrans *pTrans, const char *dbname1, const char *dbname2); void mndTransSetSerial(STrans *pTrans); int32_t mndTransPrepare(SMnode *pMnode, STrans *pTrans); diff --git a/source/dnode/mnode/impl/src/mndAuth.c b/source/dnode/mnode/impl/src/mndAuth.c index f036fc48f7b26a6dd71296b3f2bf7a765d911d99..d47fb9dfb4bfa18b337672ae01adedbe743615f8 100644 --- a/source/dnode/mnode/impl/src/mndAuth.c +++ b/source/dnode/mnode/impl/src/mndAuth.c @@ -56,7 +56,7 @@ static int32_t mndProcessAuthReq(SRpcMsg *pReq) { memcpy(authRsp.user, authReq.user, TSDB_USER_LEN); int32_t code = mndRetriveAuth(pReq->info.node, &authRsp); - mTrace("user:%s, auth req received, spi:%d encrypt:%d ruser:%s", pReq->conn.user, authRsp.spi, authRsp.encrypt, + mTrace("user:%s, auth req received, spi:%d encrypt:%d ruser:%s", pReq->info.conn.user, authRsp.spi, authRsp.encrypt, authRsp.user); int32_t contLen = tSerializeSAuthReq(NULL, 0, &authRsp); diff --git a/source/dnode/mnode/impl/src/mndBnode.c b/source/dnode/mnode/impl/src/mndBnode.c index aa908b983dd7530974b9f24b80a24c6387eb9f18..0de40ca67192d9394b44d13b9041b308ab7b3208 100644 --- a/source/dnode/mnode/impl/src/mndBnode.c +++ b/source/dnode/mnode/impl/src/mndBnode.c @@ -292,7 +292,7 @@ static int32_t mndProcessCreateBnodeReq(SRpcMsg *pReq) { goto _OVER; } - if (mndCheckOperAuth(pMnode, pReq->conn.user, MND_OPER_CREATE_BNODE) != 0) { + if (mndCheckOperAuth(pMnode, pReq->info.conn.user, MND_OPER_CREATE_BNODE) != 0) { goto _OVER; } @@ -394,7 +394,7 @@ static int32_t mndProcessDropBnodeReq(SRpcMsg *pReq) { goto _OVER; } - if (mndCheckOperAuth(pMnode, pReq->conn.user, MND_OPER_DROP_BNODE) != 0) { + if (mndCheckOperAuth(pMnode, pReq->info.conn.user, MND_OPER_DROP_BNODE) != 0) { goto _OVER; } diff --git a/source/dnode/mnode/impl/src/mndDb.c b/source/dnode/mnode/impl/src/mndDb.c index c12c6d5b4c015dbd6c3d8a78fc2d24002ad90ac7..345464399eef3b75153e9cc6d6634ade49225b00 100644 --- a/source/dnode/mnode/impl/src/mndDb.c +++ b/source/dnode/mnode/impl/src/mndDb.c @@ -477,7 +477,7 @@ static int32_t mndCreateDb(SMnode *pMnode, SRpcMsg *pReq, SCreateDbReq *pCreate, mDebug("trans:%d, used to create db:%s", pTrans->id, pCreate->db); - mndTransSetDbName(pTrans, dbObj.name); + mndTransSetDbName(pTrans, dbObj.name, NULL); if (mndSetCreateDbRedoLogs(pMnode, pTrans, &dbObj, pVgroups) != 0) goto _OVER; if (mndSetCreateDbUndoLogs(pMnode, pTrans, &dbObj, pVgroups) != 0) goto _OVER; if (mndSetCreateDbCommitLogs(pMnode, pTrans, &dbObj, pVgroups) != 0) goto _OVER; @@ -521,12 +521,12 @@ static int32_t mndProcessCreateDbReq(SRpcMsg *pReq) { goto _OVER; } - pUser = mndAcquireUser(pMnode, pReq->conn.user); + pUser = mndAcquireUser(pMnode, pReq->info.conn.user); if (pUser == NULL) { goto _OVER; } - if (mndCheckDbAuth(pMnode, pReq->conn.user, MND_OPER_CREATE_DB, NULL) != 0) { + if (mndCheckDbAuth(pMnode, pReq->info.conn.user, MND_OPER_CREATE_DB, NULL) != 0) { goto _OVER; } @@ -668,7 +668,7 @@ static int32_t mndAlterDb(SMnode *pMnode, SRpcMsg *pReq, SDbObj *pOld, SDbObj *p mDebug("trans:%d, used to alter db:%s", pTrans->id, pOld->name); int32_t code = -1; - mndTransSetDbName(pTrans, pOld->name); + mndTransSetDbName(pTrans, pOld->name, NULL); if (mndSetAlterDbRedoLogs(pMnode, pTrans, pOld, pNew) != 0) goto _OVER; if (mndSetAlterDbCommitLogs(pMnode, pTrans, pOld, pNew) != 0) goto _OVER; if (mndSetAlterDbRedoActions(pMnode, pTrans, pOld, pNew) != 0) goto _OVER; @@ -700,7 +700,7 @@ static int32_t mndProcessAlterDbReq(SRpcMsg *pReq) { goto _OVER; } - if (mndCheckDbAuth(pMnode, pReq->conn.user, MND_OPER_ALTER_DB, pDb) != 0) { + if (mndCheckDbAuth(pMnode, pReq->info.conn.user, MND_OPER_ALTER_DB, pDb) != 0) { goto _OVER; } @@ -921,7 +921,7 @@ static int32_t mndDropDb(SMnode *pMnode, SRpcMsg *pReq, SDbObj *pDb) { if (pTrans == NULL) goto _OVER; mDebug("trans:%d, used to drop db:%s", pTrans->id, pDb->name); - mndTransSetDbName(pTrans, pDb->name); + mndTransSetDbName(pTrans, pDb->name, NULL); if (mndSetDropDbRedoLogs(pMnode, pTrans, pDb) != 0) goto _OVER; if (mndSetDropDbCommitLogs(pMnode, pTrans, pDb) != 0) goto _OVER; @@ -980,7 +980,7 @@ static int32_t mndProcessDropDbReq(SRpcMsg *pReq) { } } - if (mndCheckDbAuth(pMnode, pReq->conn.user, MND_OPER_DROP_DB, pDb) != 0) { + if (mndCheckDbAuth(pMnode, pReq->info.conn.user, MND_OPER_DROP_DB, pDb) != 0) { goto _OVER; } @@ -1127,7 +1127,7 @@ static int32_t mndProcessUseDbReq(SRpcMsg *pReq) { mError("db:%s, failed to process use db req since %s", usedbReq.db, terrstr()); } else { - if (mndCheckDbAuth(pMnode, pReq->conn.user, MND_OPER_USE_DB, pDb) != 0) { + if (mndCheckDbAuth(pMnode, pReq->info.conn.user, MND_OPER_USE_DB, pDb) != 0) { goto _OVER; } @@ -1252,7 +1252,7 @@ static int32_t mndProcessCompactDbReq(SRpcMsg *pReq) { goto _OVER; } - if (mndCheckDbAuth(pMnode, pReq->conn.user, MND_OPER_COMPACT_DB, pDb) != 0) { + if (mndCheckDbAuth(pMnode, pReq->info.conn.user, MND_OPER_COMPACT_DB, pDb) != 0) { goto _OVER; } @@ -1391,11 +1391,13 @@ static void dumpDbInfoData(SSDataBlock *pBlock, SDbObj *pDb, SShowObj *pShow, in pColInfo = taosArrayGet(pBlock->pDataBlock, cols++); colDataAppend(pColInfo, rows, (const char *)strict, false); - pColInfo = taosArrayGet(pBlock->pDataBlock, cols++); - colDataAppend(pColInfo, rows, (const char *)&pDb->cfg.daysPerFile, false); - char tmp[128] = {0}; int32_t len = 0; + len = sprintf(&tmp[VARSTR_HEADER_SIZE], "%dm", pDb->cfg.daysPerFile); + varDataSetLen(tmp, len); + pColInfo = taosArrayGet(pBlock->pDataBlock, cols++); + colDataAppend(pColInfo, rows, (const char *)tmp, false); + if (pDb->cfg.daysToKeep0 > pDb->cfg.daysToKeep1 || pDb->cfg.daysToKeep0 > pDb->cfg.daysToKeep2) { len = sprintf(&tmp[VARSTR_HEADER_SIZE], "%dm,%dm,%dm", pDb->cfg.daysToKeep1, pDb->cfg.daysToKeep2, pDb->cfg.daysToKeep0); diff --git a/source/dnode/mnode/impl/src/mndDnode.c b/source/dnode/mnode/impl/src/mndDnode.c index 6d37143fc9f5bb5b871705c4c57ce180d75642d3..58c3570c3631ec0ceee76163d4afd50e047245ad 100644 --- a/source/dnode/mnode/impl/src/mndDnode.c +++ b/source/dnode/mnode/impl/src/mndDnode.c @@ -522,7 +522,7 @@ static int32_t mndProcessCreateDnodeReq(SRpcMsg *pReq) { goto _OVER; } - if (mndCheckOperAuth(pMnode, pReq->conn.user, MND_OPER_CREATE_DNODE) != 0) { + if (mndCheckOperAuth(pMnode, pReq->info.conn.user, MND_OPER_CREATE_DNODE) != 0) { goto _OVER; } @@ -623,7 +623,7 @@ static int32_t mndProcessDropDnodeReq(SRpcMsg *pReq) { } } - if (mndCheckOperAuth(pMnode, pReq->conn.user, MND_OPER_DROP_MNODE) != 0) { + if (mndCheckOperAuth(pMnode, pReq->info.conn.user, MND_OPER_DROP_MNODE) != 0) { goto _OVER; } @@ -761,11 +761,12 @@ static int32_t mndRetrieveDnodes(SRpcMsg *pReq, SShowObj *pShow, SSDataBlock *pB pColInfo = taosArrayGet(pBlock->pDataBlock, cols++); colDataAppend(pColInfo, numOfRows, (const char *)&pDnode->createdTime, false); - char b[tListLen(offlineReason) + VARSTR_HEADER_SIZE] = {0}; + char *b = taosMemoryCalloc(VARSTR_HEADER_SIZE + strlen(offlineReason[pDnode->offlineReason]) + 1, 1); STR_TO_VARSTR(b, online ? "" : offlineReason[pDnode->offlineReason]); pColInfo = taosArrayGet(pBlock->pDataBlock, cols++); colDataAppend(pColInfo, numOfRows, b, false); + taosMemoryFreeClear(b); numOfRows++; sdbRelease(pSdb, pDnode); diff --git a/source/dnode/mnode/impl/src/mndFunc.c b/source/dnode/mnode/impl/src/mndFunc.c index dfdc0a3c1abe282a75aa4b3e3523ec35daf792cc..832f1b8e68b0370d1c8ebaff890956d7f3566b0e 100644 --- a/source/dnode/mnode/impl/src/mndFunc.c +++ b/source/dnode/mnode/impl/src/mndFunc.c @@ -318,7 +318,7 @@ static int32_t mndProcessCreateFuncReq(SRpcMsg *pReq) { goto _OVER; } - if (mndCheckOperAuth(pMnode, pReq->conn.user, MND_OPER_CREATE_FUNC) != 0) { + if (mndCheckOperAuth(pMnode, pReq->info.conn.user, MND_OPER_CREATE_FUNC) != 0) { goto _OVER; } @@ -365,7 +365,7 @@ static int32_t mndProcessDropFuncReq(SRpcMsg *pReq) { } } - if (mndCheckOperAuth(pMnode, pReq->conn.user, MND_OPER_DROP_FUNC) != 0) { + if (mndCheckOperAuth(pMnode, pReq->info.conn.user, MND_OPER_DROP_FUNC) != 0) { goto _OVER; } diff --git a/source/dnode/mnode/impl/src/mndMain.c b/source/dnode/mnode/impl/src/mndMain.c index e6d3301401257fd349faf381cf18059bf320cd3d..fa6f45bdad4888e4296e77d98e29528584e30530 100644 --- a/source/dnode/mnode/impl/src/mndMain.c +++ b/source/dnode/mnode/impl/src/mndMain.c @@ -505,12 +505,12 @@ int32_t mndProcessSyncMsg(SRpcMsg *pMsg) { } static int32_t mndCheckMnodeState(SRpcMsg *pMsg) { + if (!IsReq(pMsg)) return 0; if (mndAcquireRpcRef(pMsg->info.node) == 0) return 0; if (IsReq(pMsg) && pMsg->msgType != TDMT_MND_MQ_TIMER && pMsg->msgType != TDMT_MND_TELEM_TIMER && pMsg->msgType != TDMT_MND_TRANS_TIMER) { - mError("msg:%p, failed to check mnode state since %s, app:%p type:%s", pMsg, terrstr(), pMsg->info.ahandle, - TMSG_INFO(pMsg->msgType)); + mError("msg:%p, failed to check mnode state since %s, type:%s", pMsg, terrstr(), TMSG_INFO(pMsg->msgType)); SEpSet epSet = {0}; mndGetMnodeEpSet(pMsg->info.node, &epSet); @@ -533,7 +533,8 @@ static int32_t mndCheckMsgContent(SRpcMsg *pMsg) { if (!IsReq(pMsg)) return 0; if (pMsg->contLen != 0 && pMsg->pCont != NULL) return 0; - mError("msg:%p, failed to check msg content, app:%p type:%s", pMsg, pMsg->info.ahandle, TMSG_INFO(pMsg->msgType)); + mError("msg:%p, failed to check msg, cont:%p contLen:%d, app:%p type:%s", pMsg, pMsg->pCont, pMsg->contLen, + pMsg->info.ahandle, TMSG_INFO(pMsg->msgType)); terrno = TSDB_CODE_INVALID_MSG_LEN; return -1; } @@ -558,7 +559,7 @@ int32_t mndProcessRpcMsg(SRpcMsg *pMsg) { if (code == TSDB_CODE_ACTION_IN_PROGRESS) { mTrace("msg:%p, won't response immediately since in progress", pMsg); } else if (code == 0) { - mTrace("msg:%p, successfully processed and response", pMsg); + mTrace("msg:%p, successfully processed", pMsg); } else { mError("msg:%p, failed to process since %s, app:%p type:%s", pMsg, terrstr(), pMsg->info.ahandle, TMSG_INFO(pMsg->msgType)); diff --git a/source/dnode/mnode/impl/src/mndMnode.c b/source/dnode/mnode/impl/src/mndMnode.c index e82c479cdd416ea87fdfc10cf506a3f33e2aec72..b40cd713e5b1bc002c9503744055998ef4e9b405 100644 --- a/source/dnode/mnode/impl/src/mndMnode.c +++ b/source/dnode/mnode/impl/src/mndMnode.c @@ -414,7 +414,7 @@ static int32_t mndProcessCreateMnodeReq(SRpcMsg *pReq) { goto _OVER; } - if (mndCheckOperAuth(pMnode, pReq->conn.user, MND_OPER_CREATE_MNODE) != 0) { + if (mndCheckOperAuth(pMnode, pReq->info.conn.user, MND_OPER_CREATE_MNODE) != 0) { goto _OVER; } @@ -621,7 +621,7 @@ static int32_t mndProcessDropMnodeReq(SRpcMsg *pReq) { goto _OVER; } - if (mndCheckOperAuth(pMnode, pReq->conn.user, MND_OPER_DROP_MNODE) != 0) { + if (mndCheckOperAuth(pMnode, pReq->info.conn.user, MND_OPER_DROP_MNODE) != 0) { goto _OVER; } @@ -742,7 +742,7 @@ static int32_t mndProcessAlterMnodeReq(SRpcMsg *pReq) { pMgmt->errCode = 0; pMgmt->transId = -1; tsem_wait(&pMgmt->syncSem); - mInfo("alter mnode sync result:%s", tstrerror(pMgmt->errCode)); + mInfo("alter mnode sync result:0x%x %s", pMgmt->errCode, tstrerror(pMgmt->errCode)); terrno = pMgmt->errCode; return pMgmt->errCode; } diff --git a/source/dnode/mnode/impl/src/mndProfile.c b/source/dnode/mnode/impl/src/mndProfile.c index 20d94e341c6155560f189ae28efb8e10547c8385..fd80679316ad4df7dbcac44e636bb29f42c1800d 100644 --- a/source/dnode/mnode/impl/src/mndProfile.c +++ b/source/dnode/mnode/impl/src/mndProfile.c @@ -43,6 +43,16 @@ typedef struct { SArray *pQueries; // SArray } SConnObj; +typedef struct { + int64_t appId; + uint32_t ip; + int32_t pid; + char name[TSDB_APP_NAME_LEN]; + int64_t startTime; + SAppClusterSummary summary; + int64_t lastAccessTimeMs; +} SAppObj; + static SConnObj *mndCreateConn(SMnode *pMnode, const char *user, int8_t connType, uint32_t ip, uint16_t port, int32_t pid, const char *app, int64_t startTime); static void mndFreeConn(SConnObj *pConn); @@ -57,14 +67,24 @@ static int32_t mndProcessKillConnReq(SRpcMsg *pReq); static int32_t mndRetrieveConns(SRpcMsg *pReq, SShowObj *pShow, SSDataBlock *pBlock, int32_t rows); static int32_t mndRetrieveQueries(SRpcMsg *pReq, SShowObj *pShow, SSDataBlock *pBlock, int32_t rows); static void mndCancelGetNextQuery(SMnode *pMnode, void *pIter); +static void mndFreeApp(SAppObj *pApp); +static int32_t mndRetrieveApps(SRpcMsg *pReq, SShowObj *pShow, SSDataBlock *pBlock, int32_t rows); +static void mndCancelGetNextApp(SMnode *pMnode, void *pIter); int32_t mndInitProfile(SMnode *pMnode) { SProfileMgmt *pMgmt = &pMnode->profileMgmt; // in ms - int32_t connCheckTime = tsShellActivityTimer * 2 * 1000; - pMgmt->cache = taosCacheInit(TSDB_DATA_TYPE_INT, connCheckTime, true, (__cache_free_fn_t)mndFreeConn, "conn"); - if (pMgmt->cache == NULL) { + int32_t checkTime = tsShellActivityTimer * 2 * 1000; + pMgmt->connCache = taosCacheInit(TSDB_DATA_TYPE_UINT, checkTime, true, (__cache_free_fn_t)mndFreeConn, "conn"); + if (pMgmt->connCache == NULL) { + terrno = TSDB_CODE_OUT_OF_MEMORY; + mError("failed to alloc profile cache since %s", terrstr()); + return -1; + } + + pMgmt->appCache = taosCacheInit(TSDB_DATA_TYPE_BIGINT, checkTime, true, (__cache_free_fn_t)mndFreeApp, "app"); + if (pMgmt->appCache == NULL) { terrno = TSDB_CODE_OUT_OF_MEMORY; mError("failed to alloc profile cache since %s", terrstr()); return -1; @@ -79,15 +99,22 @@ int32_t mndInitProfile(SMnode *pMnode) { mndAddShowFreeIterHandle(pMnode, TSDB_MGMT_TABLE_CONNS, mndCancelGetNextConn); mndAddShowRetrieveHandle(pMnode, TSDB_MGMT_TABLE_QUERIES, mndRetrieveQueries); mndAddShowFreeIterHandle(pMnode, TSDB_MGMT_TABLE_QUERIES, mndCancelGetNextQuery); + mndAddShowRetrieveHandle(pMnode, TSDB_MGMT_TABLE_APPS, mndRetrieveApps); + mndAddShowFreeIterHandle(pMnode, TSDB_MGMT_TABLE_APPS, mndCancelGetNextApp); return 0; } void mndCleanupProfile(SMnode *pMnode) { SProfileMgmt *pMgmt = &pMnode->profileMgmt; - if (pMgmt->cache != NULL) { - taosCacheCleanup(pMgmt->cache); - pMgmt->cache = NULL; + if (pMgmt->connCache != NULL) { + taosCacheCleanup(pMgmt->connCache); + pMgmt->connCache = NULL; + } + + if (pMgmt->appCache != NULL) { + taosCacheCleanup(pMgmt->appCache); + pMgmt->appCache = NULL; } } @@ -97,7 +124,7 @@ static SConnObj *mndCreateConn(SMnode *pMnode, const char *user, int8_t connType char connStr[255] = {0}; int32_t len = snprintf(connStr, sizeof(connStr), "%s%d%d%d%s", user, ip, port, pid, app); - int32_t connId = mndGenerateUid(connStr, len); + uint32_t connId = mndGenerateUid(connStr, len); if (startTime == 0) startTime = taosGetTimestampMs(); SConnObj connObj = {.id = connId, @@ -118,7 +145,7 @@ static SConnObj *mndCreateConn(SMnode *pMnode, const char *user, int8_t connType tstrncpy(connObj.app, app, TSDB_APP_NAME_LEN); int32_t keepTime = tsShellActivityTimer * 3; - SConnObj *pConn = taosCachePut(pMgmt->cache, &connId, sizeof(int32_t), &connObj, sizeof(connObj), keepTime * 1000); + SConnObj *pConn = taosCachePut(pMgmt->connCache, &connId, sizeof(uint32_t), &connObj, sizeof(connObj), keepTime * 1000); if (pConn == NULL) { terrno = TSDB_CODE_OUT_OF_MEMORY; mError("conn:%d, failed to put into cache since %s, user:%s", connId, user, terrstr()); @@ -133,21 +160,20 @@ static void mndFreeConn(SConnObj *pConn) { taosWLockLatch(&pConn->queryLock); taosArrayDestroyEx(pConn->pQueries, tFreeClientHbQueryDesc); taosWUnLockLatch(&pConn->queryLock); - + mTrace("conn:%u, is destroyed, data:%p", pConn->id, pConn); } static SConnObj *mndAcquireConn(SMnode *pMnode, uint32_t connId) { SProfileMgmt *pMgmt = &pMnode->profileMgmt; - SConnObj *pConn = taosCacheAcquireByKey(pMgmt->cache, &connId, sizeof(connId)); + SConnObj *pConn = taosCacheAcquireByKey(pMgmt->connCache, &connId, sizeof(connId)); if (pConn == NULL) { mDebug("conn:%u, already destroyed", connId); return NULL; } - int32_t keepTime = tsShellActivityTimer * 3; - pConn->lastAccessTimeMs = keepTime * 1000 + (uint64_t)taosGetTimestampMs(); + pConn->lastAccessTimeMs = taosGetTimestampMs(); mTrace("conn:%u, acquired from cache, data:%p", pConn->id, pConn); return pConn; @@ -158,7 +184,7 @@ static void mndReleaseConn(SMnode *pMnode, SConnObj *pConn) { mTrace("conn:%u, released from cache, data:%p", pConn->id, pConn); SProfileMgmt *pMgmt = &pMnode->profileMgmt; - taosCacheRelease(pMgmt->cache, (void **)&pConn, false); + taosCacheRelease(pMgmt->connCache, (void **)&pConn, false); } void *mndGetNextConn(SMnode *pMnode, SCacheIter *pIter) { @@ -194,15 +220,15 @@ static int32_t mndProcessConnectReq(SRpcMsg *pReq) { goto CONN_OVER; } - taosIp2String(pReq->conn.clientIp, ip); + taosIp2String(pReq->info.conn.clientIp, ip); - pUser = mndAcquireUser(pMnode, pReq->conn.user); + pUser = mndAcquireUser(pMnode, pReq->info.conn.user); if (pUser == NULL) { - mError("user:%s, failed to login while acquire user since %s", pReq->conn.user, terrstr()); + mError("user:%s, failed to login while acquire user since %s", pReq->info.conn.user, terrstr()); goto CONN_OVER; } if (0 != strncmp(connReq.passwd, pUser->pass, TSDB_PASSWORD_LEN - 1)) { - mError("user:%s, failed to auth while acquire user, input:%s", pReq->conn.user, connReq.passwd); + mError("user:%s, failed to auth while acquire user, input:%s", pReq->info.conn.user, connReq.passwd); code = TSDB_CODE_RPC_AUTH_FAILURE; goto CONN_OVER; } @@ -213,15 +239,16 @@ static int32_t mndProcessConnectReq(SRpcMsg *pReq) { pDb = mndAcquireDb(pMnode, db); if (pDb == NULL) { terrno = TSDB_CODE_MND_INVALID_DB; - mError("user:%s, failed to login from %s while use db:%s since %s", pReq->conn.user, ip, connReq.db, terrstr()); + mError("user:%s, failed to login from %s while use db:%s since %s", pReq->info.conn.user, ip, connReq.db, + terrstr()); goto CONN_OVER; } } - pConn = mndCreateConn(pMnode, pReq->conn.user, connReq.connType, pReq->conn.clientIp, pReq->conn.clientPort, - connReq.pid, connReq.app, connReq.startTime); + pConn = mndCreateConn(pMnode, pReq->info.conn.user, connReq.connType, pReq->info.conn.clientIp, + pReq->info.conn.clientPort, connReq.pid, connReq.app, connReq.startTime); if (pConn == NULL) { - mError("user:%s, failed to login from %s while create connection since %s", pReq->conn.user, ip, terrstr()); + mError("user:%s, failed to login from %s while create connection since %s", pReq->info.conn.user, ip, terrstr()); goto CONN_OVER; } @@ -246,7 +273,7 @@ static int32_t mndProcessConnectReq(SRpcMsg *pReq) { pReq->info.rspLen = contLen; pReq->info.rsp = pRsp; - mDebug("user:%s, login from %s:%d, conn:%u, app:%s", pReq->conn.user, ip, pConn->port, pConn->id, connReq.app); + mDebug("user:%s, login from %s:%d, conn:%u, app:%s", pReq->info.conn.user, ip, pConn->port, pConn->id, connReq.app); code = 0; @@ -275,6 +302,77 @@ static int32_t mndSaveQueryList(SConnObj *pConn, SQueryHbReqBasic *pBasic) { return TSDB_CODE_SUCCESS; } +static SAppObj *mndCreateApp(SMnode *pMnode, uint32_t clientIp, SAppHbReq* pReq) { + SProfileMgmt *pMgmt = &pMnode->profileMgmt; + + SAppObj app; + app.appId = pReq->appId; + app.ip = clientIp; + app.pid = pReq->pid; + strcpy(app.name, pReq->name); + app.startTime = pReq->startTime; + memcpy(&app.summary, &pReq->summary, sizeof(pReq->summary)); + app.lastAccessTimeMs = taosGetTimestampMs(); + + int32_t keepTime = tsShellActivityTimer * 3; + SAppObj *pApp = taosCachePut(pMgmt->appCache, &pReq->appId, sizeof(pReq->appId), &app, sizeof(app), keepTime * 1000); + if (pApp == NULL) { + terrno = TSDB_CODE_OUT_OF_MEMORY; + mError("failed to app %" PRIx64 " into cache since %s", pReq->appId, terrstr()); + return NULL; + } + + mTrace("app %" PRIx64 " is put into cache", pReq->appId); + return pApp; +} + +static void mndFreeApp(SAppObj *pApp) { + mTrace("app %" PRIx64 " is destroyed", pApp->appId); +} + + +static SAppObj *mndAcquireApp(SMnode *pMnode, int64_t appId) { + SProfileMgmt *pMgmt = &pMnode->profileMgmt; + + SAppObj *pApp = taosCacheAcquireByKey(pMgmt->appCache, &appId, sizeof(appId)); + if (pApp == NULL) { + mDebug("app %" PRIx64 " not in cache", appId); + return NULL; + } + + pApp->lastAccessTimeMs = (uint64_t)taosGetTimestampMs(); + + mTrace("app %" PRIx64 " acquired from cache", appId); + return pApp; +} + +static void mndReleaseApp(SMnode *pMnode, SAppObj *pApp) { + if (pApp == NULL) return; + mTrace("release app %" PRIx64 " to cache", pApp->appId); + + SProfileMgmt *pMgmt = &pMnode->profileMgmt; + taosCacheRelease(pMgmt->appCache, (void **)&pApp, false); +} + +void *mndGetNextApp(SMnode *pMnode, SCacheIter *pIter) { + SAppObj *pApp = NULL; + bool hasNext = taosCacheIterNext(pIter); + if (hasNext) { + size_t dataLen = 0; + pApp = taosCacheIterGetData(pIter, &dataLen); + } else { + taosCacheDestroyIter(pIter); + } + + return pApp; +} + +static void mndCancelGetNextApp(SMnode *pMnode, void *pIter) { + if (pIter != NULL) { + taosCacheDestroyIter(pIter); + } +} + static SClientHbRsp *mndMqHbBuildRsp(SMnode *pMnode, SClientHbReq *pReq) { #if 0 SClientHbRsp* pRsp = taosMemoryMalloc(sizeof(SClientHbRsp)); @@ -340,28 +438,51 @@ static SClientHbRsp *mndMqHbBuildRsp(SMnode *pMnode, SClientHbReq *pReq) { return NULL; } +static int32_t mndUpdateAppInfo(SMnode *pMnode, SClientHbReq *pHbReq, SRpcConnInfo *connInfo) { + SAppHbReq* pReq = &pHbReq->app; + SAppObj *pApp = mndAcquireApp(pMnode, pReq->appId); + if (pApp == NULL) { + pApp = mndCreateApp(pMnode, connInfo->clientIp, pReq); + if (pApp == NULL) { + mError("failed to create new app %" PRIx64 " since %s", pReq->appId, terrstr()); + return -1; + } else { + mDebug("a new app %" PRIx64 "created", pReq->appId); + mndReleaseApp(pMnode, pApp); + return TSDB_CODE_SUCCESS; + } + } + + memcpy(&pApp->summary, &pReq->summary, sizeof(pReq->summary)); + + mndReleaseApp(pMnode, pApp); + + return TSDB_CODE_SUCCESS; +} + static int32_t mndProcessQueryHeartBeat(SMnode *pMnode, SRpcMsg *pMsg, SClientHbReq *pHbReq, SClientHbBatchRsp *pBatchRsp) { SProfileMgmt *pMgmt = &pMnode->profileMgmt; SClientHbRsp hbRsp = {.connKey = pHbReq->connKey, .status = 0, .info = NULL, .query = NULL}; + SRpcConnInfo connInfo = pMsg->info.conn; + + mndUpdateAppInfo(pMnode, pHbReq, &connInfo); if (pHbReq->query) { SQueryHbReqBasic *pBasic = pHbReq->query; - SRpcConnInfo connInfo = pMsg->conn; - SConnObj *pConn = mndAcquireConn(pMnode, pBasic->connId); if (pConn == NULL) { pConn = mndCreateConn(pMnode, connInfo.user, CONN_TYPE__QUERY, connInfo.clientIp, connInfo.clientPort, - pBasic->pid, pBasic->app, 0); + pHbReq->app.pid, pHbReq->app.name, 0); if (pConn == NULL) { mError("user:%s, conn:%u is freed and failed to create new since %s", connInfo.user, pBasic->connId, terrstr()); return -1; } else { - mDebug("user:%s, conn:%u is freed and create a new conn:%u", connInfo.user, pBasic->connId, pConn->id); + mDebug("user:%s, conn:%u is freed, will create a new conn:%u", connInfo.user, pBasic->connId, pConn->id); } } - + SQueryHbRspBasic *rspBasic = taosMemoryCalloc(1, sizeof(SQueryHbRspBasic)); if (rspBasic == NULL) { mndReleaseConn(pMnode, pConn); @@ -386,7 +507,7 @@ static int32_t mndProcessQueryHeartBeat(SMnode *pMnode, SRpcMsg *pMsg, SClientHb mndGetMnodeEpSet(pMnode, &rspBasic->epSet); mndCreateQnodeList(pMnode, &rspBasic->pQnodeList, -1); - + mndReleaseConn(pMnode, pConn); hbRsp.query = rspBasic; @@ -500,7 +621,7 @@ static int32_t mndProcessKillQueryReq(SRpcMsg *pReq) { SMnode *pMnode = pReq->info.node; SProfileMgmt *pMgmt = &pMnode->profileMgmt; - SUserObj *pUser = mndAcquireUser(pMnode, pReq->conn.user); + SUserObj *pUser = mndAcquireUser(pMnode, pReq->info.conn.user); if (pUser == NULL) return 0; if (!pUser->superUser) { mndReleaseUser(pMnode, pUser); @@ -515,17 +636,28 @@ static int32_t mndProcessKillQueryReq(SRpcMsg *pReq) { return -1; } - mInfo("kill query msg is received, queryId:%d", killReq.queryId); + mInfo("kill query msg is received, queryId:%s", killReq.queryStrId); + int32_t connId = 0; + uint64_t queryId = 0; + char* p = strchr(killReq.queryStrId, ':'); + if (NULL == p) { + mError("invalid query id %s", killReq.queryStrId); + terrno = TSDB_CODE_MND_INVALID_QUERY_ID; + return -1; + } + *p = 0; + connId = taosStr2Int32(killReq.queryStrId, NULL, 16); + queryId = taosStr2UInt64(p + 1, NULL, 16); - SConnObj *pConn = taosCacheAcquireByKey(pMgmt->cache, &killReq.connId, sizeof(int32_t)); + SConnObj *pConn = taosCacheAcquireByKey(pMgmt->connCache, &connId, sizeof(int32_t)); if (pConn == NULL) { - mError("connId:%d, failed to kill queryId:%d, conn not exist", killReq.connId, killReq.queryId); + mError("connId:%x, failed to kill queryId:%" PRIx64 ", conn not exist", connId, queryId); terrno = TSDB_CODE_MND_INVALID_CONN_ID; return -1; } else { - mInfo("connId:%d, queryId:%d is killed by user:%s", killReq.connId, killReq.queryId, pReq->conn.user); - pConn->killId = killReq.queryId; - taosCacheRelease(pMgmt->cache, (void **)&pConn, false); + mInfo("connId:%x, queryId:%" PRIx64 " is killed by user:%s", connId, queryId, pReq->info.conn.user); + pConn->killId = queryId; + taosCacheRelease(pMgmt->connCache, (void **)&pConn, false); return 0; } } @@ -534,7 +666,7 @@ static int32_t mndProcessKillConnReq(SRpcMsg *pReq) { SMnode *pMnode = pReq->info.node; SProfileMgmt *pMgmt = &pMnode->profileMgmt; - SUserObj *pUser = mndAcquireUser(pMnode, pReq->conn.user); + SUserObj *pUser = mndAcquireUser(pMnode, pReq->info.conn.user); if (pUser == NULL) return 0; if (!pUser->superUser) { mndReleaseUser(pMnode, pUser); @@ -549,29 +681,29 @@ static int32_t mndProcessKillConnReq(SRpcMsg *pReq) { return -1; } - SConnObj *pConn = taosCacheAcquireByKey(pMgmt->cache, &killReq.connId, sizeof(int32_t)); + SConnObj *pConn = taosCacheAcquireByKey(pMgmt->connCache, &killReq.connId, sizeof(uint32_t)); if (pConn == NULL) { - mError("connId:%d, failed to kill connection, conn not exist", killReq.connId); + mError("connId:%u, failed to kill connection, conn not exist", killReq.connId); terrno = TSDB_CODE_MND_INVALID_CONN_ID; return -1; } else { - mInfo("connId:%d, is killed by user:%s", killReq.connId, pReq->conn.user); + mInfo("connId:%u, is killed by user:%s", killReq.connId, pReq->info.conn.user); pConn->killed = 1; - taosCacheRelease(pMgmt->cache, (void **)&pConn, false); + taosCacheRelease(pMgmt->connCache, (void **)&pConn, false); return TSDB_CODE_SUCCESS; } } static int32_t mndRetrieveConns(SRpcMsg *pReq, SShowObj *pShow, SSDataBlock *pBlock, int32_t rows) { - SMnode *pMnode = pReq->info.node; - SSdb *pSdb = pMnode->pSdb; - int32_t numOfRows = 0; - int32_t cols = 0; - SConnObj *pConn = NULL; - + SMnode *pMnode = pReq->info.node; + SSdb *pSdb = pMnode->pSdb; + int32_t numOfRows = 0; + int32_t cols = 0; + SConnObj *pConn = NULL; + if (pShow->pIter == NULL) { SProfileMgmt *pMgmt = &pMnode->profileMgmt; - pShow->pIter = taosCacheCreateIter(pMgmt->cache); + pShow->pIter = taosCacheCreateIter(pMgmt->connCache); } while (numOfRows < rows) { @@ -619,15 +751,15 @@ static int32_t mndRetrieveConns(SRpcMsg *pReq, SShowObj *pShow, SSDataBlock *pBl } static int32_t mndRetrieveQueries(SRpcMsg *pReq, SShowObj *pShow, SSDataBlock *pBlock, int32_t rows) { - SMnode *pMnode = pReq->info.node; - SSdb *pSdb = pMnode->pSdb; - int32_t numOfRows = 0; - int32_t cols = 0; - SConnObj *pConn = NULL; - + SMnode *pMnode = pReq->info.node; + SSdb *pSdb = pMnode->pSdb; + int32_t numOfRows = 0; + int32_t cols = 0; + SConnObj *pConn = NULL; + if (pShow->pIter == NULL) { SProfileMgmt *pMgmt = &pMnode->profileMgmt; - pShow->pIter = taosCacheCreateIter(pMgmt->cache); + pShow->pIter = taosCacheCreateIter(pMgmt->connCache); } while (numOfRows < rows) { @@ -645,7 +777,7 @@ static int32_t mndRetrieveQueries(SRpcMsg *pReq, SShowObj *pShow, SSDataBlock *p int32_t numOfQueries = taosArrayGetSize(pConn->pQueries); for (int32_t i = 0; i < numOfQueries; ++i) { - SQueryDesc* pQuery = taosArrayGet(pConn->pQueries, i); + SQueryDesc *pQuery = taosArrayGet(pConn->pQueries, i); cols = 0; char queryId[26 + VARSTR_HEADER_SIZE] = {0}; @@ -666,7 +798,7 @@ static int32_t mndRetrieveQueries(SRpcMsg *pReq, SShowObj *pShow, SSDataBlock *p colDataAppend(pColInfo, numOfRows, (const char *)app, false); pColInfo = taosArrayGet(pBlock->pDataBlock, cols++); - colDataAppend(pColInfo, numOfRows, (const char *)&pQuery->pid, false); + colDataAppend(pColInfo, numOfRows, (const char *)&pConn->pid, false); char user[TSDB_USER_LEN + VARSTR_HEADER_SIZE] = {0}; STR_TO_VARSTR(user, pConn->user); @@ -691,14 +823,14 @@ static int32_t mndRetrieveQueries(SRpcMsg *pReq, SShowObj *pShow, SSDataBlock *p pColInfo = taosArrayGet(pBlock->pDataBlock, cols++); colDataAppend(pColInfo, numOfRows, (const char *)&pQuery->subPlanNum, false); - char subStatus[TSDB_SHOW_SUBQUERY_LEN + VARSTR_HEADER_SIZE] = {0}; + char subStatus[TSDB_SHOW_SUBQUERY_LEN + VARSTR_HEADER_SIZE] = {0}; int32_t strSize = sizeof(subStatus); int32_t offset = VARSTR_HEADER_SIZE; for (int32_t i = 0; i < pQuery->subPlanNum && offset < strSize; ++i) { if (i) { offset += snprintf(subStatus + offset, strSize - offset - 1, ","); } - SQuerySubDesc* pDesc = taosArrayGet(pQuery->subDesc, i); + SQuerySubDesc *pDesc = taosArrayGet(pQuery->subDesc, i); offset += snprintf(subStatus + offset, strSize - offset - 1, "%" PRIu64 ":%s", pDesc->tid, pDesc->status); } varDataLen(subStatus) = strlen(&subStatus[VARSTR_HEADER_SIZE]); @@ -712,7 +844,7 @@ static int32_t mndRetrieveQueries(SRpcMsg *pReq, SShowObj *pShow, SSDataBlock *p numOfRows++; } - + taosRUnLockLatch(&pConn->queryLock); } @@ -720,6 +852,86 @@ static int32_t mndRetrieveQueries(SRpcMsg *pReq, SShowObj *pShow, SSDataBlock *p return numOfRows; } +static int32_t mndRetrieveApps(SRpcMsg *pReq, SShowObj *pShow, SSDataBlock *pBlock, int32_t rows) { + SMnode *pMnode = pReq->info.node; + SSdb *pSdb = pMnode->pSdb; + int32_t numOfRows = 0; + int32_t cols = 0; + SAppObj *pApp = NULL; + + if (pShow->pIter == NULL) { + SProfileMgmt *pMgmt = &pMnode->profileMgmt; + pShow->pIter = taosCacheCreateIter(pMgmt->appCache); + } + + while (numOfRows < rows) { + pApp = mndGetNextApp(pMnode, pShow->pIter); + if (pApp == NULL) { + pShow->pIter = NULL; + break; + } + + cols = 0; + + SColumnInfoData *pColInfo = taosArrayGet(pBlock->pDataBlock, cols++); + colDataAppend(pColInfo, numOfRows, (const char *)&pApp->appId, false); + + char ip[TSDB_IPv4ADDR_LEN + 6 + VARSTR_HEADER_SIZE] = {0}; + sprintf(&ip[VARSTR_HEADER_SIZE], "%s", taosIpStr(pApp->ip)); + varDataLen(ip) = strlen(&ip[VARSTR_HEADER_SIZE]); + pColInfo = taosArrayGet(pBlock->pDataBlock, cols++); + colDataAppend(pColInfo, numOfRows, (const char *)ip, false); + + pColInfo = taosArrayGet(pBlock->pDataBlock, cols++); + colDataAppend(pColInfo, numOfRows, (const char *)&pApp->pid, false); + + char name[TSDB_APP_NAME_LEN + 6 + VARSTR_HEADER_SIZE] = {0}; + sprintf(&name[VARSTR_HEADER_SIZE], "%s", pApp->name); + varDataLen(name) = strlen(&name[VARSTR_HEADER_SIZE]); + pColInfo = taosArrayGet(pBlock->pDataBlock, cols++); + colDataAppend(pColInfo, numOfRows, (const char *)name, false); + + pColInfo = taosArrayGet(pBlock->pDataBlock, cols++); + colDataAppend(pColInfo, numOfRows, (const char *)&pApp->startTime, false); + + pColInfo = taosArrayGet(pBlock->pDataBlock, cols++); + colDataAppend(pColInfo, numOfRows, (const char *)&pApp->summary.numOfInsertsReq, false); + + pColInfo = taosArrayGet(pBlock->pDataBlock, cols++); + colDataAppend(pColInfo, numOfRows, (const char *)&pApp->summary.numOfInsertRows, false); + + pColInfo = taosArrayGet(pBlock->pDataBlock, cols++); + colDataAppend(pColInfo, numOfRows, (const char *)&pApp->summary.insertElapsedTime, false); + + pColInfo = taosArrayGet(pBlock->pDataBlock, cols++); + colDataAppend(pColInfo, numOfRows, (const char *)&pApp->summary.insertBytes, false); + + pColInfo = taosArrayGet(pBlock->pDataBlock, cols++); + colDataAppend(pColInfo, numOfRows, (const char *)&pApp->summary.fetchBytes, false); + + pColInfo = taosArrayGet(pBlock->pDataBlock, cols++); + colDataAppend(pColInfo, numOfRows, (const char *)&pApp->summary.queryElapsedTime, false); + + pColInfo = taosArrayGet(pBlock->pDataBlock, cols++); + colDataAppend(pColInfo, numOfRows, (const char *)&pApp->summary.numOfSlowQueries, false); + + pColInfo = taosArrayGet(pBlock->pDataBlock, cols++); + colDataAppend(pColInfo, numOfRows, (const char *)&pApp->summary.totalRequests, false); + + pColInfo = taosArrayGet(pBlock->pDataBlock, cols++); + colDataAppend(pColInfo, numOfRows, (const char *)&pApp->summary.currentRequests, false); + + pColInfo = taosArrayGet(pBlock->pDataBlock, cols++); + colDataAppend(pColInfo, numOfRows, (const char *)&pApp->lastAccessTimeMs, false); + + numOfRows++; + } + + pShow->numOfRows += numOfRows; + return numOfRows; +} + + static void mndCancelGetNextQuery(SMnode *pMnode, void *pIter) { if (pIter != NULL) { taosCacheDestroyIter(pIter); @@ -728,5 +940,5 @@ static void mndCancelGetNextQuery(SMnode *pMnode, void *pIter) { int32_t mndGetNumOfConnections(SMnode *pMnode) { SProfileMgmt *pMgmt = &pMnode->profileMgmt; - return taosCacheGetNumOfObj(pMgmt->cache); + return taosCacheGetNumOfObj(pMgmt->connCache); } diff --git a/source/dnode/mnode/impl/src/mndQnode.c b/source/dnode/mnode/impl/src/mndQnode.c index 595287a3af9515fecc0e73345208606925152994..f5625f32d5a73af783dc0172ba949cc6d4532462 100644 --- a/source/dnode/mnode/impl/src/mndQnode.c +++ b/source/dnode/mnode/impl/src/mndQnode.c @@ -294,7 +294,7 @@ static int32_t mndProcessCreateQnodeReq(SRpcMsg *pReq) { goto _OVER; } - if (mndCheckOperAuth(pMnode, pReq->conn.user, MND_OPER_CREATE_QNODE) != 0) { + if (mndCheckOperAuth(pMnode, pReq->info.conn.user, MND_OPER_CREATE_QNODE) != 0) { goto _OVER; } @@ -396,7 +396,7 @@ static int32_t mndProcessDropQnodeReq(SRpcMsg *pReq) { goto _OVER; } - if (mndCheckOperAuth(pMnode, pReq->conn.user, MND_OPER_DROP_QNODE) != 0) { + if (mndCheckOperAuth(pMnode, pReq->info.conn.user, MND_OPER_DROP_QNODE) != 0) { goto _OVER; } diff --git a/source/dnode/mnode/impl/src/mndScheduler.c b/source/dnode/mnode/impl/src/mndScheduler.c index 6f8fc748c2693365d675bb96176827683698f2fe..39bb6798aa869277bd6e510829d3049e0bd02b87 100644 --- a/source/dnode/mnode/impl/src/mndScheduler.c +++ b/source/dnode/mnode/impl/src/mndScheduler.c @@ -105,7 +105,7 @@ int32_t mndPersistTaskDeployReq(STrans* pTrans, SStreamTask* pTask, const SEpSet int32_t size = encoder.pos; int32_t tlen = sizeof(SMsgHead) + size; tEncoderClear(&encoder); - void* buf = taosMemoryMalloc(tlen); + void* buf = taosMemoryCalloc(1, tlen); if (buf == NULL) { terrno = TSDB_CODE_OUT_OF_MEMORY; return -1; @@ -157,6 +157,7 @@ int32_t mndAddDispatcherToInnerTask(SMnode* pMnode, STrans* pTrans, SStreamObj* } sdbRelease(pMnode->pSdb, pDb); + memcpy(pTask->shuffleDispatcher.stbFullName, pStream->targetSTbName, TSDB_TABLE_FNAME_LEN); SArray* pVgs = pTask->shuffleDispatcher.dbInfo.pVgroupInfos; int32_t sz = taosArrayGetSize(pVgs); SArray* sinkLv = taosArrayGetP(pStream->tasks, 0); @@ -166,6 +167,7 @@ int32_t mndAddDispatcherToInnerTask(SMnode* pMnode, STrans* pTrans, SStreamObj* for (int32_t j = 0; j < sinkLvSize; j++) { SStreamTask* pLastLevelTask = taosArrayGetP(sinkLv, j); if (pLastLevelTask->nodeId == pVgInfo->vgId) { + ASSERT(pVgInfo->vgId > 0); pVgInfo->taskId = pLastLevelTask->taskId; ASSERT(pVgInfo->taskId != 0); break; diff --git a/source/dnode/mnode/impl/src/mndShow.c b/source/dnode/mnode/impl/src/mndShow.c index d312955202946ca530a4f583484177c9913b6ddf..fed6a5a721088fc39f24712f922ffe13e80e48d7 100644 --- a/source/dnode/mnode/impl/src/mndShow.c +++ b/source/dnode/mnode/impl/src/mndShow.c @@ -104,6 +104,8 @@ static int32_t convertToRetrieveType(char *name, int32_t len) { type = TSDB_MGMT_TABLE_TOPICS; } else if (strncasecmp(name, TSDB_PERFS_TABLE_STREAMS, len) == 0) { type = TSDB_MGMT_TABLE_STREAMS; + } else if (strncasecmp(name, TSDB_PERFS_TABLE_APPS, len) == 0) { + type = TSDB_MGMT_TABLE_APPS; } else { // ASSERT(0); } @@ -229,7 +231,7 @@ static int32_t mndProcessRetrieveSysTableReq(SRpcMsg *pReq) { mDebug("show:0x%" PRIx64 ", start retrieve data, type:%d", pShow->id, pShow->type); - // if (mndCheckShowAuth(pMnode, pReq->conn.user, pShow->type) != 0) return -1; + // if (mndCheckShowAuth(pMnode, pReq->info.conn.user, pShow->type) != 0) return -1; int32_t numOfCols = pShow->pMeta->numOfColumns; SSDataBlock *pBlock = taosMemoryCalloc(1, sizeof(SSDataBlock)); diff --git a/source/dnode/mnode/impl/src/mndSma.c b/source/dnode/mnode/impl/src/mndSma.c index 3eb5b03efd481e624980d5a0c2462bf4f6716ba3..c44fb03be23c7740759cf31233b4847756d1af08 100644 --- a/source/dnode/mnode/impl/src/mndSma.c +++ b/source/dnode/mnode/impl/src/mndSma.c @@ -530,7 +530,10 @@ static int32_t mndCreateSma(SMnode *pMnode, SRpcMsg *pReq, SMCreateSmaReq *pCrea smaObj.dbUid = pStb->dbUid; smaObj.intervalUnit = pCreate->intervalUnit; smaObj.slidingUnit = pCreate->slidingUnit; +#if 0 smaObj.timezone = pCreate->timezone; +#endif + smaObj.timezone = tsTimezone; // use timezone of server smaObj.interval = pCreate->interval; smaObj.offset = pCreate->offset; smaObj.sliding = pCreate->sliding; @@ -609,7 +612,7 @@ static int32_t mndCreateSma(SMnode *pMnode, SRpcMsg *pReq, SMCreateSmaReq *pCrea int32_t code = -1; STrans *pTrans = mndTransCreate(pMnode, TRN_POLICY_RETRY, TRN_CONFLICT_DB, pReq); if (pTrans == NULL) goto _OVER; - mndTransSetDbName(pTrans, pDb->name); + mndTransSetDbName(pTrans, pDb->name, NULL); mndTransSetSerial(pTrans); mDebug("trans:%d, used to create sma:%s", pTrans->id, pCreate->name); @@ -710,7 +713,7 @@ static int32_t mndProcessCreateSmaReq(SRpcMsg *pReq) { goto _OVER; } - if (mndCheckDbAuth(pMnode, pReq->conn.user, MND_OPER_WRITE_DB, pDb) != 0) { + if (mndCheckDbAuth(pMnode, pReq->info.conn.user, MND_OPER_WRITE_DB, pDb) != 0) { goto _OVER; } @@ -852,7 +855,7 @@ static int32_t mndDropSma(SMnode *pMnode, SRpcMsg *pReq, SDbObj *pDb, SSmaObj *p if (pTrans == NULL) goto _OVER; mDebug("trans:%d, used to drop sma:%s", pTrans->id, pSma->name); - mndTransSetDbName(pTrans, pDb->name); + mndTransSetDbName(pTrans, pDb->name, NULL); if (mndSetDropSmaRedoLogs(pMnode, pTrans, pSma) != 0) goto _OVER; if (mndSetDropSmaVgroupRedoLogs(pMnode, pTrans, pVgroup) != 0) goto _OVER; @@ -971,7 +974,7 @@ static int32_t mndProcessDropSmaReq(SRpcMsg *pReq) { goto _OVER; } - if (mndCheckDbAuth(pMnode, pReq->conn.user, MND_OPER_WRITE_DB, pDb) != 0) { + if (mndCheckDbAuth(pMnode, pReq->info.conn.user, MND_OPER_WRITE_DB, pDb) != 0) { goto _OVER; } diff --git a/source/dnode/mnode/impl/src/mndSnode.c b/source/dnode/mnode/impl/src/mndSnode.c index c84dc2f3ddc3d52b43c6a47fef3f8bf5c3f3d97b..0a99f356b194c94597d53b2f35fd011885d7b419 100644 --- a/source/dnode/mnode/impl/src/mndSnode.c +++ b/source/dnode/mnode/impl/src/mndSnode.c @@ -300,7 +300,7 @@ static int32_t mndProcessCreateSnodeReq(SRpcMsg *pReq) { goto _OVER; } - if (mndCheckOperAuth(pMnode, pReq->conn.user, MND_OPER_CREATE_SNODE) != 0) { + if (mndCheckOperAuth(pMnode, pReq->info.conn.user, MND_OPER_CREATE_SNODE) != 0) { goto _OVER; } @@ -404,7 +404,7 @@ static int32_t mndProcessDropSnodeReq(SRpcMsg *pReq) { goto _OVER; } - if (mndCheckOperAuth(pMnode, pReq->conn.user, MND_OPER_DROP_SNODE) != 0) { + if (mndCheckOperAuth(pMnode, pReq->info.conn.user, MND_OPER_DROP_SNODE) != 0) { goto _OVER; } diff --git a/source/dnode/mnode/impl/src/mndStb.c b/source/dnode/mnode/impl/src/mndStb.c index b83aa34a3a59d7e1b2e99ad998280215c995a77e..c37cdd188875bef4f16a4c5cd4827a99c761df56 100644 --- a/source/dnode/mnode/impl/src/mndStb.c +++ b/source/dnode/mnode/impl/src/mndStb.c @@ -669,8 +669,8 @@ int32_t mndBuildStbFromReq(SMnode *pMnode, SStbObj *pDst, SMCreateStbReq *pCreat pDst->tagVer = 1; pDst->colVer = 1; pDst->nextColId = 1; - pDst->xFilesFactor = pCreate->xFilesFactor; - pDst->delay = pCreate->delay; + // pDst->xFilesFactor = pCreate->xFilesFactor; + // pDst->delay = pCreate->delay; pDst->ttl = pCreate->ttl; pDst->numOfColumns = pCreate->numOfColumns; pDst->numOfTags = pCreate->numOfTags; @@ -758,7 +758,7 @@ _OVER: } int32_t mndAddStbToTrans(SMnode *pMnode, STrans *pTrans, SDbObj *pDb, SStbObj *pStb) { - mndTransSetDbName(pTrans, pDb->name); + mndTransSetDbName(pTrans, pDb->name, NULL); if (mndSetCreateStbRedoLogs(pMnode, pTrans, pDb, pStb) != 0) return -1; if (mndSetCreateStbUndoLogs(pMnode, pTrans, pDb, pStb) != 0) return -1; if (mndSetCreateStbCommitLogs(pMnode, pTrans, pDb, pStb) != 0) return -1; @@ -805,7 +805,7 @@ static int32_t mndProcessCreateStbReq(SRpcMsg *pReq) { goto _OVER; } - if (mndCheckDbAuth(pMnode, pReq->conn.user, MND_OPER_WRITE_DB, pDb) != 0) { + if (mndCheckDbAuth(pMnode, pReq->info.conn.user, MND_OPER_WRITE_DB, pDb) != 0) { goto _OVER; } @@ -1396,7 +1396,7 @@ static int32_t mndAlterStb(SMnode *pMnode, SRpcMsg *pReq, const SMAlterStbReq *p if (pTrans == NULL) goto _OVER; mDebug("trans:%d, used to alter stb:%s", pTrans->id, pAlter->name); - mndTransSetDbName(pTrans, pDb->name); + mndTransSetDbName(pTrans, pDb->name, NULL); if (needRsp) { void *pCont = NULL; @@ -1454,7 +1454,7 @@ static int32_t mndProcessAlterStbReq(SRpcMsg *pReq) { goto _OVER; } - if (mndCheckDbAuth(pMnode, pReq->conn.user, MND_OPER_WRITE_DB, pDb) != 0) { + if (mndCheckDbAuth(pMnode, pReq->info.conn.user, MND_OPER_WRITE_DB, pDb) != 0) { goto _OVER; } @@ -1537,7 +1537,7 @@ static int32_t mndDropStb(SMnode *pMnode, SRpcMsg *pReq, SDbObj *pDb, SStbObj *p if (pTrans == NULL) goto _OVER; mDebug("trans:%d, used to drop stb:%s", pTrans->id, pStb->name); - mndTransSetDbName(pTrans, pDb->name); + mndTransSetDbName(pTrans, pDb->name, NULL); if (mndSetDropStbRedoLogs(pMnode, pTrans, pStb) != 0) goto _OVER; if (mndSetDropStbCommitLogs(pMnode, pTrans, pStb) != 0) goto _OVER; @@ -1584,7 +1584,7 @@ static int32_t mndProcessDropStbReq(SRpcMsg *pReq) { goto _OVER; } - if (mndCheckDbAuth(pMnode, pReq->conn.user, MND_OPER_WRITE_DB, pDb) != 0) { + if (mndCheckDbAuth(pMnode, pReq->info.conn.user, MND_OPER_WRITE_DB, pDb) != 0) { goto _OVER; } diff --git a/source/dnode/mnode/impl/src/mndStream.c b/source/dnode/mnode/impl/src/mndStream.c index 3a42c694c849fd1d03350f87f4f2d086d589d7bf..d432256f1517ffb88bdab81e63e9aa3e94b6b07f 100644 --- a/source/dnode/mnode/impl/src/mndStream.c +++ b/source/dnode/mnode/impl/src/mndStream.c @@ -545,7 +545,7 @@ static int32_t mndCreateStream(SMnode *pMnode, SRpcMsg *pReq, SCMCreateStreamReq return -1; } - if (streamObj.targetSTbName[0] && mndCreateStbForStream(pMnode, pTrans, &streamObj, pReq->conn.user) < 0) { + if (streamObj.targetSTbName[0] && mndCreateStbForStream(pMnode, pTrans, &streamObj, pReq->info.conn.user) < 0) { mError("trans:%d, failed to create stb for stream since %s", pTrans->id, terrstr()); mndTransDrop(pTrans); return -1; @@ -602,7 +602,7 @@ static int32_t mndProcessCreateStreamReq(SRpcMsg *pReq) { goto _OVER; } - if (mndCheckDbAuth(pMnode, pReq->conn.user, MND_OPER_WRITE_DB, pDb) != 0) { + if (mndCheckDbAuth(pMnode, pReq->info.conn.user, MND_OPER_WRITE_DB, pDb) != 0) { goto _OVER; } #endif @@ -613,9 +613,9 @@ static int32_t mndProcessCreateStreamReq(SRpcMsg *pReq) { goto _OVER; } - mndTransSetDbName(pTrans, createStreamReq.sourceDB); + mndTransSetDbName(pTrans, createStreamReq.sourceDB, NULL); // TODO - /*mndTransSetDbName(pTrans, streamObj.targetDb);*/ + /*mndTransSetDbName(pTrans, streamObj.targetDb, NULL);*/ mDebug("trans:%d, used to create stream:%s", pTrans->id, createStreamReq.name); // build stream obj from request @@ -627,7 +627,7 @@ static int32_t mndProcessCreateStreamReq(SRpcMsg *pReq) { } // create stb for stream - if (mndCreateStbForStream(pMnode, pTrans, &streamObj, pReq->conn.user) < 0) { + if (mndCreateStbForStream(pMnode, pTrans, &streamObj, pReq->info.conn.user) < 0) { mError("trans:%d, failed to create stb for stream %s since %s", pTrans->id, createStreamReq.name, terrstr()); mndTransDrop(pTrans); goto _OVER; @@ -696,7 +696,7 @@ static int32_t mndProcessDropStreamReq(SRpcMsg *pReq) { #if 0 // todo check auth - pUser = mndAcquireUser(pMnode, pReq->conn.user); + pUser = mndAcquireUser(pMnode, pReq->info.conn.user); if (pUser == NULL) { goto DROP_STREAM_OVER; } diff --git a/source/dnode/mnode/impl/src/mndSubscribe.c b/source/dnode/mnode/impl/src/mndSubscribe.c index 65a5d22bec17c2a2a949677f0d54b0348f9300d4..d2b7a61e83ab5faa8ef412d1e4c9b638facfcf36 100644 --- a/source/dnode/mnode/impl/src/mndSubscribe.c +++ b/source/dnode/mnode/impl/src/mndSubscribe.c @@ -403,7 +403,7 @@ static int32_t mndDoRebalance(SMnode *pMnode, const SMqRebInputObj *pInput, SMqR static int32_t mndPersistRebResult(SMnode *pMnode, SRpcMsg *pMsg, const SMqRebOutputObj *pOutput) { STrans *pTrans = mndTransCreate(pMnode, TRN_POLICY_ROLLBACK, TRN_CONFLICT_DB_INSIDE, pMsg); - mndTransSetDbName(pTrans, pOutput->pSub->dbName); + mndTransSetDbName(pTrans, pOutput->pSub->dbName, NULL); if (pTrans == NULL) return -1; // make txn: diff --git a/source/dnode/mnode/impl/src/mndSync.c b/source/dnode/mnode/impl/src/mndSync.c index 5d28181f00454a6febfad05e9615ea2c14177cce..3e3850de1ac7e917de798fcfea38af6de04629b7 100644 --- a/source/dnode/mnode/impl/src/mndSync.c +++ b/source/dnode/mnode/impl/src/mndSync.c @@ -46,19 +46,21 @@ void mndSyncCommitMsg(struct SSyncFSM *pFsm, const SRpcMsg *pMsg, SFsmCbMeta cbM int32_t transId = sdbGetIdFromRaw(pMnode->pSdb, pRaw); pMgmt->errCode = cbMeta.code; - mDebug("trans:%d, is proposed, saved:%d code:0x%x, index:%" PRId64 " term:%" PRId64 " role:%s raw:%p", transId, - pMgmt->transId, cbMeta.code, cbMeta.index, cbMeta.term, syncStr(cbMeta.state), pRaw); + mDebug("trans:%d, is proposed, saved:%d code:0x%x, apply index:%" PRId64 " term:%" PRIu64 " config:%" PRId64 + " role:%s raw:%p", + transId, pMgmt->transId, cbMeta.code, cbMeta.index, cbMeta.term, cbMeta.lastConfigIndex, syncStr(cbMeta.state), + pRaw); if (pMgmt->errCode == 0) { sdbWriteWithoutFree(pMnode->pSdb, pRaw); - sdbSetApplyIndex(pMnode->pSdb, cbMeta.index); - sdbSetApplyTerm(pMnode->pSdb, cbMeta.term); + sdbSetApplyInfo(pMnode->pSdb, cbMeta.index, cbMeta.term, cbMeta.lastConfigIndex); } if (pMgmt->transId == transId) { if (pMgmt->errCode != 0) { mError("trans:%d, failed to propose since %s", transId, tstrerror(pMgmt->errCode)); } + pMgmt->transId = 0; tsem_post(&pMgmt->syncSem); } else { STrans *pTrans = mndAcquireTrans(pMnode, transId); @@ -67,32 +69,19 @@ void mndSyncCommitMsg(struct SSyncFSM *pFsm, const SRpcMsg *pMsg, SFsmCbMeta cbM mndReleaseTrans(pMnode, pTrans); } - if (cbMeta.index - sdbGetApplyIndex(pMnode->pSdb) > 100) { - SSnapshotMeta sMeta = {0}; - if (syncGetSnapshotMeta(pMnode->syncMgmt.sync, &sMeta) == 0) { - sdbSetCurConfig(pMnode->pSdb, sMeta.lastConfigIndex); - } - sdbWriteFile(pMnode->pSdb); - } + sdbWriteFile(pMnode->pSdb, SDB_WRITE_DELTA); } } int32_t mndSyncGetSnapshot(struct SSyncFSM *pFsm, SSnapshot *pSnapshot) { SMnode *pMnode = pFsm->data; - pSnapshot->lastApplyIndex = sdbGetCommitIndex(pMnode->pSdb); - pSnapshot->lastApplyTerm = sdbGetCommitTerm(pMnode->pSdb); - pSnapshot->lastConfigIndex = sdbGetCurConfig(pMnode->pSdb); + sdbGetCommitInfo(pMnode->pSdb, &pSnapshot->lastApplyIndex, &pSnapshot->lastApplyTerm, &pSnapshot->lastConfigIndex); return 0; } void mndRestoreFinish(struct SSyncFSM *pFsm) { SMnode *pMnode = pFsm->data; - SSnapshotMeta sMeta = {0}; - if (syncGetSnapshotMeta(pMnode->syncMgmt.sync, &sMeta) == 0) { - sdbSetCurConfig(pMnode->pSdb, sMeta.lastConfigIndex); - } - if (!pMnode->deploy) { mInfo("mnode sync restore finished, and will handle outstanding transactions"); mndTransPullup(pMnode); @@ -122,6 +111,7 @@ void mndReConfig(struct SSyncFSM *pFsm, const SRpcMsg *pMsg, SReConfigCbMeta cbM if (pMgmt->errCode != 0) { mError("trans:-1, failed to propose sync reconfig since %s", tstrerror(pMgmt->errCode)); } + pMgmt->transId = 0; tsem_post(&pMgmt->syncSem); } } @@ -258,13 +248,17 @@ void mndSyncStart(SMnode *pMnode) { mDebug("mnode sync started, id:%" PRId64 " standby:%d", pMgmt->sync, pMgmt->standby); } -void mndSyncStop(SMnode *pMnode) {} +void mndSyncStop(SMnode *pMnode) { + if (pMnode->syncMgmt.transId != 0) { + pMnode->syncMgmt.transId = 0; + tsem_post(&pMnode->syncMgmt.syncSem); + } +} bool mndIsMaster(SMnode *pMnode) { SSyncMgmt *pMgmt = &pMnode->syncMgmt; - ESyncState state = syncGetMyRole(pMgmt->sync); - if (state != TAOS_SYNC_STATE_LEADER) { + if (!syncIsReady(pMgmt->sync)) { terrno = TSDB_CODE_SYN_NOT_LEADER; return false; } diff --git a/source/dnode/mnode/impl/src/mndTopic.c b/source/dnode/mnode/impl/src/mndTopic.c index 4c2730ce9432bbc0827d23da7b9b58df8938580f..9632c04f4c4166d62ffaad64b8af572e3983aeec 100644 --- a/source/dnode/mnode/impl/src/mndTopic.c +++ b/source/dnode/mnode/impl/src/mndTopic.c @@ -474,7 +474,7 @@ static int32_t mndProcessCreateTopicReq(SRpcMsg *pReq) { goto _OVER; } - if (mndCheckDbAuth(pMnode, pReq->conn.user, MND_OPER_WRITE_DB, pDb) != 0) { + if (mndCheckDbAuth(pMnode, pReq->info.conn.user, MND_OPER_WRITE_DB, pDb) != 0) { goto _OVER; } @@ -566,7 +566,7 @@ static int32_t mndProcessDropTopicReq(SRpcMsg *pReq) { #endif STrans *pTrans = mndTransCreate(pMnode, TRN_POLICY_ROLLBACK, TRN_CONFLICT_DB_INSIDE, pReq); - mndTransSetDbName(pTrans, pTopic->db); + mndTransSetDbName(pTrans, pTopic->db, NULL); if (pTrans == NULL) { mError("topic:%s, failed to drop since %s", pTopic->name, terrstr()); return -1; diff --git a/source/dnode/mnode/impl/src/mndTrans.c b/source/dnode/mnode/impl/src/mndTrans.c index dafb2fa1d1151eff10d29af0d41a1160218139b9..31a955b030d01187e719a120afc16b163fb304e8 100644 --- a/source/dnode/mnode/impl/src/mndTrans.c +++ b/source/dnode/mnode/impl/src/mndTrans.c @@ -52,7 +52,7 @@ static bool mndTransPerformCommitActionStage(SMnode *pMnode, STrans *pTrans); static bool mndTransPerformCommitStage(SMnode *pMnode, STrans *pTrans); static bool mndTransPerformRollbackStage(SMnode *pMnode, STrans *pTrans); static bool mndTransPerfromFinishedStage(SMnode *pMnode, STrans *pTrans); -static bool mndCantExecuteTransAction(SMnode *pMnode) { return !pMnode->deploy && !mndIsMaster(pMnode); } +static bool mndCannotExecuteTransAction(SMnode *pMnode) { return !pMnode->deploy && !mndIsMaster(pMnode); } static void mndTransSendRpcRsp(SMnode *pMnode, STrans *pTrans); static int32_t mndProcessTransReq(SRpcMsg *pReq); @@ -122,7 +122,8 @@ static SSdbRaw *mndTransActionEncode(STrans *pTrans) { SDB_SET_INT8(pRaw, dataPos, pTrans->conflict, _OVER) SDB_SET_INT8(pRaw, dataPos, pTrans->exec, _OVER) SDB_SET_INT64(pRaw, dataPos, pTrans->createdTime, _OVER) - SDB_SET_BINARY(pRaw, dataPos, pTrans->dbname, TSDB_DB_FNAME_LEN, _OVER) + SDB_SET_BINARY(pRaw, dataPos, pTrans->dbname1, TSDB_DB_FNAME_LEN, _OVER) + SDB_SET_BINARY(pRaw, dataPos, pTrans->dbname2, TSDB_DB_FNAME_LEN, _OVER) SDB_SET_INT32(pRaw, dataPos, pTrans->redoActionPos, _OVER) int32_t redoActionNum = taosArrayGetSize(pTrans->redoActions); @@ -270,7 +271,8 @@ static SSdbRow *mndTransActionDecode(SSdbRaw *pRaw) { pTrans->conflict = conflict; pTrans->exec = exec; SDB_GET_INT64(pRaw, dataPos, &pTrans->createdTime, _OVER) - SDB_GET_BINARY(pRaw, dataPos, pTrans->dbname, TSDB_DB_FNAME_LEN, _OVER) + SDB_GET_BINARY(pRaw, dataPos, pTrans->dbname1, TSDB_DB_FNAME_LEN, _OVER) + SDB_GET_BINARY(pRaw, dataPos, pTrans->dbname2, TSDB_DB_FNAME_LEN, _OVER) SDB_GET_INT32(pRaw, dataPos, &pTrans->redoActionPos, _OVER) SDB_GET_INT32(pRaw, dataPos, &redoActionNum, _OVER) SDB_GET_INT32(pRaw, dataPos, &undoActionNum, _OVER) @@ -521,9 +523,10 @@ static int32_t mndTransActionUpdate(SSdb *pSdb, STrans *pOld, STrans *pNew) { } if (pOld->stage == TRN_STAGE_ROLLBACK) { - pOld->stage = TRN_STAGE_FINISHED; - mTrace("trans:%d, stage from rollback to finished since perform update action", pNew->id); + pOld->stage = TRN_STAGE_REDO_ACTION; + mTrace("trans:%d, stage from rollback to undoAction since perform update action", pNew->id); } + return 0; } @@ -649,7 +652,14 @@ void mndTransSetCb(STrans *pTrans, ETrnFunc startFunc, ETrnFunc stopFunc, void * pTrans->paramLen = paramLen; } -void mndTransSetDbName(STrans *pTrans, const char *dbname) { memcpy(pTrans->dbname, dbname, TSDB_DB_FNAME_LEN); } +void mndTransSetDbName(STrans *pTrans, const char *dbname1, const char *dbname2) { + if (dbname1 != NULL) { + memcpy(pTrans->dbname1, dbname1, TSDB_DB_FNAME_LEN); + } + if (dbname2 != NULL) { + memcpy(pTrans->dbname2, dbname2, TSDB_DB_FNAME_LEN); + } +} void mndTransSetSerial(STrans *pTrans) { pTrans->exec = TRN_EXEC_SERIAL; } @@ -661,7 +671,7 @@ static int32_t mndTransSync(SMnode *pMnode, STrans *pTrans) { } sdbSetRawStatus(pRaw, SDB_STATUS_READY); - mDebug("trans:%d, sync to other mnodes", pTrans->id); + mDebug("trans:%d, sync to other mnodes, stage:%s", pTrans->id, mndTransStr(pTrans->stage)); int32_t code = mndSyncPropose(pMnode, pRaw, pTrans->id); if (code != 0) { mError("trans:%d, failed to sync since %s", pTrans->id, terrstr()); @@ -674,6 +684,12 @@ static int32_t mndTransSync(SMnode *pMnode, STrans *pTrans) { return 0; } +static bool mndCheckDbConflict(const char *db, STrans *pTrans) { + if (db[0] == 0) return false; + if (strcmp(db, pTrans->dbname1) == 0 || strcmp(db, pTrans->dbname2) == 0) return true; + return false; +} + static bool mndCheckTransConflict(SMnode *pMnode, STrans *pNew) { STrans *pTrans = NULL; void *pIter = NULL; @@ -688,14 +704,21 @@ static bool mndCheckTransConflict(SMnode *pMnode, STrans *pNew) { if (pNew->conflict == TRN_CONFLICT_GLOBAL) conflict = true; if (pNew->conflict == TRN_CONFLICT_DB) { if (pTrans->conflict == TRN_CONFLICT_GLOBAL) conflict = true; - if (pTrans->conflict == TRN_CONFLICT_DB && strcmp(pNew->dbname, pTrans->dbname) == 0) conflict = true; - if (pTrans->conflict == TRN_CONFLICT_DB_INSIDE && strcmp(pNew->dbname, pTrans->dbname) == 0) conflict = true; + if (pTrans->conflict == TRN_CONFLICT_DB || pTrans->conflict == TRN_CONFLICT_DB_INSIDE) { + if (mndCheckDbConflict(pNew->dbname1, pTrans)) conflict = true; + if (mndCheckDbConflict(pNew->dbname2, pTrans)) conflict = true; + } } if (pNew->conflict == TRN_CONFLICT_DB_INSIDE) { if (pTrans->conflict == TRN_CONFLICT_GLOBAL) conflict = true; - if (pTrans->conflict == TRN_CONFLICT_DB && strcmp(pNew->dbname, pTrans->dbname) == 0) conflict = true; + if (pTrans->conflict == TRN_CONFLICT_DB) { + if (mndCheckDbConflict(pNew->dbname1, pTrans)) conflict = true; + if (mndCheckDbConflict(pNew->dbname2, pTrans)) conflict = true; + } } - mError("trans:%d, can't execute since conflict with trans:%d, db:%s", pNew->id, pTrans->id, pTrans->dbname); + + mError("trans:%d, can't execute since conflict with trans:%d, db1:%s db2:%s", pNew->id, pTrans->id, pTrans->dbname1, + pTrans->dbname2); sdbRelease(pMnode->pSdb, pTrans); } @@ -704,7 +727,7 @@ static bool mndCheckTransConflict(SMnode *pMnode, STrans *pNew) { int32_t mndTransPrepare(SMnode *pMnode, STrans *pTrans) { if (pTrans->conflict == TRN_CONFLICT_DB || pTrans->conflict == TRN_CONFLICT_DB_INSIDE) { - if (strlen(pTrans->dbname) == 0) { + if (strlen(pTrans->dbname1) == 0 && strlen(pTrans->dbname2) == 0) { terrno = TSDB_CODE_MND_TRANS_CONFLICT; mError("trans:%d, failed to prepare conflict db not set", pTrans->id); return -1; @@ -915,7 +938,7 @@ static int32_t mndTransWriteSingleLog(SMnode *pMnode, STrans *pTrans, STransActi static int32_t mndTransSendSingleMsg(SMnode *pMnode, STrans *pTrans, STransAction *pAction) { if (pAction->msgSent) return 0; - if (mndCantExecuteTransAction(pMnode)) return -1; + if (mndCannotExecuteTransAction(pMnode)) return -1; int64_t signature = pTrans->id; signature = (signature << 32); @@ -1115,7 +1138,7 @@ static int32_t mndTransExecuteRedoActionsSerial(SMnode *pMnode, STrans *pTrans) pTrans->lastEpset = pAction->epSet; } - if (mndCantExecuteTransAction(pMnode)) break; + if (mndCannotExecuteTransAction(pMnode)) break; if (code == 0) { pTrans->code = 0; @@ -1158,7 +1181,7 @@ static bool mndTransPerformRedoActionStage(SMnode *pMnode, STrans *pTrans) { code = mndTransExecuteRedoActions(pMnode, pTrans); } - if (mndCantExecuteTransAction(pMnode)) return false; + if (mndCannotExecuteTransAction(pMnode)) return false; if (code == 0) { pTrans->code = 0; @@ -1171,8 +1194,8 @@ static bool mndTransPerformRedoActionStage(SMnode *pMnode, STrans *pTrans) { } else { pTrans->code = terrno; if (pTrans->policy == TRN_POLICY_ROLLBACK) { - pTrans->stage = TRN_STAGE_UNDO_ACTION; - mError("trans:%d, stage from redoAction to undoAction since %s", pTrans->id, terrstr()); + pTrans->stage = TRN_STAGE_ROLLBACK; + mError("trans:%d, stage from redoAction to rollback since %s", pTrans->id, terrstr()); continueExec = true; } else { pTrans->failedTimes++; @@ -1185,7 +1208,7 @@ static bool mndTransPerformRedoActionStage(SMnode *pMnode, STrans *pTrans) { } static bool mndTransPerformCommitStage(SMnode *pMnode, STrans *pTrans) { - if (mndCantExecuteTransAction(pMnode)) return false; + if (mndCannotExecuteTransAction(pMnode)) return false; bool continueExec = true; int32_t code = mndTransCommit(pMnode, pTrans); @@ -1197,16 +1220,9 @@ static bool mndTransPerformCommitStage(SMnode *pMnode, STrans *pTrans) { continueExec = true; } else { pTrans->code = terrno; - if (pTrans->policy == TRN_POLICY_ROLLBACK) { - pTrans->stage = TRN_STAGE_UNDO_ACTION; - mError("trans:%d, stage from commit to undoAction since %s, failedTimes:%d", pTrans->id, terrstr(), - pTrans->failedTimes); - continueExec = true; - } else { - pTrans->failedTimes++; - mError("trans:%d, stage keep on commit since %s, failedTimes:%d", pTrans->id, terrstr(), pTrans->failedTimes); - continueExec = false; - } + pTrans->failedTimes++; + mError("trans:%d, stage keep on commit since %s, failedTimes:%d", pTrans->id, terrstr(), pTrans->failedTimes); + continueExec = false; } return continueExec; @@ -1235,11 +1251,9 @@ static bool mndTransPerformUndoActionStage(SMnode *pMnode, STrans *pTrans) { bool continueExec = true; int32_t code = mndTransExecuteUndoActions(pMnode, pTrans); - if (mndCantExecuteTransAction(pMnode)) return false; - if (code == 0) { - pTrans->stage = TRN_STAGE_ROLLBACK; - mDebug("trans:%d, stage from undoAction to rollback", pTrans->id); + pTrans->stage = TRN_STAGE_FINISHED; + mDebug("trans:%d, stage from undoAction to finished", pTrans->id); continueExec = true; } else if (code == TSDB_CODE_ACTION_IN_PROGRESS) { mDebug("trans:%d, stage keep on undoAction since %s", pTrans->id, tstrerror(code)); @@ -1254,14 +1268,14 @@ static bool mndTransPerformUndoActionStage(SMnode *pMnode, STrans *pTrans) { } static bool mndTransPerformRollbackStage(SMnode *pMnode, STrans *pTrans) { - if (mndCantExecuteTransAction(pMnode)) return false; + if (mndCannotExecuteTransAction(pMnode)) return false; bool continueExec = true; int32_t code = mndTransRollback(pMnode, pTrans); if (code == 0) { - pTrans->stage = TRN_STAGE_FINISHED; - mDebug("trans:%d, stage from rollback to finished", pTrans->id); + pTrans->stage = TRN_STAGE_UNDO_ACTION; + mDebug("trans:%d, stage from rollback to undoAction", pTrans->id); continueExec = true; } else { pTrans->failedTimes++; @@ -1309,12 +1323,12 @@ void mndTransExecute(SMnode *pMnode, STrans *pTrans) { case TRN_STAGE_COMMIT_ACTION: continueExec = mndTransPerformCommitActionStage(pMnode, pTrans); break; - case TRN_STAGE_UNDO_ACTION: - continueExec = mndTransPerformUndoActionStage(pMnode, pTrans); - break; case TRN_STAGE_ROLLBACK: continueExec = mndTransPerformRollbackStage(pMnode, pTrans); break; + case TRN_STAGE_UNDO_ACTION: + continueExec = mndTransPerformUndoActionStage(pMnode, pTrans); + break; case TRN_STAGE_FINISHED: continueExec = mndTransPerfromFinishedStage(pMnode, pTrans); break; @@ -1369,7 +1383,7 @@ static int32_t mndProcessKillTransReq(SRpcMsg *pReq) { mInfo("trans:%d, start to kill", killReq.transId); - if (mndCheckOperAuth(pMnode, pReq->conn.user, MND_OPER_KILL_TRANS) != 0) { + if (mndCheckOperAuth(pMnode, pReq->info.conn.user, MND_OPER_KILL_TRANS) != 0) { goto _OVER; } @@ -1416,11 +1430,8 @@ void mndTransPullup(SMnode *pMnode) { mndReleaseTrans(pMnode, pTrans); } - SSnapshotMeta sMeta = {0}; - if (syncGetSnapshotMeta(pMnode->syncMgmt.sync, &sMeta) == 0) { - sdbSetCurConfig(pMnode->pSdb, sMeta.lastConfigIndex); - } - sdbWriteFile(pMnode->pSdb); + // todo, set to SDB_WRITE_DELTA + sdbWriteFile(pMnode->pSdb, 0); taosArrayDestroy(pArray); } @@ -1449,10 +1460,15 @@ static int32_t mndRetrieveTrans(SRpcMsg *pReq, SShowObj *pShow, SSDataBlock *pBl pColInfo = taosArrayGet(pBlock->pDataBlock, cols++); colDataAppend(pColInfo, numOfRows, (const char *)stage, false); - char dbname[TSDB_DB_NAME_LEN + VARSTR_HEADER_SIZE] = {0}; - STR_WITH_MAXSIZE_TO_VARSTR(dbname, mndGetDbStr(pTrans->dbname), pShow->pMeta->pSchemas[cols].bytes); + char dbname1[TSDB_DB_NAME_LEN + VARSTR_HEADER_SIZE] = {0}; + STR_WITH_MAXSIZE_TO_VARSTR(dbname1, mndGetDbStr(pTrans->dbname1), pShow->pMeta->pSchemas[cols].bytes); + pColInfo = taosArrayGet(pBlock->pDataBlock, cols++); + colDataAppend(pColInfo, numOfRows, (const char *)dbname1, false); + + char dbname2[TSDB_DB_NAME_LEN + VARSTR_HEADER_SIZE] = {0}; + STR_WITH_MAXSIZE_TO_VARSTR(dbname2, mndGetDbStr(pTrans->dbname2), pShow->pMeta->pSchemas[cols].bytes); pColInfo = taosArrayGet(pBlock->pDataBlock, cols++); - colDataAppend(pColInfo, numOfRows, (const char *)dbname, false); + colDataAppend(pColInfo, numOfRows, (const char *)dbname2, false); pColInfo = taosArrayGet(pBlock->pDataBlock, cols++); colDataAppend(pColInfo, numOfRows, (const char *)&pTrans->failedTimes, false); diff --git a/source/dnode/mnode/impl/src/mndUser.c b/source/dnode/mnode/impl/src/mndUser.c index 9590823106124ecd8595052d31a608adf1226a46..eb0a818a604da23b995c0493b2e96281d5077e53 100644 --- a/source/dnode/mnode/impl/src/mndUser.c +++ b/source/dnode/mnode/impl/src/mndUser.c @@ -354,13 +354,13 @@ static int32_t mndProcessCreateUserReq(SRpcMsg *pReq) { goto _OVER; } - pOperUser = mndAcquireUser(pMnode, pReq->conn.user); + pOperUser = mndAcquireUser(pMnode, pReq->info.conn.user); if (pOperUser == NULL) { terrno = TSDB_CODE_MND_NO_USER_FROM_CONN; goto _OVER; } - if (mndCheckOperAuth(pMnode, pReq->conn.user, MND_OPER_CREATE_USER) != 0) { + if (mndCheckOperAuth(pMnode, pReq->info.conn.user, MND_OPER_CREATE_USER) != 0) { goto _OVER; } @@ -460,7 +460,7 @@ static int32_t mndProcessAlterUserReq(SRpcMsg *pReq) { goto _OVER; } - pOperUser = mndAcquireUser(pMnode, pReq->conn.user); + pOperUser = mndAcquireUser(pMnode, pReq->info.conn.user); if (pOperUser == NULL) { terrno = TSDB_CODE_MND_NO_USER_FROM_CONN; goto _OVER; @@ -643,7 +643,7 @@ static int32_t mndProcessDropUserReq(SRpcMsg *pReq) { goto _OVER; } - if (mndCheckOperAuth(pMnode, pReq->conn.user, MND_OPER_DROP_USER) != 0) { + if (mndCheckOperAuth(pMnode, pReq->info.conn.user, MND_OPER_DROP_USER) != 0) { goto _OVER; } diff --git a/source/dnode/mnode/impl/src/mndVgroup.c b/source/dnode/mnode/impl/src/mndVgroup.c index 755f4ef0b2afd3b63c634b94ef369eb8ed18d1f9..94ddbcd409c8ef4b1ad6bf8d97bda63edb63ea92 100644 --- a/source/dnode/mnode/impl/src/mndVgroup.c +++ b/source/dnode/mnode/impl/src/mndVgroup.c @@ -1058,7 +1058,7 @@ int32_t mndSetMoveVgroupsInfoToTrans(SMnode *pMnode, STrans *pTrans, int32_t del static int32_t mndAddIncVgroupReplicaToTrans(SMnode *pMnode, STrans *pTrans, SDbObj *pDb, SVgObj *pVgroup, int32_t newDnodeId) { - mDebug("vgId:%d, will add 1 vnode, replica:%d, dnode:%d", pVgroup->vgId, pVgroup->replica, newDnodeId); + mDebug("vgId:%d, will add 1 vnode, replica:%d dnode:%d", pVgroup->vgId, pVgroup->replica, newDnodeId); SVnodeGid *pGid = &pVgroup->vnodeGid[pVgroup->replica]; pVgroup->replica++; @@ -1074,7 +1074,7 @@ static int32_t mndAddIncVgroupReplicaToTrans(SMnode *pMnode, STrans *pTrans, SDb static int32_t mndAddDecVgroupReplicaFromTrans(SMnode *pMnode, STrans *pTrans, SDbObj *pDb, SVgObj *pVgroup, int32_t delDnodeId) { - mDebug("vgId:%d, will remove 1 vnode, replica:%d, dnode:%d", pVgroup->vgId, pVgroup->replica, delDnodeId); + mDebug("vgId:%d, will remove 1 vnode, replica:%d dnode:%d", pVgroup->vgId, pVgroup->replica, delDnodeId); SVnodeGid *pGid = NULL; SVnodeGid delGid = {0}; @@ -1116,7 +1116,8 @@ static int32_t mndRedistributeVgroup(SMnode *pMnode, SRpcMsg *pReq, SDbObj *pDb, memcpy(&newVg, pVgroup, sizeof(SVgObj)); mInfo("vgId:%d, vgroup info before redistribute, replica:%d", newVg.vgId, newVg.replica); for (int32_t i = 0; i < newVg.replica; ++i) { - mInfo("vgId:%d, vnode:%d dnode:%d", newVg.vgId, i, newVg.vnodeGid[i].dnodeId); + mInfo("vgId:%d, vnode:%d dnode:%d role:%s", newVg.vgId, i, newVg.vnodeGid[i].dnodeId, + syncStr(newVg.vnodeGid[i].role)); } if (pNew1 != NULL && pOld1 != NULL) { @@ -1198,9 +1199,9 @@ static int32_t mndProcessRedistributeVgroupMsg(SRpcMsg *pReq) { goto _OVER; } - mInfo("vgId:%d, start to redistribute to dnode %d:%d:%d", req.vgId, req.dnodeId1, req.dnodeId2, req.dnodeId3); + mInfo("vgId:%d, start to redistribute vgroup to dnode %d:%d:%d", req.vgId, req.dnodeId1, req.dnodeId2, req.dnodeId3); - if (mndCheckOperAuth(pMnode, pReq->conn.user, MND_OPER_REDISTRIBUTE_VGROUP) != 0) goto _OVER; + if (mndCheckOperAuth(pMnode, pReq->info.conn.user, MND_OPER_REDISTRIBUTE_VGROUP) != 0) goto _OVER; pVgroup = mndAcquireVgroup(pMnode, req.vgId); if (pVgroup == NULL) goto _OVER; @@ -1500,7 +1501,7 @@ static int32_t mndProcessSplitVgroupMsg(SRpcMsg *pReq) { pDb = mndAcquireDb(pMnode, pVgroup->dbName); if (pDb == NULL) goto _OVER; - if (mndCheckOperAuth(pMnode, pReq->conn.user, MND_OPER_SPLIT_VGROUP) != 0) goto _OVER; + if (mndCheckOperAuth(pMnode, pReq->info.conn.user, MND_OPER_SPLIT_VGROUP) != 0) goto _OVER; code = mndSplitVgroup(pMnode, pReq, pDb, pVgroup); if (code == 0) code = TSDB_CODE_ACTION_IN_PROGRESS; @@ -1624,7 +1625,7 @@ static int32_t mndProcessBalanceVgroupMsg(SRpcMsg *pReq) { mDebug("start to balance vgroup"); - if (mndCheckOperAuth(pMnode, pReq->conn.user, MND_OPER_BALANCE_VGROUP) != 0) goto _OVER; + if (mndCheckOperAuth(pMnode, pReq->info.conn.user, MND_OPER_BALANCE_VGROUP) != 0) goto _OVER; while (1) { SDnodeObj *pDnode = NULL; diff --git a/source/dnode/mnode/impl/test/profile/profile.cpp b/source/dnode/mnode/impl/test/profile/profile.cpp index 039700ae9e5d8bfeab1eadb39aa83e4b38827445..e784a41d6f08ac98a5a12710a012e85efd43a2fc 100644 --- a/source/dnode/mnode/impl/test/profile/profile.cpp +++ b/source/dnode/mnode/impl/test/profile/profile.cpp @@ -295,8 +295,7 @@ TEST_F(MndTestProfile, 07_KillQueryMsg) { TEST_F(MndTestProfile, 08_KillQueryMsg_InvalidConn) { SKillQueryReq killReq = {0}; - killReq.connId = 2345; - killReq.queryId = 2345; + strcpy(killReq.queryStrId, "2345:2345"); int32_t contLen = tSerializeSKillQueryReq(NULL, 0, &killReq); void* pReq = rpcMallocCont(contLen); diff --git a/source/dnode/mnode/impl/test/sdb/sdbTest.cpp b/source/dnode/mnode/impl/test/sdb/sdbTest.cpp index 43be55dd1de822d098475747a7b5b6452f379058..bc118ee26e5996e28e6ef3c1380a71a7f868dd3f 100644 --- a/source/dnode/mnode/impl/test/sdb/sdbTest.cpp +++ b/source/dnode/mnode/impl/test/sdb/sdbTest.cpp @@ -493,8 +493,11 @@ TEST_F(MndTestSdb, 01_Write_Str) { ASSERT_EQ(sdbGetSize(pSdb, SDB_USER), 2); ASSERT_EQ(sdbGetMaxId(pSdb, SDB_USER), -1); ASSERT_EQ(sdbGetTableVer(pSdb, SDB_USER), 2); - sdbSetApplyIndex(pSdb, -1); - ASSERT_EQ(sdbGetApplyIndex(pSdb), -1); + sdbSetApplyInfo(pSdb, -1, -1, -1); + // int64_t index, config; + // int64_t term; + // sdbGetCommitInfo(pSdb, &index, &term, &config); + // ASSERT_EQ(index, -1); ASSERT_EQ(mnode.insertTimes, 2); ASSERT_EQ(mnode.deleteTimes, 0); @@ -700,11 +703,12 @@ TEST_F(MndTestSdb, 01_Write_Str) { } // write version - sdbSetApplyIndex(pSdb, 0); - sdbSetApplyIndex(pSdb, 1); - ASSERT_EQ(sdbGetApplyIndex(pSdb), 1); - ASSERT_EQ(sdbWriteFile(pSdb), 0); - ASSERT_EQ(sdbWriteFile(pSdb), 0); + sdbSetApplyInfo(pSdb, 0, 0, 0); + sdbSetApplyInfo(pSdb, 1, 0, 0); + // sdbGetApplyInfo(pSdb, &index, &term, &config); + // ASSERT_EQ(index, 1); + ASSERT_EQ(sdbWriteFile(pSdb, 0), 0); + ASSERT_EQ(sdbWriteFile(pSdb, 0), 0); sdbCleanup(pSdb); ASSERT_EQ(mnode.insertTimes, 7); @@ -772,7 +776,11 @@ TEST_F(MndTestSdb, 01_Read_Str) { ASSERT_EQ(sdbGetSize(pSdb, SDB_USER), 2); ASSERT_EQ(sdbGetMaxId(pSdb, SDB_USER), -1); ASSERT_EQ(sdbGetTableVer(pSdb, SDB_USER), 5); - ASSERT_EQ(sdbGetApplyIndex(pSdb), 1); + + int64_t index, config; + int64_t term; + sdbGetCommitInfo(pSdb, &index, &term, &config); + ASSERT_EQ(index, 1); ASSERT_EQ(mnode.insertTimes, 4); ASSERT_EQ(mnode.deleteTimes, 0); diff --git a/source/dnode/mnode/impl/test/trans/trans2.cpp b/source/dnode/mnode/impl/test/trans/trans2.cpp index 022c82c73d66ab39f9cf07aeb34642278018722d..aee8aa27488da6dc6b8b0cbd06a3b34741e66a18 100644 --- a/source/dnode/mnode/impl/test/trans/trans2.cpp +++ b/source/dnode/mnode/impl/test/trans/trans2.cpp @@ -128,7 +128,7 @@ class MndTestTrans2 : public ::testing::Test { mndTransSetCb(pTrans, TRANS_START_FUNC_TEST, TRANS_STOP_FUNC_TEST, param, strlen(param) + 1); if (pDb != NULL) { - mndTransSetDbName(pTrans, pDb->name); + mndTransSetDbName(pTrans, pDb->name, NULL); } int32_t code = mndTransPrepare(pMnode, pTrans); @@ -201,7 +201,7 @@ class MndTestTrans2 : public ::testing::Test { } if (pDb != NULL) { - mndTransSetDbName(pTrans, pDb->name); + mndTransSetDbName(pTrans, pDb->name, NULL); } int32_t code = mndTransPrepare(pMnode, pTrans); diff --git a/source/dnode/mnode/sdb/inc/sdb.h b/source/dnode/mnode/sdb/inc/sdb.h index 8536c451b7d9c33bfc9fc9abfbe499af60718cde..1bd09aef63b802f36c03973f0dc8b1de2b336433 100644 --- a/source/dnode/mnode/sdb/inc/sdb.h +++ b/source/dnode/mnode/sdb/inc/sdb.h @@ -37,6 +37,8 @@ extern "C" { #define mTrace(...) { if (mDebugFlag & DEBUG_TRACE) { taosPrintLog("MND ", DEBUG_TRACE, mDebugFlag, __VA_ARGS__); }} // clang-format on +#define SDB_WRITE_DELTA 100 + #define SDB_GET_VAL(pData, dataPos, val, pos, func, type) \ { \ if (func(pRaw, dataPos, val) != 0) { \ @@ -169,11 +171,12 @@ typedef struct SSdb { SWal *pWal; char *currDir; char *tmpDir; - int64_t lastCommitVer; - int64_t lastCommitTerm; - int64_t curVer; - int64_t curTerm; - int64_t curConfig; + int64_t commitIndex; + int64_t commitTerm; + int64_t commitConfig; + int64_t applyIndex; + int64_t applyTerm; + int64_t applyConfig; int64_t tableVer[SDB_MAX]; int64_t maxId[SDB_MAX]; EKeyType keyTypes[SDB_MAX]; @@ -257,7 +260,7 @@ int32_t sdbReadFile(SSdb *pSdb); * @param pSdb The sdb object. * @return int32_t 0 for success, -1 for failure. */ -int32_t sdbWriteFile(SSdb *pSdb); +int32_t sdbWriteFile(SSdb *pSdb, int32_t delta); /** * @brief Parse and write raw data to sdb, then free the pRaw object @@ -361,14 +364,8 @@ int64_t sdbGetTableVer(SSdb *pSdb, ESdbType type); * @param index The update value of the apply index. * @return int32_t The current index of sdb */ -void sdbSetApplyIndex(SSdb *pSdb, int64_t index); -void sdbSetApplyTerm(SSdb *pSdb, int64_t term); -void sdbSetCurConfig(SSdb *pSdb, int64_t config); -int64_t sdbGetApplyIndex(SSdb *pSdb); -int64_t sdbGetApplyTerm(SSdb *pSdb); -int64_t sdbGetCommitIndex(SSdb *pSdb); -int64_t sdbGetCommitTerm(SSdb *pSdb); -int64_t sdbGetCurConfig(SSdb *pSdb); +void sdbSetApplyInfo(SSdb *pSdb, int64_t index, int64_t term, int64_t config); +void sdbGetCommitInfo(SSdb *pSdb, int64_t *index, int64_t *term, int64_t *config); SSdbRaw *sdbAllocRaw(ESdbType type, int8_t sver, int32_t dataLen); void sdbFreeRaw(SSdbRaw *pRaw); diff --git a/source/dnode/mnode/sdb/src/sdb.c b/source/dnode/mnode/sdb/src/sdb.c index 39e9c7588833c0185129ef0dea51f8bc8d665df4..c44f1670c30bc17336d35a72d2e5fdc59156af38 100644 --- a/source/dnode/mnode/sdb/src/sdb.c +++ b/source/dnode/mnode/sdb/src/sdb.c @@ -53,11 +53,12 @@ SSdb *sdbInit(SSdbOpt *pOption) { } pSdb->pWal = pOption->pWal; - pSdb->curVer = -1; - pSdb->curTerm = -1; - pSdb->lastCommitVer = -1; - pSdb->lastCommitTerm = -1; - pSdb->curConfig = -1; + pSdb->applyIndex = -1; + pSdb->applyTerm = -1; + pSdb->applyConfig = -1; + pSdb->commitIndex = -1; + pSdb->commitTerm = -1; + pSdb->commitConfig = -1; pSdb->pMnode = pOption->pMnode; taosThreadMutexInit(&pSdb->filelock, NULL); mDebug("sdb init successfully"); @@ -67,7 +68,7 @@ SSdb *sdbInit(SSdbOpt *pOption) { void sdbCleanup(SSdb *pSdb) { mDebug("start to cleanup sdb"); - sdbWriteFile(pSdb); + sdbWriteFile(pSdb, 0); if (pSdb->currDir != NULL) { taosMemoryFreeClear(pSdb->currDir); @@ -159,23 +160,20 @@ static int32_t sdbCreateDir(SSdb *pSdb) { return 0; } -void sdbSetApplyIndex(SSdb *pSdb, int64_t index) { pSdb->curVer = index; } - -void sdbSetApplyTerm(SSdb *pSdb, int64_t term) { pSdb->curTerm = term; } - -void sdbSetCurConfig(SSdb *pSdb, int64_t config) { - if (pSdb->curConfig != config) { - mDebug("mnode sync config set from %" PRId64 " to %" PRId64, pSdb->curConfig, config); - pSdb->curConfig = config; - } +void sdbSetApplyInfo(SSdb *pSdb, int64_t index, int64_t term, int64_t config) { + mTrace("mnode apply info changed, from index:%" PRId64 " term:%" PRId64 " config:%" PRId64 ", to index:%" PRId64 + " term:%" PRId64 " config:%" PRId64, + pSdb->applyIndex, pSdb->applyTerm, pSdb->applyConfig, index, term, config); + pSdb->applyIndex = index; + pSdb->applyTerm = term; + pSdb->applyConfig = config; } -int64_t sdbGetApplyIndex(SSdb *pSdb) { return pSdb->curVer; } - -int64_t sdbGetApplyTerm(SSdb *pSdb) { return pSdb->curTerm; } - -int64_t sdbGetCommitIndex(SSdb *pSdb) { return pSdb->lastCommitVer; } - -int64_t sdbGetCommitTerm(SSdb *pSdb) { return pSdb->lastCommitTerm; } - -int64_t sdbGetCurConfig(SSdb *pSdb) { return pSdb->curConfig; } \ No newline at end of file +void sdbGetCommitInfo(SSdb *pSdb, int64_t *index, int64_t *term, int64_t *config) { + *index = pSdb->commitIndex; + *term = pSdb->commitTerm; + *config = pSdb->commitConfig; + mTrace("mnode current info, apply index:%" PRId64 " term:%" PRId64 " config:%" PRId64 ", commit index:%" PRId64 + " term:%" PRId64 " config:%" PRId64, + pSdb->applyIndex, pSdb->applyTerm, pSdb->applyConfig, *index, *term, *config); +} diff --git a/source/dnode/mnode/sdb/src/sdbFile.c b/source/dnode/mnode/sdb/src/sdbFile.c index 34f5d6f23d541710cbc475f433d1c8d5ff8b4c3d..0f4e1276c1612ad0f1707057d9e052eb307fe452 100644 --- a/source/dnode/mnode/sdb/src/sdbFile.c +++ b/source/dnode/mnode/sdb/src/sdbFile.c @@ -67,10 +67,12 @@ static void sdbResetData(SSdb *pSdb) { mDebug("sdb:%s is reset", sdbTableName(i)); } - pSdb->curVer = -1; - pSdb->curTerm = -1; - pSdb->lastCommitVer = -1; - pSdb->lastCommitTerm = -1; + pSdb->applyIndex = -1; + pSdb->applyTerm = -1; + pSdb->applyConfig = -1; + pSdb->commitIndex = -1; + pSdb->commitTerm = -1; + pSdb->commitConfig = -1; mDebug("sdb reset successfully"); } @@ -90,7 +92,7 @@ static int32_t sdbReadFileHead(SSdb *pSdb, TdFilePtr pFile) { return -1; } - ret = taosReadFile(pFile, &pSdb->curVer, sizeof(int64_t)); + ret = taosReadFile(pFile, &pSdb->applyIndex, sizeof(int64_t)); if (ret < 0) { terrno = TAOS_SYSTEM_ERROR(errno); return -1; @@ -100,7 +102,7 @@ static int32_t sdbReadFileHead(SSdb *pSdb, TdFilePtr pFile) { return -1; } - ret = taosReadFile(pFile, &pSdb->curTerm, sizeof(int64_t)); + ret = taosReadFile(pFile, &pSdb->applyTerm, sizeof(int64_t)); if (ret < 0) { terrno = TAOS_SYSTEM_ERROR(errno); return -1; @@ -110,7 +112,7 @@ static int32_t sdbReadFileHead(SSdb *pSdb, TdFilePtr pFile) { return -1; } - ret = taosReadFile(pFile, &pSdb->curConfig, sizeof(int64_t)); + ret = taosReadFile(pFile, &pSdb->applyConfig, sizeof(int64_t)); if (ret < 0) { terrno = TAOS_SYSTEM_ERROR(errno); return -1; @@ -173,17 +175,17 @@ static int32_t sdbWriteFileHead(SSdb *pSdb, TdFilePtr pFile) { return -1; } - if (taosWriteFile(pFile, &pSdb->curVer, sizeof(int64_t)) != sizeof(int64_t)) { + if (taosWriteFile(pFile, &pSdb->applyIndex, sizeof(int64_t)) != sizeof(int64_t)) { terrno = TAOS_SYSTEM_ERROR(errno); return -1; } - if (taosWriteFile(pFile, &pSdb->curTerm, sizeof(int64_t)) != sizeof(int64_t)) { + if (taosWriteFile(pFile, &pSdb->applyTerm, sizeof(int64_t)) != sizeof(int64_t)) { terrno = TAOS_SYSTEM_ERROR(errno); return -1; } - if (taosWriteFile(pFile, &pSdb->curConfig, sizeof(int64_t)) != sizeof(int64_t)) { + if (taosWriteFile(pFile, &pSdb->applyConfig, sizeof(int64_t)) != sizeof(int64_t)) { terrno = TAOS_SYSTEM_ERROR(errno); return -1; } @@ -300,11 +302,12 @@ static int32_t sdbReadFileImp(SSdb *pSdb) { } code = 0; - pSdb->lastCommitVer = pSdb->curVer; - pSdb->lastCommitTerm = pSdb->curTerm; + pSdb->commitIndex = pSdb->applyIndex; + pSdb->commitTerm = pSdb->applyTerm; + pSdb->commitConfig = pSdb->applyConfig; memcpy(pSdb->tableVer, tableVer, sizeof(tableVer)); - mDebug("read sdb file:%s successfully, index:%" PRId64 " term:%" PRId64 " config:%" PRId64, file, pSdb->lastCommitVer, - pSdb->lastCommitTerm, pSdb->curConfig); + mDebug("read sdb file:%s successfully, commit index:%" PRId64 " term:%" PRId64 " config:%" PRId64, file, + pSdb->commitIndex, pSdb->commitTerm, pSdb->commitConfig); _OVER: taosCloseFile(&pFile); @@ -336,9 +339,10 @@ static int32_t sdbWriteFileImp(SSdb *pSdb) { char curfile[PATH_MAX] = {0}; snprintf(curfile, sizeof(curfile), "%s%ssdb.data", pSdb->currDir, TD_DIRSEP); - mDebug("start to write sdb file, current ver:%" PRId64 " term:%" PRId64 ", commit ver:%" PRId64 " term:%" PRId64 - " file:%s", - pSdb->curVer, pSdb->curTerm, pSdb->lastCommitVer, pSdb->lastCommitTerm, curfile); + mDebug("start to write sdb file, apply index:%" PRId64 " term:%" PRId64 " config:%" PRId64 ", commit index:%" PRId64 + " term:%" PRId64 " config:%" PRId64 ", file:%s", + pSdb->applyIndex, pSdb->applyTerm, pSdb->applyConfig, pSdb->commitIndex, pSdb->commitTerm, pSdb->commitConfig, + curfile); TdFilePtr pFile = taosOpenFile(tmpfile, TD_FILE_CREATE | TD_FILE_WRITE | TD_FILE_TRUNC); if (pFile == NULL) { @@ -430,25 +434,30 @@ static int32_t sdbWriteFileImp(SSdb *pSdb) { if (code != 0) { mError("failed to write sdb file:%s since %s", curfile, tstrerror(code)); } else { - pSdb->lastCommitVer = pSdb->curVer; - pSdb->lastCommitTerm = pSdb->curTerm; - mDebug("write sdb file successfully, index:%" PRId64 " term:%" PRId64 " config:%" PRId64 " file:%s", - pSdb->lastCommitVer, pSdb->lastCommitTerm, pSdb->curConfig, curfile); + pSdb->commitIndex = pSdb->applyIndex; + pSdb->commitTerm = pSdb->applyTerm; + pSdb->commitConfig = pSdb->applyConfig; + mDebug("write sdb file successfully, commit index:%" PRId64 " term:%" PRId64 " config:%" PRId64 " file:%s", + pSdb->commitIndex, pSdb->commitTerm, pSdb->commitConfig, curfile); } terrno = code; return code; } -int32_t sdbWriteFile(SSdb *pSdb) { +int32_t sdbWriteFile(SSdb *pSdb, int32_t delta) { int32_t code = 0; - if (pSdb->curVer == pSdb->lastCommitVer) { + if (pSdb->applyIndex == pSdb->commitIndex) { + return 0; + } + + if (pSdb->applyIndex - pSdb->commitIndex < delta) { return 0; } taosThreadMutexLock(&pSdb->filelock); if (pSdb->pWal != NULL) { - code = walBeginSnapshot(pSdb->pWal, pSdb->curVer); + code = walBeginSnapshot(pSdb->pWal, pSdb->applyIndex); } if (code == 0) { code = sdbWriteFileImp(pSdb); @@ -470,7 +479,7 @@ int32_t sdbDeploy(SSdb *pSdb) { return -1; } - if (sdbWriteFile(pSdb) != 0) { + if (sdbWriteFile(pSdb, 0) != 0) { return -1; } @@ -522,9 +531,9 @@ int32_t sdbStartRead(SSdb *pSdb, SSdbIter **ppIter) { snprintf(datafile, sizeof(datafile), "%s%ssdb.data", pSdb->currDir, TD_DIRSEP); taosThreadMutexLock(&pSdb->filelock); - int64_t commitIndex = pSdb->lastCommitVer; - int64_t commitTerm = pSdb->lastCommitTerm; - int64_t curConfig = pSdb->curConfig; + int64_t commitIndex = pSdb->commitIndex; + int64_t commitTerm = pSdb->commitTerm; + int64_t commitConfig = pSdb->commitConfig; if (taosCopyFile(datafile, pIter->name) < 0) { taosThreadMutexUnlock(&pSdb->filelock); terrno = TAOS_SYSTEM_ERROR(errno); @@ -543,8 +552,8 @@ int32_t sdbStartRead(SSdb *pSdb, SSdbIter **ppIter) { } *ppIter = pIter; - mInfo("sdbiter:%p, is created to read snapshot, index:%" PRId64 " term:%" PRId64 " config:%" PRId64 " file:%s", pIter, - commitIndex, commitTerm, curConfig, pIter->name); + mInfo("sdbiter:%p, is created to read snapshot, commit index:%" PRId64 " term:%" PRId64 " config:%" PRId64 " file:%s", + pIter, commitIndex, commitTerm, commitConfig, pIter->name); return 0; } diff --git a/source/dnode/vnode/src/meta/metaQuery.c b/source/dnode/vnode/src/meta/metaQuery.c index a6339125c42e873a62b231ab3932736023f2204f..8cfebe1b98c8e81f462f2243f9c9a9eebadb78a8 100644 --- a/source/dnode/vnode/src/meta/metaQuery.c +++ b/source/dnode/vnode/src/meta/metaQuery.c @@ -628,7 +628,7 @@ int32_t metaFilteTableIds(SMeta *pMeta, SMetaFltParam *param, SArray *pUids) { void * tagData = NULL; if (param->val == NULL) { - metaError("vgId:%d failed to filter NULL data", TD_VID(pMeta->pVnode)); + metaError("vgId:%d, failed to filter NULL data", TD_VID(pMeta->pVnode)); return -1; } else { if (IS_VAR_DATA_TYPE(param->type)) { diff --git a/source/dnode/vnode/src/tq/tq.c b/source/dnode/vnode/src/tq/tq.c index c9d3267adb8414009bfd427f1d2c335070fe5815..70b09ec701af09888f37434367b72bcfe3c46938 100644 --- a/source/dnode/vnode/src/tq/tq.c +++ b/source/dnode/vnode/src/tq/tq.c @@ -124,17 +124,20 @@ int32_t tqProcessOffsetCommitReq(STQ* pTq, char* msg, int32_t msgLen) { tDecoderClear(&decoder); if (offset.type == TMQ_OFFSET__SNAPSHOT) { - tqDebug("receive offset commit msg to %s, offset(type:snapshot) uid: %ld, ts: %ld", offset.subKey, offset.uid, - offset.ts); + tqDebug("receive offset commit msg to %s on vg %d, offset(type:snapshot) uid: %ld, ts: %ld", offset.subKey, + pTq->pVnode->config.vgId, offset.uid, offset.ts); } else if (offset.type == TMQ_OFFSET__LOG) { - tqDebug("receive offset commit msg to %s, offset(type:log) version: %ld", offset.subKey, offset.version); + tqDebug("receive offset commit msg to %s on vg %d, offset(type:log) version: %ld", offset.subKey, + pTq->pVnode->config.vgId, offset.version); } else { ASSERT(0); } - - if (tqOffsetWrite(pTq->pOffsetStore, &offset) < 0) { - ASSERT(0); - return -1; + STqOffset* pOffset = tqOffsetRead(pTq->pOffsetStore, offset.subKey); + if (pOffset == NULL || pOffset->version < offset.version) { + if (tqOffsetWrite(pTq->pOffsetStore, &offset) < 0) { + ASSERT(0); + return -1; + } } return 0; @@ -149,16 +152,33 @@ int32_t tqProcessPollReq(STQ* pTq, SRpcMsg* pMsg, int32_t workerId) { int32_t code = 0; // get offset to fetch message - if (pReq->currentOffset == TMQ_CONF__RESET_OFFSET__EARLIEAST) { - fetchOffset = walGetFirstVer(pTq->pWal); - } else if (pReq->currentOffset == TMQ_CONF__RESET_OFFSET__LATEST) { - fetchOffset = walGetCommittedVer(pTq->pWal); - } else { + if (pReq->currentOffset >= 0) { fetchOffset = pReq->currentOffset + 1; + } else { + STqOffset* pOffset = tqOffsetRead(pTq->pOffsetStore, pReq->subKey); + if (pOffset != NULL) { + ASSERT(pOffset->type == TMQ_OFFSET__LOG); + tqDebug("consumer %ld, restore offset of %s on vg %d, offset(type:log) version: %ld", consumerId, pReq->subKey, + pTq->pVnode->config.vgId, pOffset->version); + fetchOffset = pOffset->version + 1; + } else { + if (pReq->currentOffset == TMQ_CONF__RESET_OFFSET__EARLIEAST) { + fetchOffset = walGetFirstVer(pTq->pWal); + } else if (pReq->currentOffset == TMQ_CONF__RESET_OFFSET__LATEST) { + fetchOffset = walGetCommittedVer(pTq->pWal); + } else if (pReq->currentOffset == TMQ_CONF__RESET_OFFSET__NONE) { + tqError("tmq poll: no offset committed for consumer %ld in vg %d, subkey %s", consumerId, + pTq->pVnode->config.vgId, pReq->subKey); + terrno = TSDB_CODE_TQ_NO_COMMITTED_OFFSET; + return -1; + } + tqDebug("consumer %ld, restore offset of %s on vg %d failed, config is %ld, set to %ld", consumerId, pReq->subKey, + pTq->pVnode->config.vgId, pReq->currentOffset, fetchOffset); + } } - tqDebug("tmq poll: consumer %ld (epoch %d) recv poll req in vg %d, req %ld %ld", consumerId, pReq->epoch, - TD_VID(pTq->pVnode), pReq->currentOffset, fetchOffset); + tqDebug("tmq poll: consumer %ld (epoch %d) recv poll req in vg %d, req offset %ld fetch offset %ld", consumerId, + pReq->epoch, TD_VID(pTq->pVnode), pReq->currentOffset, fetchOffset); STqHandle* pHandle = taosHashGet(pTq->handles, pReq->subKey, strlen(pReq->subKey)); /*ASSERT(pHandle);*/ diff --git a/source/dnode/vnode/src/tq/tqCommit.c b/source/dnode/vnode/src/tq/tqCommit.c index 7b116bff2e942bf1a461458ea443548e708756eb..639da22b1c16f39a953ae0b7885344a7ca95768e 100644 --- a/source/dnode/vnode/src/tq/tqCommit.c +++ b/source/dnode/vnode/src/tq/tqCommit.c @@ -15,7 +15,4 @@ #include "tq.h" -int tqCommit(STQ* pTq) { - // do nothing - return 0; -} +int tqCommit(STQ* pTq) { return tqOffsetSnapshot(pTq->pOffsetStore); } diff --git a/source/dnode/vnode/src/tq/tqOffset.c b/source/dnode/vnode/src/tq/tqOffset.c index 8d6cb280659949489a301ad68ff7740ccba5bbc4..e5475d7c300531b7b3e0d8cce369fa9eb0415930 100644 --- a/source/dnode/vnode/src/tq/tqOffset.c +++ b/source/dnode/vnode/src/tq/tqOffset.c @@ -92,6 +92,8 @@ STqOffset* tqOffsetRead(STqOffsetStore* pStore, const char* subscribeKey) { } int32_t tqOffsetWrite(STqOffsetStore* pStore, const STqOffset* pOffset) { + ASSERT(pOffset->type == TMQ_OFFSET__LOG); + ASSERT(pOffset->version >= 0); return taosHashPut(pStore->pHash, pOffset->subKey, strlen(pOffset->subKey), pOffset, sizeof(STqOffset)); } @@ -129,7 +131,7 @@ int32_t tqOffsetSnapshot(STqOffsetStore* pStore) { tEncodeSTqOffset(&encoder, pOffset); // write file int64_t writeLen; - if ((writeLen = taosWriteFile(pFile, buf, totLen)) != bodyLen) { + if ((writeLen = taosWriteFile(pFile, buf, totLen)) != totLen) { ASSERT(0); tqError("write offset incomplete, len %d, write len %ld", bodyLen, writeLen); taosHashCancelIterate(pStore->pHash, pIter); diff --git a/source/dnode/vnode/src/tsdb/tsdbCommit.c b/source/dnode/vnode/src/tsdb/tsdbCommit.c index fe89321ae9606387c0869e4d8f63328d0002e140..e98fd8ae1f6dc6cd0d6f47a2563055f7e452870a 100644 --- a/source/dnode/vnode/src/tsdb/tsdbCommit.c +++ b/source/dnode/vnode/src/tsdb/tsdbCommit.c @@ -111,7 +111,7 @@ int32_t tsdbBegin(STsdb *pTsdb) { int32_t tsdbCommit(STsdb *pTsdb) { if (!pTsdb) return 0; - + int32_t code = 0; SCommitH commith = {0}; SDFileSet *pSet = NULL; @@ -495,7 +495,9 @@ static int32_t tsdbCommitToFile(SCommitH *pCommith, SDFileSet *pSet, int fid) { break; } - if (pIter && pIter->pTable && (!pIdx || (pIter->pTable->suid <= pIdx->suid || pIter->pTable->uid <= pIdx->uid))) { + if (pIter && pIter->pTable && + (!pIdx || ((pIter->pTable->suid < pIdx->suid) || + ((pIter->pTable->suid == pIdx->suid) && (pIter->pTable->uid <= pIdx->uid))))) { if (tsdbCommitToTable(pCommith, mIter) < 0) { tsdbCloseCommitFile(pCommith, true); // revert the file change @@ -503,7 +505,7 @@ static int32_t tsdbCommitToFile(SCommitH *pCommith, SDFileSet *pSet, int fid) { return -1; } - if (pIdx && (pIter->pTable->uid == pIdx->uid)) { + if (pIdx && ((pIter->pTable->uid == pIdx->uid) && (pIter->pTable->suid == pIdx->suid))) { ++fIter; } ++mIter; @@ -518,6 +520,8 @@ static int32_t tsdbCommitToFile(SCommitH *pCommith, SDFileSet *pSet, int fid) { return -1; } ++fIter; + } else { + ASSERT(0); } } diff --git a/source/dnode/vnode/src/vnd/vnodeSvr.c b/source/dnode/vnode/src/vnd/vnodeSvr.c index da1a126a765289fa8520df33519ecd57d3f7c1d9..feb8419bdf3a1333e962ff164bd87b21bb0f26ca 100644 --- a/source/dnode/vnode/src/vnd/vnodeSvr.c +++ b/source/dnode/vnode/src/vnd/vnodeSvr.c @@ -774,9 +774,6 @@ static int32_t vnodeProcessCreateTSmaReq(SVnode *pVnode, int64_t version, void * goto _err; } - // record current timezone of server side - req.timezoneInt = tsTimezone; - if (tdProcessTSmaCreate(pVnode->pSma, version, (const char *)&req) < 0) { if (pRsp) pRsp->code = terrno; goto _err; diff --git a/source/dnode/vnode/src/vnd/vnodeSync.c b/source/dnode/vnode/src/vnd/vnodeSync.c index a680b88950941a5359e71c39cf6bb5279e4b7afb..ae61aa454e6663c1edcb395f505f907745acfa65 100644 --- a/source/dnode/vnode/src/vnd/vnodeSync.c +++ b/source/dnode/vnode/src/vnd/vnodeSync.c @@ -61,6 +61,7 @@ static int32_t vnodeSetStandBy(SVnode *pVnode) { return -1; } + vInfo("vgId:%d, start to transfer leader", TD_VID(pVnode)); if (syncLeaderTransfer(pVnode->sync) != 0) { vError("vgId:%d, failed to transfer leader since:%s", TD_VID(pVnode), terrstr()); return -1; @@ -186,7 +187,7 @@ void vnodeApplyMsg(SQueueInfo *pInfo, STaosQall *qall, int32_t numOfMsgs) { SRpcMsg rsp = {.code = pMsg->code, .info = pMsg->info}; if (rsp.code == 0) { - if (vnodeProcessWriteReq(pVnode, pMsg, pMsg->conn.applyIndex, &rsp) < 0) { + if (vnodeProcessWriteReq(pVnode, pMsg, pMsg->info.conn.applyIndex, &rsp) < 0) { rsp.code = terrno; vError("vgId:%d, msg:%p failed to apply since %s", vgId, pMsg, terrstr()); } @@ -300,7 +301,7 @@ int32_t vnodeProcessSyncReq(SVnode *pVnode, SRpcMsg *pMsg, SRpcMsg **pRsp) { ret = -1; } - if (ret != 0) { + if (ret != 0 && terrno == 0) { terrno = TSDB_CODE_SYN_INTERNAL_ERROR; } return ret; @@ -332,8 +333,9 @@ static int32_t vnodeSyncGetSnapshot(SSyncFSM *pFsm, SSnapshot *pSnapshot) { static void vnodeSyncReconfig(struct SSyncFSM *pFsm, const SRpcMsg *pMsg, SReConfigCbMeta cbMeta) { SVnode *pVnode = pFsm->data; - SRpcMsg rpcMsg = {.msgType = pMsg->msgType, .contLen = pMsg->contLen, .conn.applyIndex = cbMeta.index}; + SRpcMsg rpcMsg = {.msgType = pMsg->msgType, .contLen = pMsg->contLen}; syncGetAndDelRespRpc(pVnode->sync, cbMeta.seqNum, &rpcMsg.info); + rpcMsg.info.conn.applyIndex = cbMeta.index; STraceId *trace = (STraceId *)&pMsg->info.traceId; vGTrace("vgId:%d, alter vnode replica is confirmed, type:%s contLen:%d seq:%" PRIu64 " handle:%p", TD_VID(pVnode), @@ -363,10 +365,11 @@ static void vnodeSyncCommitMsg(SSyncFSM *pFsm, const SRpcMsg *pMsg, SFsmCbMeta c pFsm, cbMeta.index, cbMeta.isWeak, cbMeta.code, cbMeta.state, syncUtilState2String(cbMeta.state), beginIndex); syncRpcMsgLog2(logBuf, (SRpcMsg *)pMsg); - SRpcMsg rpcMsg = {.msgType = pMsg->msgType, .contLen = pMsg->contLen, .conn.applyIndex = cbMeta.index}; + SRpcMsg rpcMsg = {.msgType = pMsg->msgType, .contLen = pMsg->contLen}; rpcMsg.pCont = rpcMallocCont(rpcMsg.contLen); memcpy(rpcMsg.pCont, pMsg->pCont, pMsg->contLen); syncGetAndDelRespRpc(pVnode->sync, cbMeta.seqNum, &rpcMsg.info); + rpcMsg.info.conn.applyIndex = cbMeta.index; tmsgPutToQueue(&pVnode->msgCb, APPLY_QUEUE, &rpcMsg); } else { diff --git a/source/libs/catalog/inc/catalogInt.h b/source/libs/catalog/inc/catalogInt.h index df4b9b6ea6c9b30b83b0bb1c25ab9d86a82fa487..7c90c3538cc3428d6c6456c8c67f9acb4818e82c 100644 --- a/source/libs/catalog/inc/catalogInt.h +++ b/source/libs/catalog/inc/catalogInt.h @@ -59,6 +59,7 @@ enum { CTG_OP_UPDATE_VG_EPSET, CTG_OP_UPDATE_TB_INDEX, CTG_OP_DROP_TB_INDEX, + CTG_OP_CLEAR_CACHE, CTG_OP_MAX }; @@ -328,6 +329,10 @@ typedef struct SCtgDropTbIndexMsg { char tbName[TSDB_TABLE_NAME_LEN]; } SCtgDropTbIndexMsg; +typedef struct SCtgClearCacheMsg { + SCatalog* pCtg; +} SCtgClearCacheMsg; + typedef struct SCtgUpdateEpsetMsg { SCatalog* pCtg; char dbFName[TSDB_DB_FNAME_LEN]; @@ -471,8 +476,8 @@ typedef struct SCtgOperation { #define CTG_API_LEAVE(c) do { int32_t __code = c; CTG_UNLOCK(CTG_READ, &gCtgMgmt.lock); CTG_API_DEBUG("CTG API leave %s", __FUNCTION__); CTG_RET(__code); } while (0) #define CTG_API_ENTER() do { CTG_API_DEBUG("CTG API enter %s", __FUNCTION__); CTG_LOCK(CTG_READ, &gCtgMgmt.lock); if (atomic_load_8((int8_t*)&gCtgMgmt.exit)) { CTG_API_LEAVE(TSDB_CODE_CTG_OUT_OF_SERVICE); } } while (0) -void ctgdShowTableMeta(SCatalog* pCtg, const char *tbName, STableMeta* p); -void ctgdShowClusterCache(SCatalog* pCtg); +void ctgdShowTableMeta(SCatalog* pCtg, const char *tbName, STableMeta* p); +void ctgdShowClusterCache(SCatalog* pCtg); int32_t ctgdShowCacheInfo(void); int32_t ctgRemoveTbMetaFromCache(SCatalog* pCtg, SName* pTableName, bool syncReq); @@ -487,8 +492,8 @@ int32_t ctgOpDropTbMeta(SCtgCacheOperation *action); int32_t ctgOpUpdateUser(SCtgCacheOperation *action); int32_t ctgOpUpdateEpset(SCtgCacheOperation *operation); int32_t ctgAcquireVgInfoFromCache(SCatalog* pCtg, const char *dbFName, SCtgDBCache **pCache); -void ctgReleaseDBCache(SCatalog *pCtg, SCtgDBCache *dbCache); -void ctgRUnlockVgInfo(SCtgDBCache *dbCache); +void ctgReleaseDBCache(SCatalog *pCtg, SCtgDBCache *dbCache); +void ctgRUnlockVgInfo(SCtgDBCache *dbCache); int32_t ctgTbMetaExistInCache(SCatalog* pCtg, char *dbFName, char* tbName, int32_t *exist); int32_t ctgReadTbMetaFromCache(SCatalog* pCtg, SCtgTbMetaCtx* ctx, STableMeta** pTableMeta); int32_t ctgReadTbVerFromCache(SCatalog *pCtg, SName *pTableName, int32_t *sver, int32_t *tver, int32_t *tbType, uint64_t *suid, char *stbName); @@ -502,17 +507,20 @@ int32_t ctgUpdateTbMetaEnqueue(SCatalog* pCtg, STableMetaOutput *output, bool sy int32_t ctgUpdateUserEnqueue(SCatalog* pCtg, SGetUserAuthRsp *pAuth, bool syncReq); int32_t ctgUpdateVgEpsetEnqueue(SCatalog* pCtg, char *dbFName, int32_t vgId, SEpSet* pEpSet); int32_t ctgUpdateTbIndexEnqueue(SCatalog* pCtg, STableIndex **pIndex, bool syncOp); +int32_t ctgClearCacheEnqueue(SCatalog* pCtg, bool syncOp); int32_t ctgMetaRentInit(SCtgRentMgmt *mgmt, uint32_t rentSec, int8_t type); int32_t ctgMetaRentAdd(SCtgRentMgmt *mgmt, void *meta, int64_t id, int32_t size); int32_t ctgMetaRentGet(SCtgRentMgmt *mgmt, void **res, uint32_t *num, int32_t size); int32_t ctgUpdateTbMetaToCache(SCatalog* pCtg, STableMetaOutput* pOut, bool syncReq); int32_t ctgStartUpdateThread(); int32_t ctgRelaunchGetTbMetaTask(SCtgTask *pTask); -void ctgReleaseVgInfoToCache(SCatalog* pCtg, SCtgDBCache *dbCache); +void ctgReleaseVgInfoToCache(SCatalog* pCtg, SCtgDBCache *dbCache); int32_t ctgReadTbIndexFromCache(SCatalog* pCtg, SName* pTableName, SArray** pRes); int32_t ctgDropTbIndexEnqueue(SCatalog* pCtg, SName* pName, bool syncOp); int32_t ctgOpDropTbIndex(SCtgCacheOperation *operation); int32_t ctgOpUpdateTbIndex(SCtgCacheOperation *operation); +int32_t ctgOpClearCache(SCtgCacheOperation *operation); + @@ -535,22 +543,22 @@ int32_t ctgMakeAsyncRes(SCtgJob *pJob); int32_t ctgCloneVgInfo(SDBVgInfo *src, SDBVgInfo **dst); int32_t ctgCloneMetaOutput(STableMetaOutput *output, STableMetaOutput **pOutput); int32_t ctgGenerateVgList(SCatalog *pCtg, SHashObj *vgHash, SArray** pList); -void ctgFreeJob(void* job); -void ctgFreeHandle(SCatalog* pCtg); -void ctgFreeVgInfo(SDBVgInfo *vgInfo); +void ctgFreeJob(void* job); +void ctgFreeHandle(SCatalog* pCtg); +void ctgFreeVgInfo(SDBVgInfo *vgInfo); int32_t ctgGetVgInfoFromHashValue(SCatalog *pCtg, SDBVgInfo *dbInfo, const SName *pTableName, SVgroupInfo *pVgroup); -void ctgResetTbMetaTask(SCtgTask* pTask); -void ctgFreeDbCache(SCtgDBCache *dbCache); +void ctgResetTbMetaTask(SCtgTask* pTask); +void ctgFreeDbCache(SCtgDBCache *dbCache); int32_t ctgStbVersionSortCompare(const void* key1, const void* key2); int32_t ctgDbVgVersionSortCompare(const void* key1, const void* key2); int32_t ctgStbVersionSearchCompare(const void* key1, const void* key2); int32_t ctgDbVgVersionSearchCompare(const void* key1, const void* key2); -void ctgFreeSTableMetaOutput(STableMetaOutput* pOutput); +void ctgFreeSTableMetaOutput(STableMetaOutput* pOutput); int32_t ctgUpdateMsgCtx(SCtgMsgCtx* pCtx, int32_t reqType, void* out, char* target); -char *ctgTaskTypeStr(CTG_TASK_TYPE type); +char * ctgTaskTypeStr(CTG_TASK_TYPE type); int32_t ctgUpdateSendTargetInfo(SMsgSendInfo *pMsgSendInfo, int32_t msgType, SCtgTask* pTask); int32_t ctgCloneTableIndex(SArray* pIndex, SArray** pRes); -void ctgFreeSTableIndex(void *info); +void ctgFreeSTableIndex(void *info); extern SCatalogMgmt gCtgMgmt; diff --git a/source/libs/catalog/src/catalog.c b/source/libs/catalog/src/catalog.c index a3576f8738a13715430b4629fe68194511106dbc..bb02895569c180ea442992f60818e541a842eb8d 100644 --- a/source/libs/catalog/src/catalog.c +++ b/source/libs/catalog/src/catalog.c @@ -105,7 +105,7 @@ int32_t ctgRefreshDBVgInfo(SCatalog* pCtg, SRequestConnInfo *pConn, const char* code = ctgGetDBVgInfoFromMnode(pCtg, pConn, &input, &DbOut, NULL); if (code) { if (CTG_DB_NOT_EXIST(code) && (NULL != dbCache)) { - ctgDebug("db no longer exist, dbFName:%s, dbId:%" PRIx64, input.db, input.dbId); + ctgDebug("db no longer exist, dbFName:%s, dbId:0x%" PRIx64, input.db, input.dbId); ctgDropDbCacheEnqueue(pCtg, input.db, input.dbId); } @@ -571,7 +571,7 @@ int32_t catalogGetHandle(uint64_t clusterId, SCatalog** catalogHandle) { } if (NULL == gCtgMgmt.pCluster) { - qError("catalog cluster cache are not ready, clusterId:%" PRIx64, clusterId); + qError("catalog cluster cache are not ready, clusterId:0x%" PRIx64, clusterId); CTG_ERR_RET(TSDB_CODE_CTG_NOT_READY); } @@ -583,7 +583,7 @@ int32_t catalogGetHandle(uint64_t clusterId, SCatalog** catalogHandle) { if (ctg && (*ctg)) { *catalogHandle = *ctg; - qDebug("got catalog handle from cache, clusterId:%" PRIx64 ", CTG:%p", clusterId, *ctg); + qDebug("got catalog handle from cache, clusterId:0x%" PRIx64 ", CTG:%p", clusterId, *ctg); return TSDB_CODE_SUCCESS; } @@ -612,11 +612,11 @@ int32_t catalogGetHandle(uint64_t clusterId, SCatalog** catalogHandle) { continue; } - qError("taosHashPut CTG to cache failed, clusterId:%" PRIx64, clusterId); + qError("taosHashPut CTG to cache failed, clusterId:0x%" PRIx64, clusterId); CTG_ERR_JRET(TSDB_CODE_CTG_INTERNAL_ERROR); } - qDebug("add CTG to cache, clusterId:%" PRIx64 ", CTG:%p", clusterId, clusterCtg); + qDebug("add CTG to cache, clusterId:0x%" PRIx64 ", CTG:%p", clusterId, clusterCtg); break; } @@ -640,7 +640,7 @@ void catalogFreeHandle(SCatalog* pCtg) { } if (taosHashRemove(gCtgMgmt.pCluster, &pCtg->clusterId, sizeof(pCtg->clusterId))) { - ctgWarn("taosHashRemove from cluster failed, may already be freed, clusterId:%" PRIx64, pCtg->clusterId); + ctgWarn("taosHashRemove from cluster failed, may already be freed, clusterId:0x%" PRIx64, pCtg->clusterId); return; } @@ -650,7 +650,7 @@ void catalogFreeHandle(SCatalog* pCtg) { ctgFreeHandle(pCtg); - ctgInfo("handle freed, culsterId:%" PRIx64, clusterId); + ctgInfo("handle freed, culsterId:0x%" PRIx64, clusterId); } int32_t catalogGetDBVgVersion(SCatalog* pCtg, const char* dbFName, int32_t* version, int64_t* dbId, int32_t* tableNum) { @@ -1247,6 +1247,23 @@ int32_t catalogUpdateUserAuthInfo(SCatalog* pCtg, SGetUserAuthRsp* pAuth) { CTG_API_LEAVE(ctgUpdateUserEnqueue(pCtg, pAuth, false)); } +int32_t catalogClearCache(void) { + CTG_API_ENTER(); + + qInfo("start to clear catalog cache"); + + if (NULL == gCtgMgmt.pCluster || atomic_load_8((int8_t*)&gCtgMgmt.exit)) { + CTG_API_LEAVE(TSDB_CODE_SUCCESS); + } + + int32_t code = ctgClearCacheEnqueue(NULL, true); + + qInfo("clear catalog cache end, code: %s", tstrerror(code)); + + CTG_API_LEAVE(code); +} + + void catalogDestroy(void) { qInfo("start to destroy catalog"); diff --git a/source/libs/catalog/src/ctgAsync.c b/source/libs/catalog/src/ctgAsync.c index 8dabd56934677917e11f0ccf2c97c7490087f7d7..6adadf50455950e5d49bf0a5a9e178673c6fcf6b 100644 --- a/source/libs/catalog/src/ctgAsync.c +++ b/source/libs/catalog/src/ctgAsync.c @@ -622,7 +622,7 @@ int32_t ctgHandleTaskEnd(SCtgTask* pTask, int32_t rspCode) { SCtgJob* pJob = pTask->pJob; int32_t code = 0; - qDebug("QID:0x%" PRIx64 " task %d end with rsp %s", pJob->queryId, pTask->taskId, tstrerror(rspCode)); + qDebug("QID:0x%" PRIx64 " task %d end with res %s", pJob->queryId, pTask->taskId, tstrerror(rspCode)); pTask->code = rspCode; @@ -1276,7 +1276,7 @@ int32_t ctgLaunchJob(SCtgJob *pJob) { for (int32_t i = 0; i < taskNum; ++i) { SCtgTask *pTask = taosArrayGet(pJob->pTasks, i); - qDebug("QID:0x%" PRIx64 " start to launch task %d", pJob->queryId, pTask->taskId); + qDebug("QID:0x%" PRIx64 " ctg start to launch task %d", pJob->queryId, pTask->taskId); CTG_ERR_RET((*gCtgAsyncFps[pTask->type].launchFp)(pTask)); } diff --git a/source/libs/catalog/src/ctgCache.c b/source/libs/catalog/src/ctgCache.c index 0948c01270996d11ba7da78c8d86327341d41093..1de5ee3d7d9ea6c3ce24b6890e12cfc0cb9c75d8 100644 --- a/source/libs/catalog/src/ctgCache.c +++ b/source/libs/catalog/src/ctgCache.c @@ -69,6 +69,11 @@ SCtgOperation gCtgCacheOperation[CTG_OP_MAX] = { CTG_OP_DROP_TB_INDEX, "drop tbIndex", ctgOpDropTbIndex + }, + { + CTG_OP_CLEAR_CACHE, + "clear cache", + ctgOpClearCache } }; @@ -81,7 +86,7 @@ int32_t ctgRLockVgInfo(SCatalog *pCtg, SCtgDBCache *dbCache, bool *inCache) { if (dbCache->deleted) { CTG_UNLOCK(CTG_READ, &dbCache->vgCache.vgLock); - ctgDebug("db is dropping, dbId:%"PRIx64, dbCache->dbId); + ctgDebug("db is dropping, dbId:0x%"PRIx64, dbCache->dbId); *inCache = false; return TSDB_CODE_SUCCESS; @@ -92,7 +97,7 @@ int32_t ctgRLockVgInfo(SCatalog *pCtg, SCtgDBCache *dbCache, bool *inCache) { CTG_UNLOCK(CTG_READ, &dbCache->vgCache.vgLock); *inCache = false; - ctgDebug("db vgInfo is empty, dbId:%"PRIx64, dbCache->dbId); + ctgDebug("db vgInfo is empty, dbId:0x%"PRIx64, dbCache->dbId); return TSDB_CODE_SUCCESS; } @@ -105,7 +110,7 @@ int32_t ctgWLockVgInfo(SCatalog *pCtg, SCtgDBCache *dbCache) { CTG_LOCK(CTG_WRITE, &dbCache->vgCache.vgLock); if (dbCache->deleted) { - ctgDebug("db is dropping, dbId:%"PRIx64, dbCache->dbId); + ctgDebug("db is dropping, dbId:0x%"PRIx64, dbCache->dbId); CTG_UNLOCK(CTG_WRITE, &dbCache->vgCache.vgLock); CTG_ERR_RET(TSDB_CODE_CTG_DB_DROPPED); } @@ -280,27 +285,27 @@ int32_t ctgAcquireStbMetaFromCache(SCatalog* pCtg, char *dbFName, uint64_t suid, int32_t sz = 0; char* stName = taosHashAcquire(dbCache->stbCache, &suid, sizeof(suid)); if (NULL == stName) { - ctgDebug("stb %" PRIx64 " not in cache, dbFName:%s", suid, dbFName); + ctgDebug("stb 0x%" PRIx64 " not in cache, dbFName:%s", suid, dbFName); goto _return; } pCache = taosHashAcquire(dbCache->tbCache, stName, strlen(stName)); if (NULL == pCache) { - ctgDebug("stb %" PRIx64 " name %s not in cache, dbFName:%s", suid, stName, dbFName); + ctgDebug("stb 0x%" PRIx64 " name %s not in cache, dbFName:%s", suid, stName, dbFName); taosHashRelease(dbCache->stbCache, stName); goto _return; } CTG_LOCK(CTG_READ, &pCache->metaLock); if (NULL == pCache->pMeta) { - ctgDebug("stb %" PRIx64 " meta not in cache, dbFName:%s", suid, dbFName); + ctgDebug("stb 0x%" PRIx64 " meta not in cache, dbFName:%s", suid, dbFName); goto _return; } *pDb = dbCache; *pTb = pCache; - ctgDebug("stb %" PRIx64 " meta got in cache, dbFName:%s", suid, dbFName); + ctgDebug("stb 0x%" PRIx64 " meta got in cache, dbFName:%s", suid, dbFName); CTG_CACHE_STAT_INC(tbMetaHitNum, 1); @@ -434,14 +439,14 @@ int32_t ctgReadTbMetaFromCache(SCatalog* pCtg, SCtgTbMetaCtx* ctx, STableMeta** if (NULL == tbCache) { ctgReleaseTbMetaToCache(pCtg, dbCache, tbCache); taosMemoryFreeClear(*pTableMeta); - ctgDebug("stb %" PRIx64 " meta not in cache", ctx->tbInfo.suid); + ctgDebug("stb 0x%" PRIx64 " meta not in cache", ctx->tbInfo.suid); return TSDB_CODE_SUCCESS; } STableMeta* stbMeta = tbCache->pMeta; if (stbMeta->suid != ctx->tbInfo.suid) { ctgReleaseTbMetaToCache(pCtg, dbCache, tbCache); - ctgError("stb suid %" PRIx64 " in stbCache mis-match, expected suid:%"PRIx64 , stbMeta->suid, ctx->tbInfo.suid); + ctgError("stb suid 0x%" PRIx64 " in stbCache mis-match, expected suid 0x%"PRIx64 , stbMeta->suid, ctx->tbInfo.suid); CTG_ERR_JRET(TSDB_CODE_CTG_INTERNAL_ERROR); } @@ -492,7 +497,7 @@ int32_t ctgReadTbVerFromCache(SCatalog *pCtg, SName *pTableName, int32_t *sver, *sver = tbMeta->sversion; *tver = tbMeta->tversion; - ctgDebug("Got tb %s ver from cache, dbFName:%s, tbType:%d, sver:%d, tver:%d, suid:%" PRIx64, + ctgDebug("Got tb %s ver from cache, dbFName:%s, tbType:%d, sver:%d, tver:%d, suid:0x%" PRIx64, pTableName->tname, dbFName, *tbType, *sver, *tver, *suid); ctgReleaseTbMetaToCache(pCtg, dbCache, tbCache); @@ -507,14 +512,14 @@ int32_t ctgReadTbVerFromCache(SCatalog *pCtg, SName *pTableName, int32_t *sver, ctgAcquireStbMetaFromCache(pCtg, dbFName, *suid, &dbCache, &tbCache); if (NULL == tbCache) { ctgReleaseTbMetaToCache(pCtg, dbCache, tbCache); - ctgDebug("stb %" PRIx64 " meta not in cache", *suid); + ctgDebug("stb 0x%" PRIx64 " meta not in cache", *suid); return TSDB_CODE_SUCCESS; } STableMeta* stbMeta = tbCache->pMeta; if (stbMeta->suid != *suid) { ctgReleaseTbMetaToCache(pCtg, dbCache, tbCache); - ctgError("stb suid %" PRIx64 " in stbCache mis-match, expected suid:%" PRIx64 , stbMeta->suid, *suid); + ctgError("stb suid 0x%" PRIx64 " in stbCache mis-match, expected suid:0x%" PRIx64 , stbMeta->suid, *suid); CTG_ERR_RET(TSDB_CODE_CTG_INTERNAL_ERROR); } @@ -990,6 +995,33 @@ _return: } +int32_t ctgClearCacheEnqueue(SCatalog* pCtg, bool syncOp) { + int32_t code = 0; + SCtgCacheOperation *op = taosMemoryCalloc(1, sizeof(SCtgCacheOperation)); + op->opId = CTG_OP_CLEAR_CACHE; + op->syncOp = syncOp; + + SCtgClearCacheMsg *msg = taosMemoryMalloc(sizeof(SCtgClearCacheMsg)); + if (NULL == msg) { + ctgError("malloc %d failed", (int32_t)sizeof(SCtgClearCacheMsg)); + CTG_ERR_RET(TSDB_CODE_CTG_MEM_ERROR); + } + + msg->pCtg = pCtg; + op->data = msg; + + CTG_ERR_JRET(ctgEnqueue(pCtg, op)); + + return TSDB_CODE_SUCCESS; + +_return: + + taosMemoryFreeClear(msg); + + CTG_RET(code); +} + + int32_t ctgMetaRentInit(SCtgRentMgmt *mgmt, uint32_t rentSec, int8_t type) { mgmt->slotRIdx = 0; mgmt->slotNum = rentSec / CTG_RENT_SLOT_SECOND; @@ -1019,19 +1051,19 @@ int32_t ctgMetaRentAdd(SCtgRentMgmt *mgmt, void *meta, int64_t id, int32_t size) if (NULL == slot->meta) { slot->meta = taosArrayInit(CTG_DEFAULT_RENT_SLOT_SIZE, size); if (NULL == slot->meta) { - qError("taosArrayInit %d failed, id:%"PRIx64", slot idx:%d, type:%d", CTG_DEFAULT_RENT_SLOT_SIZE, id, widx, mgmt->type); + qError("taosArrayInit %d failed, id:0x%"PRIx64", slot idx:%d, type:%d", CTG_DEFAULT_RENT_SLOT_SIZE, id, widx, mgmt->type); CTG_ERR_JRET(TSDB_CODE_CTG_MEM_ERROR); } } if (NULL == taosArrayPush(slot->meta, meta)) { - qError("taosArrayPush meta to rent failed, id:%"PRIx64", slot idx:%d, type:%d", id, widx, mgmt->type); + qError("taosArrayPush meta to rent failed, id:0x%"PRIx64", slot idx:%d, type:%d", id, widx, mgmt->type); CTG_ERR_JRET(TSDB_CODE_CTG_MEM_ERROR); } slot->needSort = true; - qDebug("add meta to rent, id:%"PRIx64", slot idx:%d, type:%d", id, widx, mgmt->type); + qDebug("add meta to rent, id:0x%"PRIx64", slot idx:%d, type:%d", id, widx, mgmt->type); _return: @@ -1047,7 +1079,7 @@ int32_t ctgMetaRentUpdate(SCtgRentMgmt *mgmt, void *meta, int64_t id, int32_t si CTG_LOCK(CTG_WRITE, &slot->lock); if (NULL == slot->meta) { - qError("empty meta slot, id:%"PRIx64", slot idx:%d, type:%d", id, widx, mgmt->type); + qError("empty meta slot, id:0x%"PRIx64", slot idx:%d, type:%d", id, widx, mgmt->type); CTG_ERR_JRET(TSDB_CODE_CTG_INTERNAL_ERROR); } @@ -1060,20 +1092,20 @@ int32_t ctgMetaRentUpdate(SCtgRentMgmt *mgmt, void *meta, int64_t id, int32_t si void *orig = taosArraySearch(slot->meta, &id, searchCompare, TD_EQ); if (NULL == orig) { - qDebug("meta not found in slot, id:%"PRIx64", slot idx:%d, type:%d, size:%d", id, widx, mgmt->type, (int32_t)taosArrayGetSize(slot->meta)); + qDebug("meta not found in slot, id:0x%"PRIx64", slot idx:%d, type:%d, size:%d", id, widx, mgmt->type, (int32_t)taosArrayGetSize(slot->meta)); CTG_ERR_JRET(TSDB_CODE_CTG_INTERNAL_ERROR); } memcpy(orig, meta, size); - qDebug("meta in rent updated, id:%"PRIx64", slot idx:%d, type:%d", id, widx, mgmt->type); + qDebug("meta in rent updated, id:0x%"PRIx64", slot idx:%d, type:%d", id, widx, mgmt->type); _return: CTG_UNLOCK(CTG_WRITE, &slot->lock); if (code) { - qDebug("meta in rent update failed, will try to add it, code:%x, id:%"PRIx64", slot idx:%d, type:%d", code, id, widx, mgmt->type); + qDebug("meta in rent update failed, will try to add it, code:%x, id:0x%"PRIx64", slot idx:%d, type:%d", code, id, widx, mgmt->type); CTG_RET(ctgMetaRentAdd(mgmt, meta, id, size)); } @@ -1088,7 +1120,7 @@ int32_t ctgMetaRentRemove(SCtgRentMgmt *mgmt, int64_t id, __compar_fn_t sortComp CTG_LOCK(CTG_WRITE, &slot->lock); if (NULL == slot->meta) { - qError("empty meta slot, id:%"PRIx64", slot idx:%d, type:%d", id, widx, mgmt->type); + qError("empty meta slot, id:0x%"PRIx64", slot idx:%d, type:%d", id, widx, mgmt->type); CTG_ERR_JRET(TSDB_CODE_CTG_INTERNAL_ERROR); } @@ -1100,13 +1132,13 @@ int32_t ctgMetaRentRemove(SCtgRentMgmt *mgmt, int64_t id, __compar_fn_t sortComp int32_t idx = taosArraySearchIdx(slot->meta, &id, searchCompare, TD_EQ); if (idx < 0) { - qError("meta not found in slot, id:%"PRIx64", slot idx:%d, type:%d", id, widx, mgmt->type); + qError("meta not found in slot, id:0x%"PRIx64", slot idx:%d, type:%d", id, widx, mgmt->type); CTG_ERR_JRET(TSDB_CODE_CTG_INTERNAL_ERROR); } taosArrayRemove(slot->meta, idx); - qDebug("meta in rent removed, id:%"PRIx64", slot idx:%d, type:%d", id, widx, mgmt->type); + qDebug("meta in rent removed, id:0x%"PRIx64", slot idx:%d, type:%d", id, widx, mgmt->type); _return: @@ -1219,11 +1251,11 @@ int32_t ctgAddNewDBCache(SCatalog *pCtg, const char *dbFName, uint64_t dbId) { SDbVgVersion vgVersion = {.dbId = newDBCache.dbId, .vgVersion = -1}; strncpy(vgVersion.dbFName, dbFName, sizeof(vgVersion.dbFName)); - ctgDebug("db added to cache, dbFName:%s, dbId:%"PRIx64, dbFName, dbId); + ctgDebug("db added to cache, dbFName:%s, dbId:0x%"PRIx64, dbFName, dbId); CTG_ERR_RET(ctgMetaRentAdd(&pCtg->dbRent, &vgVersion, dbId, sizeof(SDbVgVersion))); - ctgDebug("db added to rent, dbFName:%s, vgVersion:%d, dbId:%"PRIx64, dbFName, vgVersion.vgVersion, dbId); + ctgDebug("db added to rent, dbFName:%s, vgVersion:%d, dbId:0x%"PRIx64, dbFName, vgVersion.vgVersion, dbId); return TSDB_CODE_SUCCESS; @@ -1246,7 +1278,7 @@ void ctgRemoveStbRent(SCatalog* pCtg, SCtgDBCache *dbCache) { suid = taosHashGetKey(pIter, NULL); if (TSDB_CODE_SUCCESS == ctgMetaRentRemove(&pCtg->stbRent, *suid, ctgStbVersionSortCompare, ctgStbVersionSearchCompare)) { - ctgDebug("stb removed from rent, suid:%"PRIx64, *suid); + ctgDebug("stb removed from rent, suid:0x%"PRIx64, *suid); } pIter = taosHashIterate(dbCache->stbCache, pIter); @@ -1257,7 +1289,7 @@ void ctgRemoveStbRent(SCatalog* pCtg, SCtgDBCache *dbCache) { int32_t ctgRemoveDBFromCache(SCatalog* pCtg, SCtgDBCache *dbCache, const char* dbFName) { uint64_t dbId = dbCache->dbId; - ctgInfo("start to remove db from cache, dbFName:%s, dbId:%"PRIx64, dbFName, dbCache->dbId); + ctgInfo("start to remove db from cache, dbFName:%s, dbId:0x%"PRIx64, dbFName, dbCache->dbId); CTG_LOCK(CTG_WRITE, &dbCache->dbLock); @@ -1268,7 +1300,7 @@ int32_t ctgRemoveDBFromCache(SCatalog* pCtg, SCtgDBCache *dbCache, const char* d CTG_UNLOCK(CTG_WRITE, &dbCache->dbLock); CTG_ERR_RET(ctgMetaRentRemove(&pCtg->dbRent, dbId, ctgDbVgVersionSortCompare, ctgDbVgVersionSearchCompare)); - ctgDebug("db removed from rent, dbFName:%s, dbId:%"PRIx64, dbFName, dbId); + ctgDebug("db removed from rent, dbFName:%s, dbId:0x%"PRIx64, dbFName, dbId); if (taosHashRemove(pCtg->dbCache, dbFName, strlen(dbFName))) { ctgInfo("taosHashRemove from dbCache failed, may be removed, dbFName:%s", dbFName); @@ -1276,7 +1308,7 @@ int32_t ctgRemoveDBFromCache(SCatalog* pCtg, SCtgDBCache *dbCache, const char* d } CTG_CACHE_STAT_DEC(dbNum, 1); - ctgInfo("db removed from cache, dbFName:%s, dbId:%"PRIx64, dbFName, dbId); + ctgInfo("db removed from cache, dbFName:%s, dbId:0x%"PRIx64, dbFName, dbId); return TSDB_CODE_SUCCESS; } @@ -1339,7 +1371,7 @@ int32_t ctgUpdateRentStbVersion(SCatalog *pCtg, char* dbFName, char* tbName, uin CTG_ERR_RET(ctgMetaRentUpdate(&pCtg->stbRent, &metaRent, metaRent.suid, sizeof(SSTableVersion), ctgStbVersionSortCompare, ctgStbVersionSearchCompare)); - ctgDebug("db %s,%" PRIx64 " stb %s,%" PRIx64 " sver %d tver %d smaVer %d updated to stbRent", + ctgDebug("db %s,0x%" PRIx64 " stb %s,0x%" PRIx64 " sver %d tver %d smaVer %d updated to stbRent", dbFName, dbId, tbName, suid, metaRent.sversion, metaRent.tversion, metaRent.smaVer); return TSDB_CODE_SUCCESS; @@ -1349,7 +1381,7 @@ int32_t ctgUpdateRentStbVersion(SCatalog *pCtg, char* dbFName, char* tbName, uin int32_t ctgWriteTbMetaToCache(SCatalog *pCtg, SCtgDBCache *dbCache, char *dbFName, uint64_t dbId, char *tbName, STableMeta *meta, int32_t metaSize) { if (NULL == dbCache->tbCache || NULL == dbCache->stbCache) { taosMemoryFree(meta); - ctgError("db is dropping, dbId:%"PRIx64, dbCache->dbId); + ctgError("db is dropping, dbId:0x%"PRIx64, dbCache->dbId); CTG_ERR_RET(TSDB_CODE_CTG_DB_DROPPED); } @@ -1370,10 +1402,10 @@ int32_t ctgWriteTbMetaToCache(SCatalog *pCtg, SCtgDBCache *dbCache, char *dbFNam if (origType == TSDB_SUPER_TABLE) { if (taosHashRemove(dbCache->stbCache, &orig->suid, sizeof(orig->suid))) { - ctgError("stb not exist in stbCache, dbFName:%s, stb:%s, suid:%"PRIx64, dbFName, tbName, orig->suid); + ctgError("stb not exist in stbCache, dbFName:%s, stb:%s, suid:0x%"PRIx64, dbFName, tbName, orig->suid); } else { CTG_CACHE_STAT_DEC(stblNum, 1); - ctgDebug("stb removed from stbCache, dbFName:%s, stb:%s, suid:%"PRIx64, dbFName, tbName, orig->suid); + ctgDebug("stb removed from stbCache, dbFName:%s, stb:%s, suid:0x%"PRIx64, dbFName, tbName, orig->suid); } origSuid = orig->suid; @@ -1407,13 +1439,13 @@ int32_t ctgWriteTbMetaToCache(SCatalog *pCtg, SCtgDBCache *dbCache, char *dbFNam } if (origSuid != meta->suid && taosHashPut(dbCache->stbCache, &meta->suid, sizeof(meta->suid), tbName, strlen(tbName) + 1) != 0) { - ctgError("taosHashPut to stable cache failed, suid:%"PRIx64, meta->suid); + ctgError("taosHashPut to stable cache failed, suid:0x%"PRIx64, meta->suid); CTG_ERR_RET(TSDB_CODE_CTG_MEM_ERROR); } CTG_CACHE_STAT_INC(stblNum, 1); - ctgDebug("stb %" PRIx64 " updated to cache, dbFName:%s, tbName:%s, tbType:%d", meta->suid, dbFName, tbName, meta->tableType); + ctgDebug("stb 0x%" PRIx64 " updated to cache, dbFName:%s, tbName:%s, tbType:%d", meta->suid, dbFName, tbName, meta->tableType); CTG_ERR_RET(ctgUpdateRentStbVersion(pCtg, dbFName, tbName, dbId, meta->suid, pCache)); @@ -1424,7 +1456,7 @@ int32_t ctgWriteTbIndexToCache(SCatalog *pCtg, SCtgDBCache *dbCache, char* dbFNa if (NULL == dbCache->tbCache) { ctgFreeSTableIndex(*index); taosMemoryFreeClear(*index); - ctgError("db is dropping, dbId:%"PRIx64, dbCache->dbId); + ctgError("db is dropping, dbId:0x%"PRIx64, dbCache->dbId); CTG_ERR_RET(TSDB_CODE_CTG_DB_DROPPED); } @@ -1510,7 +1542,7 @@ int32_t ctgOpUpdateVgroup(SCtgCacheOperation *operation) { SCtgDBCache *dbCache = NULL; CTG_ERR_RET(ctgGetAddDBCache(msg->pCtg, dbFName, msg->dbId, &dbCache)); if (NULL == dbCache) { - ctgInfo("conflict db update, ignore this update, dbFName:%s, dbId:%"PRIx64, dbFName, msg->dbId); + ctgInfo("conflict db update, ignore this update, dbFName:%s, dbId:0x%"PRIx64, dbFName, msg->dbId); CTG_ERR_RET(TSDB_CODE_CTG_INTERNAL_ERROR); } @@ -1540,7 +1572,7 @@ int32_t ctgOpUpdateVgroup(SCtgCacheOperation *operation) { vgCache->vgInfo = dbInfo; msg->dbInfo = NULL; - ctgDebug("db vgInfo updated, dbFName:%s, vgVer:%d, dbId:%"PRIx64, dbFName, vgVersion.vgVersion, vgVersion.dbId); + ctgDebug("db vgInfo updated, dbFName:%s, vgVer:%d, dbId:0x%"PRIx64, dbFName, vgVersion.vgVersion, vgVersion.dbId); ctgWUnlockVgInfo(dbCache); @@ -1569,7 +1601,7 @@ int32_t ctgOpDropDbCache(SCtgCacheOperation *operation) { } if (dbCache->dbId != msg->dbId) { - ctgInfo("dbId already updated, dbFName:%s, dbId:%"PRIx64 ", targetId:%"PRIx64, msg->dbFName, dbCache->dbId, msg->dbId); + ctgInfo("dbId already updated, dbFName:%s, dbId:0x%"PRIx64 ", targetId:0x%"PRIx64, msg->dbFName, dbCache->dbId, msg->dbId); goto _return; } @@ -1629,7 +1661,7 @@ int32_t ctgOpUpdateTbMeta(SCtgCacheOperation *operation) { CTG_ERR_JRET(ctgGetAddDBCache(pCtg, pMeta->dbFName, pMeta->dbId, &dbCache)); if (NULL == dbCache) { - ctgInfo("conflict db update, ignore this update, dbFName:%s, dbId:%" PRIx64, pMeta->dbFName, pMeta->dbId); + ctgInfo("conflict db update, ignore this update, dbFName:%s, dbId:0x%" PRIx64, pMeta->dbFName, pMeta->dbId); CTG_ERR_JRET(TSDB_CODE_CTG_INTERNAL_ERROR); } @@ -1673,27 +1705,28 @@ int32_t ctgOpDropStbMeta(SCtgCacheOperation *operation) { } if (msg->dbId && (dbCache->dbId != msg->dbId)) { - ctgDebug("dbId already modified, dbFName:%s, current:%"PRIx64", dbId:%"PRIx64", stb:%s, suid:%"PRIx64, msg->dbFName, dbCache->dbId, msg->dbId, msg->stbName, msg->suid); + ctgDebug("dbId already modified, dbFName:%s, current:0x%"PRIx64", dbId:0x%"PRIx64", stb:%s, suid:0x%"PRIx64, + msg->dbFName, dbCache->dbId, msg->dbId, msg->stbName, msg->suid); return TSDB_CODE_SUCCESS; } if (taosHashRemove(dbCache->stbCache, &msg->suid, sizeof(msg->suid))) { - ctgDebug("stb not exist in stbCache, may be removed, dbFName:%s, stb:%s, suid:%"PRIx64, msg->dbFName, msg->stbName, msg->suid); + ctgDebug("stb not exist in stbCache, may be removed, dbFName:%s, stb:%s, suid:0x%"PRIx64, msg->dbFName, msg->stbName, msg->suid); } else { CTG_CACHE_STAT_DEC(stblNum, 1); } if (taosHashRemove(dbCache->tbCache, msg->stbName, strlen(msg->stbName))) { - ctgError("stb not exist in cache, dbFName:%s, stb:%s, suid:%"PRIx64, msg->dbFName, msg->stbName, msg->suid); + ctgError("stb not exist in cache, dbFName:%s, stb:%s, suid:0x%"PRIx64, msg->dbFName, msg->stbName, msg->suid); } else { CTG_CACHE_STAT_DEC(tblNum, 1); } - ctgInfo("stb removed from cache, dbFName:%s, stbName:%s, suid:%"PRIx64, msg->dbFName, msg->stbName, msg->suid); + ctgInfo("stb removed from cache, dbFName:%s, stbName:%s, suid:0x%"PRIx64, msg->dbFName, msg->stbName, msg->suid); CTG_ERR_JRET(ctgMetaRentRemove(&msg->pCtg->stbRent, msg->suid, ctgStbVersionSortCompare, ctgStbVersionSearchCompare)); - ctgDebug("stb removed from rent, dbFName:%s, stbName:%s, suid:%"PRIx64, msg->dbFName, msg->stbName, msg->suid); + ctgDebug("stb removed from rent, dbFName:%s, stbName:%s, suid:0x%"PRIx64, msg->dbFName, msg->stbName, msg->suid); _return: @@ -1714,7 +1747,7 @@ int32_t ctgOpDropTbMeta(SCtgCacheOperation *operation) { } if (dbCache->dbId != msg->dbId) { - ctgDebug("dbId %" PRIx64 " not match with curId %"PRIx64", dbFName:%s, tbName:%s", msg->dbId, dbCache->dbId, msg->dbFName, msg->tbName); + ctgDebug("dbId 0x%" PRIx64 " not match with curId 0x%"PRIx64", dbFName:%s, tbName:%s", msg->dbId, dbCache->dbId, msg->dbFName, msg->tbName); return TSDB_CODE_SUCCESS; } @@ -1898,6 +1931,37 @@ _return: } +int32_t ctgOpClearCache(SCtgCacheOperation *operation) { + int32_t code = 0; + SCtgClearCacheMsg *msg = operation->data; + SCatalog* pCtg = msg->pCtg; + + if (pCtg) { + catalogFreeHandle(pCtg); + goto _return; + } + + void* pIter = taosHashIterate(gCtgMgmt.pCluster, NULL); + while (pIter) { + pCtg = *(SCatalog**)pIter; + + if (pCtg) { + catalogFreeHandle(pCtg); + } + + pIter = taosHashIterate(gCtgMgmt.pCluster, pIter); + } + + taosHashClear(gCtgMgmt.pCluster); + +_return: + + taosMemoryFreeClear(msg); + + CTG_RET(code); +} + + void ctgUpdateThreadUnexpectedStopped(void) { if (!atomic_load_8((int8_t*)&gCtgMgmt.exit) && CTG_IS_LOCKED(&gCtgMgmt.lock) > 0) CTG_UNLOCK(CTG_READ, &gCtgMgmt.lock); } @@ -1971,11 +2035,10 @@ void* ctgUpdateThreadFunc(void* param) { CTG_RT_STAT_INC(qDoneNum, 1); + ctgdShowCacheInfo(); ctgdShowClusterCache(pCtg); } - if (CTG_IS_LOCKED(&gCtgMgmt.lock)) CTG_UNLOCK(CTG_READ, &gCtgMgmt.lock); - qInfo("catalog update thread stopped"); return NULL; diff --git a/source/libs/catalog/src/ctgDbg.c b/source/libs/catalog/src/ctgDbg.c index ff93bedb213607c3592b057c12beb2ed2f733d54..7f2b919f175adfa79a5bc3916813c8b6619e1b3f 100644 --- a/source/libs/catalog/src/ctgDbg.c +++ b/source/libs/catalog/src/ctgDbg.c @@ -19,7 +19,7 @@ #include "catalogInt.h" extern SCatalogMgmt gCtgMgmt; -SCtgDebug gCTGDebug = {.lockEnable = true, .apiEnable = true}; +SCtgDebug gCTGDebug = {.cacheEnable = true}; void ctgdUserCallback(SMetaData* pResult, void* param, int32_t code) { ASSERT(*(int32_t*)param == 1); @@ -40,9 +40,9 @@ void ctgdUserCallback(SMetaData* pResult, void* param, int32_t code) { STableComInfo *c = &p->tableInfo; if (TSDB_CHILD_TABLE == p->tableType) { - qDebug("table meta: type:%d, vgId:%d, uid:%" PRIx64 ",suid:%" PRIx64, p->tableType, p->vgId, p->uid, p->suid); + qDebug("table meta: type:%d, vgId:%d, uid:0x%" PRIx64 ",suid:0x%" PRIx64, p->tableType, p->vgId, p->uid, p->suid); } else { - qDebug("table meta: type:%d, vgId:%d, uid:%" PRIx64 ",suid:%" PRIx64 ",sv:%d, tv:%d, tagNum:%d, precision:%d, colNum:%d, rowSize:%d", + qDebug("table meta: type:%d, vgId:%d, uid:0x%" PRIx64 ",suid:0x%" PRIx64 ",sv:%d, tv:%d, tagNum:%d, precision:%d, colNum:%d, rowSize:%d", p->tableType, p->vgId, p->uid, p->suid, p->sversion, p->tversion, c->numOfTags, c->precision, c->numOfColumns, c->rowSize); } @@ -75,7 +75,7 @@ void ctgdUserCallback(SMetaData* pResult, void* param, int32_t code) { num = taosArrayGetSize(pResult->pDbInfo); for (int32_t i = 0; i < num; ++i) { SDbInfo *pDb = taosArrayGet(pResult->pDbInfo, i); - qDebug("db %d dbInfo: vgVer:%d, tbNum:%d, dbId:%" PRIx64, i, pDb->vgVer, pDb->tbNum, pDb->dbId); + qDebug("db %d dbInfo: vgVer:%d, tbNum:%d, dbId:0x%" PRIx64, i, pDb->vgVer, pDb->tbNum, pDb->dbId); } } else { qDebug("empty db info"); @@ -333,10 +333,10 @@ void ctgdShowTableMeta(SCatalog* pCtg, const char *tbName, STableMeta* p) { STableComInfo *c = &p->tableInfo; if (TSDB_CHILD_TABLE == p->tableType) { - ctgDebug("table [%s] meta: type:%d, vgId:%d, uid:%" PRIx64 ",suid:%" PRIx64, tbName, p->tableType, p->vgId, p->uid, p->suid); + ctgDebug("table [%s] meta: type:%d, vgId:%d, uid:0x%" PRIx64 ",suid:0x%" PRIx64, tbName, p->tableType, p->vgId, p->uid, p->suid); return; } else { - ctgDebug("table [%s] meta: type:%d, vgId:%d, uid:%" PRIx64 ",suid:%" PRIx64 ",sv:%d, tv:%d, tagNum:%d, precision:%d, colNum:%d, rowSize:%d", + ctgDebug("table [%s] meta: type:%d, vgId:%d, uid:0x%" PRIx64 ",suid:0x%" PRIx64 ",sv:%d, tv:%d, tagNum:%d, precision:%d, colNum:%d, rowSize:%d", tbName, p->tableType, p->vgId, p->uid, p->suid, p->sversion, p->tversion, c->numOfTags, c->precision, c->numOfColumns, c->rowSize); } @@ -377,7 +377,7 @@ void ctgdShowDBCache(SCatalog* pCtg, SHashObj *dbHash) { } } - ctgDebug("[%d] db [%.*s][%"PRIx64"] %s: metaNum:%d, stbNum:%d, vgVersion:%d, hashMethod:%d, vgNum:%d", + ctgDebug("[%d] db [%.*s][0x%"PRIx64"] %s: metaNum:%d, stbNum:%d, vgVersion:%d, hashMethod:%d, vgNum:%d", i, (int32_t)len, dbFName, dbCache->dbId, dbCache->deleted?"deleted":"", metaNum, stbNum, vgVersion, hashMethod, vgNum); pIter = taosHashIterate(dbHash, pIter); @@ -392,13 +392,13 @@ void ctgdShowClusterCache(SCatalog* pCtg) { return; } - ctgDebug("## cluster %"PRIx64" %p cache Info BEGIN ##", pCtg->clusterId, pCtg); + ctgDebug("## cluster 0x%"PRIx64" %p cache Info BEGIN ##", pCtg->clusterId, pCtg); ctgDebug("db:%d meta:%d stb:%d dbRent:%d stbRent:%d", ctgdGetClusterCacheNum(pCtg, CTG_DBG_DB_NUM), ctgdGetClusterCacheNum(pCtg, CTG_DBG_META_NUM), ctgdGetClusterCacheNum(pCtg, CTG_DBG_STB_NUM), ctgdGetClusterCacheNum(pCtg, CTG_DBG_DB_RENT_NUM), ctgdGetClusterCacheNum(pCtg, CTG_DBG_STB_RENT_NUM)); ctgdShowDBCache(pCtg, pCtg->dbCache); - ctgDebug("## cluster %"PRIx64" %p cache Info END ##", pCtg->clusterId, pCtg); + ctgDebug("## cluster 0x%"PRIx64" %p cache Info END ##", pCtg->clusterId, pCtg); } int32_t ctgdShowCacheInfo(void) { @@ -407,6 +407,8 @@ int32_t ctgdShowCacheInfo(void) { } CTG_API_ENTER(); + + qDebug("# total catalog cluster number %d #", taosHashGetSize(gCtgMgmt.pCluster)); SCatalog *pCtg = NULL; void *pIter = taosHashIterate(gCtgMgmt.pCluster, NULL); diff --git a/source/libs/catalog/src/ctgRemote.c b/source/libs/catalog/src/ctgRemote.c index 178025704f7e91045e8970844e0d5590baa5a052..fa1a26283286d01cc4574c4babbb28ad5fb36996 100644 --- a/source/libs/catalog/src/ctgRemote.c +++ b/source/libs/catalog/src/ctgRemote.c @@ -186,13 +186,13 @@ int32_t ctgHandleMsgCallback(void *param, const SDataBuf *pMsg, int32_t rspCode) SCtgJob* pJob = taosAcquireRef(gCtgMgmt.jobPool, cbParam->refId); if (NULL == pJob) { - qDebug("job refId %" PRIx64 " already dropped", cbParam->refId); + qDebug("ctg job refId 0x%" PRIx64 " already dropped", cbParam->refId); goto _return; } SCtgTask *pTask = taosArrayGet(pJob->pTasks, cbParam->taskId); - qDebug("QID:0x%" PRIx64 " task %d start to handle rsp %s", pJob->queryId, pTask->taskId, TMSG_INFO(cbParam->reqType + 1)); + qDebug("QID:0x%" PRIx64 " ctg task %d start to handle rsp %s", pJob->queryId, pTask->taskId, TMSG_INFO(cbParam->reqType + 1)); CTG_ERR_JRET((*gCtgAsyncFps[pTask->type].handleRspFp)(pTask, cbParam->reqType, pMsg, rspCode)); @@ -263,7 +263,7 @@ int32_t ctgAsyncSendMsg(SCatalog* pCtg, SRequestConnInfo *pConn, SCtgTask* pTask CTG_ERR_JRET(code); } - ctgDebug("req msg sent, reqId:0x%" PRIx64 ", msg type:%d, %s", pTask->pJob->queryId, msgType, TMSG_INFO(msgType)); + ctgDebug("ctg req msg sent, reqId:0x%" PRIx64 ", msg type:%d, %s", pTask->pJob->queryId, msgType, TMSG_INFO(msgType)); return TSDB_CODE_SUCCESS; _return: diff --git a/source/libs/catalog/src/ctgUtil.c b/source/libs/catalog/src/ctgUtil.c index e97c34dc268b61b402304aeb51689a99926484ec..476eb371b057e27a2420dd6b35582b58db8aca72 100644 --- a/source/libs/catalog/src/ctgUtil.c +++ b/source/libs/catalog/src/ctgUtil.c @@ -434,7 +434,7 @@ void ctgFreeJob(void* job) { taosMemoryFree(job); - qDebug("QID:%" PRIx64 ", job %" PRIx64 " freed", qid, rid); + qDebug("QID:0x%" PRIx64 ", ctg job 0x%" PRIx64 " freed", qid, rid); } int32_t ctgUpdateMsgCtx(SCtgMsgCtx* pCtx, int32_t reqType, void* out, char* target) { diff --git a/source/libs/command/src/command.c b/source/libs/command/src/command.c index f330b7ce162f9c79189a66c0772fffbc96cb8dcc..c6a0e52a42f6238c7ee3b2cd3af714408debafa2 100644 --- a/source/libs/command/src/command.c +++ b/source/libs/command/src/command.c @@ -14,6 +14,7 @@ */ #include "command.h" +#include "catalog.h" #include "tdatablock.h" static int32_t getSchemaBytes(const SSchema* pSchema) { @@ -46,7 +47,8 @@ static SSDataBlock* buildDescResultDataBlock() { taosArrayPush(pBlock->pDataBlock, &infoData); infoData.info.type = TSDB_DATA_TYPE_INT; - infoData.info.bytes = tDataTypes[TSDB_DATA_TYPE_INT].bytes;; + infoData.info.bytes = tDataTypes[TSDB_DATA_TYPE_INT].bytes; + taosArrayPush(pBlock->pDataBlock, &infoData); infoData.info.type = TSDB_DATA_TYPE_VARCHAR; @@ -62,7 +64,7 @@ static void setDescResultIntoDataBlock(SSDataBlock* pBlock, int32_t numOfRows, S // field SColumnInfoData* pCol1 = taosArrayGet(pBlock->pDataBlock, 0); - char buf[DESCRIBE_RESULT_FIELD_LEN] = {0}; + char buf[DESCRIBE_RESULT_FIELD_LEN] = {0}; for (int32_t i = 0; i < numOfRows; ++i) { STR_TO_VARSTR(buf, pMeta->schema[i].name); colDataAppend(pCol1, i, buf, false); @@ -91,8 +93,8 @@ static void setDescResultIntoDataBlock(SSDataBlock* pBlock, int32_t numOfRows, S } static int32_t execDescribe(SNode* pStmt, SRetrieveTableRsp** pRsp) { - SDescribeStmt* pDesc = (SDescribeStmt*) pStmt; - int32_t numOfRows = TABLE_TOTAL_COL_NUM(pDesc->pMeta); + SDescribeStmt* pDesc = (SDescribeStmt*)pStmt; + int32_t numOfRows = TABLE_TOTAL_COL_NUM(pDesc->pMeta); SSDataBlock* pBlock = buildDescResultDataBlock(); setDescResultIntoDataBlock(pBlock, numOfRows, pDesc->pMeta); @@ -119,10 +121,15 @@ static int32_t execDescribe(SNode* pStmt, SRetrieveTableRsp** pRsp) { return TSDB_CODE_SUCCESS; } -static int32_t execResetQueryCache() { - // todo - return TSDB_CODE_SUCCESS; -} +static int32_t execResetQueryCache() { return catalogClearCache(); } + +static int32_t execShowCreateDatabase(SShowCreateDatabaseStmt* pStmt) { return TSDB_CODE_FAILED; } + +static int32_t execShowCreateTable(SShowCreateTableStmt* pStmt) { return TSDB_CODE_FAILED; } + +static int32_t execShowCreateSTable(SShowCreateTableStmt* pStmt) { return TSDB_CODE_FAILED; } + +static int32_t execAlterLocal(SAlterLocalStmt* pStmt) { return TSDB_CODE_FAILED; } int32_t qExecCommand(SNode* pStmt, SRetrieveTableRsp** pRsp) { switch (nodeType(pStmt)) { @@ -130,6 +137,14 @@ int32_t qExecCommand(SNode* pStmt, SRetrieveTableRsp** pRsp) { return execDescribe(pStmt, pRsp); case QUERY_NODE_RESET_QUERY_CACHE_STMT: return execResetQueryCache(); + case QUERY_NODE_SHOW_CREATE_DATABASE_STMT: + return execShowCreateDatabase((SShowCreateDatabaseStmt*)pStmt); + case QUERY_NODE_SHOW_CREATE_TABLE_STMT: + return execShowCreateTable((SShowCreateTableStmt*)pStmt); + case QUERY_NODE_SHOW_CREATE_STABLE_STMT: + return execShowCreateSTable((SShowCreateTableStmt*)pStmt); + case QUERY_NODE_ALTER_LOCAL_STMT: + return execAlterLocal((SAlterLocalStmt*)pStmt); default: break; } diff --git a/source/libs/command/src/explain.c b/source/libs/command/src/explain.c index 79861dfa05549a2f5faaa903b7d01bedfe73cd43..8f91282480ba8d0d662628358ed3034848de9297 100644 --- a/source/libs/command/src/explain.c +++ b/source/libs/command/src/explain.c @@ -194,6 +194,11 @@ int32_t qExplainGenerateResChildren(SPhysiNode *pNode, SExplainGroup *group, SNo pPhysiChildren = fillPhysiNode->node.pChildren; break; } + case QUERY_NODE_PHYSICAL_PLAN_TABLE_MERGE_SCAN: { + STableMergeScanPhysiNode *mergePhysiNode = (STableMergeScanPhysiNode *)pNode; + pPhysiChildren = mergePhysiNode->scan.node.pChildren; + break; + } default: qError("not supported physical node type %d", pNode->type); QRY_ERR_RET(TSDB_CODE_QRY_APP_ERROR); @@ -398,6 +403,7 @@ int32_t qExplainResNodeToRowsImpl(SExplainResNode *pResNode, SExplainCtx *ctx, i break; } case QUERY_NODE_PHYSICAL_PLAN_TABLE_SEQ_SCAN: + case QUERY_NODE_PHYSICAL_PLAN_TABLE_MERGE_SCAN: case QUERY_NODE_PHYSICAL_PLAN_TABLE_SCAN: { STableScanPhysiNode *pTblScanNode = (STableScanPhysiNode *)pNode; EXPLAIN_ROW_NEW(level, EXPLAIN_TBL_SCAN_FORMAT, pTblScanNode->scan.tableName.tname); diff --git a/source/libs/executor/inc/executil.h b/source/libs/executor/inc/executil.h index a70cff555297ec3ee3b3e497dc38d65c1d5e460d..07686893db5412e4bb3a512c102a1dd39af2ecbb 100644 --- a/source/libs/executor/inc/executil.h +++ b/source/libs/executor/inc/executil.h @@ -15,7 +15,9 @@ #ifndef TDENGINE_QUERYUTIL_H #define TDENGINE_QUERYUTIL_H -#include +#include "function.h" +#include "nodes.h" +#include "plannodes.h" #include "tbuffer.h" #include "tcommon.h" #include "tpagedbuf.h" @@ -77,7 +79,7 @@ typedef struct SResultRowInfo { struct SqlFunctionCtx; size_t getResultRowSize(struct SqlFunctionCtx* pCtx, int32_t numOfOutput); -int32_t initResultRowInfo(SResultRowInfo* pResultRowInfo, int32_t size); +void initResultRowInfo(SResultRowInfo* pResultRowInfo); void cleanupResultRowInfo(SResultRowInfo* pResultRowInfo); void closeAllResultRows(SResultRowInfo* pResultRowInfo); @@ -86,7 +88,7 @@ void initResultRow(SResultRow *pResultRow); void closeResultRow(SResultRow* pResultRow); bool isResultRowClosed(SResultRow* pResultRow); -struct SResultRowEntryInfo* getResultCell(const SResultRow* pRow, int32_t index, const int32_t* offset); +struct SResultRowEntryInfo* getResultEntryInfo(const SResultRow* pRow, int32_t index, const int32_t* offset); static FORCE_INLINE SResultRow *getResultRowByPos(SDiskbasedBuf* pBuf, SResultRowPosition* pos) { SFilePage* bufPage = (SFilePage*) getBufPage(pBuf, pos->pageId); @@ -98,9 +100,27 @@ void initGroupedResultInfo(SGroupResInfo* pGroupResInfo, SHashObj* pHashmap, void initMultiResInfoFromArrayList(SGroupResInfo* pGroupResInfo, SArray* pArrayList); void cleanupGroupResInfo(SGroupResInfo* pGroupResInfo); -bool hashRemainDataInGroupInfo(SGroupResInfo* pGroupResInfo); +bool hasDataInGroupInfo(SGroupResInfo* pGroupResInfo); -bool incNextGroup(SGroupResInfo* pGroupResInfo); int32_t getNumOfTotalRes(SGroupResInfo* pGroupResInfo); +SSDataBlock* createResDataBlock(SDataBlockDescNode* pNode); + +int32_t getTableList(void* metaHandle, SScanPhysiNode* pScanNode, STableListInfo* pListInfo, SNode* pTagCond); +SArray* createSortInfo(SNodeList* pNodeList); +SArray* extractPartitionColInfo(SNodeList* pNodeList); +SArray* extractColMatchInfo(SNodeList* pNodeList, SDataBlockDescNode* pOutputNodeList, int32_t* numOfOutputCols, int32_t type); + +SExprInfo* createExprInfo(SNodeList* pNodeList, SNodeList* pGroupKeys, int32_t* numOfExprs); + +SqlFunctionCtx* createSqlFunctionCtx(SExprInfo* pExprInfo, int32_t numOfOutput, int32_t** rowEntryInfoOffset); +void relocateColumnData(SSDataBlock* pBlock, const SArray* pColMatchInfo, SArray* pCols); +void initExecTimeWindowInfo(SColumnInfoData* pColData, STimeWindow* pQueryWindow); + +SInterval extractIntervalInfo(const STableScanPhysiNode* pTableScanNode); +SColumn extractColumnFromColumnNode(SColumnNode* pColNode); + +int32_t initQueryTableDataCond(SQueryTableDataCond* pCond, const STableScanPhysiNode* pTableScanNode); +void cleanupQueryTableDataCond(SQueryTableDataCond* pCond); + #endif // TDENGINE_QUERYUTIL_H diff --git a/source/libs/executor/inc/executorimpl.h b/source/libs/executor/inc/executorimpl.h index 034e2893df17a64fb1514d3b56eae6c305a45dee..28241d778c77af6a24abdc9d63c169bff50d021b 100644 --- a/source/libs/executor/inc/executorimpl.h +++ b/source/libs/executor/inc/executorimpl.h @@ -49,8 +49,6 @@ typedef int32_t (*__block_search_fn_t)(char* data, int32_t num, int64_t key, int #define Q_STATUS_EQUAL(p, s) (((p) & (s)) != 0u) #define QUERY_IS_ASC_QUERY(q) (GET_FORWARD_DIRECTION_FACTOR((q)->order.order) == QUERY_ASC_FORWARD_STEP) -//#define GET_TABLEGROUP(q, _index) ((SArray*)taosArrayGetP((q)->tableqinfoGroupInfo.pGroupList, (_index))) - #define NEEDTO_COMPRESS_QUERY(size) ((size) > tsCompressColData ? 1 : 0) enum { @@ -64,11 +62,6 @@ enum { TASK_COMPLETED = 0x2u, }; -typedef struct SResultRowCell { - uint64_t groupId; - SResultRowPosition pos; -} SResultRowCell; - /** * If the number of generated results is greater than this value, * query query will be halt and return results to client immediate. @@ -83,7 +76,6 @@ typedef struct SResultInfo { // TODO refactor typedef struct STableQueryInfo { TSKEY lastKey; // last check ts, todo remove it later SResultRowPosition pos; // current active time window -// SVariant tag; } STableQueryInfo; typedef struct SLimit { @@ -123,41 +115,7 @@ typedef struct SOperatorCostInfo { double totalCost; } SOperatorCostInfo; -// The basic query information extracted from the SQueryInfo tree to support the -// execution of query in a data node. -typedef struct STaskAttr { - SLimit limit; - SLimit slimit; - bool stableQuery; // super table query or not - bool topBotQuery; // TODO used bitwise flag - bool groupbyColumn; // denote if this is a groupby normal column query - bool timeWindowInterpo; // if the time window start/end required interpolation - bool tsCompQuery; // is tscomp query - bool diffQuery; // is diff query - bool pointInterpQuery; // point interpolation query - int32_t havingNum; // having expr number - int16_t numOfCols; - int16_t numOfTags; - STimeWindow window; - SInterval interval; - int16_t precision; - int16_t numOfOutput; - int16_t fillType; - int32_t resultRowSize; - int32_t tagLen; // tag value length of current query - - SExprInfo* pExpr1; - SColumnInfo* tagColList; - int32_t numOfFilterCols; - int64_t* fillVal; - void* tsdb; -// STableListInfo tableGroupInfo; // table list - int32_t vgId; -} STaskAttr; - struct SOperatorInfo; -//struct SAggSupporter; -//struct SOptrBasicInfo; typedef int32_t (*__optr_encode_fn_t)(struct SOperatorInfo* pOperator, char** result, int32_t* length); typedef int32_t (*__optr_decode_fn_t)(struct SOperatorInfo* pOperator, char* result); @@ -195,31 +153,6 @@ typedef struct SExecTaskInfo { struct SOperatorInfo* pRoot; } SExecTaskInfo; -typedef struct STaskRuntimeEnv { - STaskAttr* pQueryAttr; - uint32_t status; // query status - uint8_t scanFlag; // denotes reversed scan of data or not - SDiskbasedBuf* pResultBuf; // query result buffer based on blocked-wised disk file - SHashObj* pResultRowHashTable; // quick locate the window object for each result - SHashObj* pResultRowListSet; // used to check if current ResultRowInfo has ResultRow object or not - SArray* pResultRowArrayList; // The array list that contains the Result rows - char* keyBuf; // window key buffer - // The window result objects pool, all the resultRow Objects are allocated and managed by this object. - char** prevRow; - STSBuf* pTsBuf; // timestamp filter list - STSCursor cur; - - char* tagVal; // tag value of current data block -// STableGroupInfo tableqinfoGroupInfo; // this is a table list - struct SOperatorInfo* proot; - SGroupResInfo groupResInfo; - int64_t currentOffset; // dynamic offset value - - STableQueryInfo* current; - SResultInfo resultInfo; - struct SUdfInfo* pUdfInfo; -} STaskRuntimeEnv; - enum { OP_NOT_OPENED = 0x0, OP_OPENED = 0x1, @@ -238,14 +171,20 @@ typedef struct SOperatorFpSet { __optr_explain_fn_t getExplainFn; } SOperatorFpSet; +typedef struct SExprSupp { + SExprInfo* pExprInfo; + int32_t numOfExprs; // the number of scalar expression in group operator + SqlFunctionCtx* pCtx; + int32_t* rowEntryInfoOffset; // offset value for each row result cell info +} SExprSupp; + typedef struct SOperatorInfo { uint8_t operatorType; bool blocking; // block operator or not uint8_t status; // denote if current operator is completed - int32_t numOfExprs; // number of columns of the current operator results - char* name; // name, used to show the query execution plan + char* name; // name, for debug purpose void* info; // extension attribution - SExprInfo* pExpr; + SExprSupp exprSupp; SExecTaskInfo* pTaskInfo; SOperatorCostInfo cost; SResultInfo resultInfo; @@ -260,6 +199,9 @@ typedef enum { EX_SOURCE_DATA_EXHAUSTED = 0x3, } EX_SOURCE_STATUS; +#define COL_MATCH_FROM_COL_ID 0x1 +#define COL_MATCH_FROM_SLOT_ID 0x2 + typedef struct SSourceDataInfo { int32_t index; SRetrieveTableRsp* pRsp; @@ -287,9 +229,6 @@ typedef struct SExchangeInfo { uint64_t self; } SExchangeInfo; -#define COL_MATCH_FROM_COL_ID 0x1 -#define COL_MATCH_FROM_SLOT_ID 0x2 - typedef struct SColMatchInfo { int32_t srcSlotId; // source slot id int32_t colId; @@ -299,8 +238,8 @@ typedef struct SColMatchInfo { } SColMatchInfo; typedef struct SScanInfo { - int32_t numOfAsc; - int32_t numOfDesc; + int32_t numOfAsc; + int32_t numOfDesc; } SScanInfo; typedef struct SSampleExecInfo { @@ -320,17 +259,13 @@ typedef struct STableScanInfo { SNode* pFilterNode; // filter info, which is push down by optimizer SqlFunctionCtx* pCtx; // which belongs to the direct upstream operator operator query context SResultRowInfo* pResultRowInfo; - int32_t* rowCellInfoOffset; + int32_t* rowEntryInfoOffset; SExprInfo* pExpr; SSDataBlock* pResBlock; SArray* pColMatchInfo; int32_t numOfOutput; - SExprInfo* pPseudoExpr; - int32_t numOfPseudoExpr; - SqlFunctionCtx* pPseudoCtx; -// int32_t* rowCellInfoOffset; - + SExprSupp pseudoSup; SQueryTableDataCond cond; int32_t scanFlag; // table scan flag to denote if it is a repeat/reverse/main scan int32_t dataBlockLoadFlag; @@ -436,14 +371,12 @@ typedef struct SBlockDistInfo { void* pHandle; } SBlockDistInfo; +// todo remove this typedef struct SOptrBasicInfo { SResultRowInfo resultRowInfo; - int32_t* rowCellInfoOffset; // offset value for each row result cell info - SqlFunctionCtx* pCtx; SSDataBlock* pRes; } SOptrBasicInfo; -// TODO move the resultrowsiz together with SOptrBasicInfo:rowCellInfoOffset typedef struct SAggSupporter { SHashObj* pResultRowHashTable; // quick locate the window object for each result char* keyBuf; // window key buffer @@ -500,10 +433,7 @@ typedef struct SAggOperatorInfo { STableQueryInfo *current; uint64_t groupId; SGroupResInfo groupResInfo; - SExprInfo *pScalarExprInfo; - int32_t numOfScalarExpr; // the number of scalar expression before the aggregate function can be applied - SqlFunctionCtx *pScalarCtx; // scalar function requried sql function struct. - int32_t *rowCellInfoOffset; // offset value for each row result cell info + SExprSupp scalarExprSup; } SAggOperatorInfo; typedef struct SProjectOperatorInfo { @@ -528,11 +458,7 @@ typedef struct SIndefOperatorInfo { SOptrBasicInfo binfo; SAggSupporter aggSup; SArray* pPseudoColInfo; - - SExprInfo* pScalarExpr; - int32_t numOfScalarExpr; - SqlFunctionCtx* pScalarCtx; - int32_t* rowCellInfoOffset; + SExprSupp scalarSup; } SIndefOperatorInfo; typedef struct SFillOperatorInfo { @@ -544,13 +470,6 @@ typedef struct SFillOperatorInfo { bool multigroupResult; } SFillOperatorInfo; -typedef struct SScalarSupp { - SExprInfo* pScalarExprInfo; - int32_t numOfScalarExpr; // the number of scalar expression in group operator - SqlFunctionCtx* pScalarFuncCtx; - int32_t* rowCellInfoOffset; // offset value for each row result cell info -} SScalarSupp; - typedef struct SGroupbyOperatorInfo { // SOptrBasicInfo should be first, SAggSupporter should be second for stream encode SOptrBasicInfo binfo; @@ -563,7 +482,7 @@ typedef struct SGroupbyOperatorInfo { char* keyBuf; // group by keys for hash int32_t groupKeyLen; // total group by column width SGroupResInfo groupResInfo; - SScalarSupp scalarSup; + SExprSupp scalarSup; } SGroupbyOperatorInfo; typedef struct SDataGroupInfo { @@ -587,7 +506,7 @@ typedef struct SPartitionOperatorInfo { void* pGroupIter; // group iterator int32_t pageIndex; // page index of current group SSDataBlock* pUpdateRes; - SScalarSupp scalarSupp; + SExprSupp scalarSup; } SPartitionOperatorInfo; typedef struct SWindowRowsSup { @@ -740,49 +659,39 @@ SOperatorFpSet createOperatorFpSet(__optr_open_fn_t openFn, __optr_fn_t nextFn, int32_t operatorDummyOpenFn(SOperatorInfo* pOperator); void operatorDummyCloseFn(void* param, int32_t numOfCols); int32_t appendDownstream(SOperatorInfo* p, SOperatorInfo** pDownstream, int32_t num); -int32_t initAggInfo(SOptrBasicInfo* pBasicInfo, SAggSupporter* pAggSup, SExprInfo* pExprInfo, int32_t numOfCols, - SSDataBlock* pResultBlock, size_t keyBufSize, const char* pkey); + +void initBasicInfo(SOptrBasicInfo* pInfo, SSDataBlock* pBlock); +void cleanupBasicInfo(SOptrBasicInfo* pInfo); +void initExprSupp(SExprSupp* pSup, SExprInfo* pExprInfo, int32_t numOfExpr); +void cleanupExprSup(SExprSupp* pSup); +int32_t initAggInfo(SExprSupp *pSup, SAggSupporter* pAggSup, SExprInfo* pExprInfo, int32_t numOfCols, size_t keyBufSize, + const char* pkey); void initResultSizeInfo(SOperatorInfo* pOperator, int32_t numOfRows); void doBuildResultDatablock(SOperatorInfo* pOperator, SOptrBasicInfo* pbInfo, SGroupResInfo* pGroupResInfo, SDiskbasedBuf* pBuf); void doApplyFunctions(SExecTaskInfo* taskInfo, SqlFunctionCtx* pCtx, STimeWindow* pWin, SColumnInfoData* pTimeWindowData, int32_t offset, int32_t forwardStep, TSKEY* tsCol, int32_t numOfTotal, int32_t numOfOutput, int32_t order); -int32_t setGroupResultOutputBuf(SOptrBasicInfo* binfo, int32_t numOfCols, char* pData, int16_t type, int16_t bytes, - int32_t groupId, SDiskbasedBuf* pBuf, SExecTaskInfo* pTaskInfo, SAggSupporter* pAggSup); -void doDestroyBasicInfo(SOptrBasicInfo* pInfo, int32_t numOfOutput); -int32_t setDataBlockFromFetchRsp(SSDataBlock* pRes, SLoadRemoteDataInfo* pLoadInfo, int32_t numOfRows, char* pData, + +int32_t extractDataBlockFromFetchRsp(SSDataBlock* pRes, SLoadRemoteDataInfo* pLoadInfo, int32_t numOfRows, char* pData, int32_t compLen, int32_t numOfOutput, int64_t startTs, uint64_t* total, SArray* pColList); void getAlignQueryTimeWindow(SInterval* pInterval, int32_t precision, int64_t key, STimeWindow* win); int32_t getTableScanInfo(SOperatorInfo* pOperator, int32_t *order, int32_t* scanFlag); int32_t getBufferPgSize(int32_t rowSize, uint32_t* defaultPgsz, uint32_t* defaultBufsz); -SArray* extractPartitionColInfo(SNodeList* pNodeList); - void doSetOperatorCompleted(SOperatorInfo* pOperator); void doFilter(const SNode* pFilterNode, SSDataBlock* pBlock); -SqlFunctionCtx* createSqlFunctionCtx(SExprInfo* pExprInfo, int32_t numOfOutput, int32_t** rowCellInfoOffset); -void relocateColumnData(SSDataBlock* pBlock, const SArray* pColMatchInfo, SArray* pCols); -void initExecTimeWindowInfo(SColumnInfoData* pColData, STimeWindow* pQueryWindow); + void cleanupAggSup(SAggSupporter* pAggSup); void destroyBasicOperatorInfo(void* param, int32_t numOfOutput); void appendOneRowToDataBlock(SSDataBlock* pBlock, STupleHandle* pTupleHandle); void setTbNameColData(void* pMeta, const SSDataBlock* pBlock, SColumnInfoData* pColInfoData, int32_t functionId); -SInterval extractIntervalInfo(const STableScanPhysiNode* pTableScanNode); -SColumn extractColumnFromColumnNode(SColumnNode* pColNode); - -SSDataBlock* getSortedBlockData(SSortHandle* pHandle, SSDataBlock* pDataBlock, int32_t capacity, SArray* pColMatchInfo, SSortOperatorInfo* pInfo); -SSDataBlock* loadNextDataBlock(void* param); -void setResultRowInitCtx(SResultRow* pResult, SqlFunctionCtx* pCtx, int32_t numOfOutput, int32_t* rowCellInfoOffset); +void cleanupExecSupp(SExprSupp* pSupp); -SArray* extractColMatchInfo(SNodeList* pNodeList, SDataBlockDescNode* pOutputNodeList, int32_t* numOfOutputCols, - SExecTaskInfo* pTaskInfo, int32_t type); +SSDataBlock* loadNextDataBlock(void* param); -SExprInfo* createExprInfo(SNodeList* pNodeList, SNodeList* pGroupKeys, int32_t* numOfExprs); -SSDataBlock* createResDataBlock(SDataBlockDescNode* pNode); -int32_t initQueryTableDataCond(SQueryTableDataCond* pCond, const STableScanPhysiNode* pTableScanNode); -void clearupQueryTableDataCond(SQueryTableDataCond* pCond); +void setResultRowInitCtx(SResultRow* pResult, SqlFunctionCtx* pCtx, int32_t numOfOutput, int32_t* rowEntryInfoOffset); SResultRow* doSetResultOutBufByKey(SDiskbasedBuf* pResultBuf, SResultRowInfo* pResultRowInfo, char* pData, int16_t bytes, bool masterscan, uint64_t groupId, @@ -799,9 +708,9 @@ SOperatorInfo* createAggregateOperatorInfo(SOperatorInfo* downstream, SExprInfo* int32_t numOfScalarExpr, SExecTaskInfo* pTaskInfo); SOperatorInfo* createIndefinitOutputOperatorInfo(SOperatorInfo* downstream, SPhysiNode *pNode, SExecTaskInfo* pTaskInfo); -SOperatorInfo* createProjectOperatorInfo(SOperatorInfo* downstream, SExprInfo* pExprInfo, int32_t num, SSDataBlock* pResBlock, SLimit* pLimit, SLimit* pSlimit, SNode* pCondition, SExecTaskInfo* pTaskInfo); -SOperatorInfo *createSortOperatorInfo(SOperatorInfo* downstream, SSDataBlock* pResBlock, SArray* pSortInfo, SExprInfo* pExprInfo, int32_t numOfCols, - SArray* pIndexMap, SExecTaskInfo* pTaskInfo); +SOperatorInfo* createProjectOperatorInfo(SOperatorInfo* downstream, SProjectPhysiNode* pProjPhyNode, SExecTaskInfo* pTaskInfo); +SOperatorInfo* createSortOperatorInfo(SOperatorInfo* downstream, SSortPhysiNode* pSortPhyNode, SExecTaskInfo* pTaskInfo); + SOperatorInfo* createMultiwaySortMergeOperatorInfo(SOperatorInfo** downStreams, int32_t numStreams, SSDataBlock* pInputBlock, SSDataBlock* pResBlock, SArray* pSortInfo, SArray* pColMatchColInfo, SExecTaskInfo* pTaskInfo); @@ -831,10 +740,9 @@ SOperatorInfo* createDataBlockInfoScanOperator(void* dataReader, SExecTaskInfo* SOperatorInfo* createStreamScanOperatorInfo(void* pDataReader, SReadHandle* pHandle, STableScanPhysiNode* pTableScanNode, SExecTaskInfo* pTaskInfo, STimeWindowAggSupp* pTwSup); +SOperatorInfo* createFillOperatorInfo(SOperatorInfo* downstream, SFillPhysiNode* pPhyFillNode, bool multigroupResult, + SExecTaskInfo* pTaskInfo); -SOperatorInfo* createFillOperatorInfo(SOperatorInfo* downstream, SExprInfo* pExpr, int32_t numOfCols, - SInterval* pInterval, STimeWindow* pWindow, SSDataBlock* pResBlock, int32_t fillType, SNodeListNode* fillVal, - bool multigroupResult, SExecTaskInfo* pTaskInfo); SOperatorInfo* createStatewindowOperatorInfo(SOperatorInfo* downstream, SExprInfo* pExpr, int32_t numOfCols, SSDataBlock* pResBlock, STimeWindowAggSupp *pTwAggSupp, int32_t tsSlotId, SColumn* pStateKeyCol, SExecTaskInfo* pTaskInfo); @@ -843,10 +751,13 @@ SOperatorInfo* createPartitionOperatorInfo(SOperatorInfo* downstream, SPartition SOperatorInfo* createTimeSliceOperatorInfo(SOperatorInfo* downstream, SExprInfo* pExprInfo, int32_t numOfCols, SSDataBlock* pResultBlock, const SNodeListNode* pValNode, SExecTaskInfo* pTaskInfo); -SOperatorInfo* createMergeJoinOperatorInfo(SOperatorInfo** pDownstream, int32_t numOfDownstream, SExprInfo* pExprInfo, int32_t numOfCols, SSDataBlock* pResBlock, SNode* pOnCondition, SExecTaskInfo* pTaskInfo); +SOperatorInfo* createMergeJoinOperatorInfo(SOperatorInfo** pDownstream, int32_t numOfDownstream, SJoinPhysiNode* pJoinNode, + SExecTaskInfo* pTaskInfo); -SOperatorInfo* createStreamSessionAggOperatorInfo(SOperatorInfo* downstream, SExprInfo* pExprInfo, int32_t numOfCols, - SSDataBlock* pResBlock, int64_t gap, int32_t tsSlotId, STimeWindowAggSupp* pTwAggSupp, SExecTaskInfo* pTaskInfo); +SOperatorInfo* createStreamSessionAggOperatorInfo(SOperatorInfo* downstream, + SPhysiNode* pPhyNode, SExecTaskInfo* pTaskInfo); +SOperatorInfo* createStreamFinalSessionAggOperatorInfo(SOperatorInfo* downstream, + SPhysiNode* pPhyNode, SExecTaskInfo* pTaskInfo, int32_t numOfChild); SOperatorInfo* createStreamStateAggOperatorInfo(SOperatorInfo* downstream, SPhysiNode* pPhyNode, SExecTaskInfo* pTaskInfo); @@ -862,8 +773,8 @@ void setInputDataBlock(SOperatorInfo* pOperator, SqlFunctionCtx* pCtx, SSDataBlo bool isTaskKilled(SExecTaskInfo* pTaskInfo); int32_t checkForQueryBuf(size_t numOfTables); -void setTaskKilled(SExecTaskInfo* pTaskInfo); -void queryCostStatis(SExecTaskInfo* pTaskInfo); +void setTaskKilled(SExecTaskInfo* pTaskInfo); +void queryCostStatis(SExecTaskInfo* pTaskInfo); void doDestroyTask(SExecTaskInfo* pTaskInfo); int32_t getMaximumIdleDurationSec(); @@ -882,7 +793,7 @@ int32_t encodeOperator(SOperatorInfo* ops, char** data, int32_t *length); * length: the length of data * return: result code, 0 means success */ -int32_t decodeOperator(SOperatorInfo* ops, char* data, int32_t length); +int32_t decodeOperator(SOperatorInfo* ops, const char* data, int32_t length); void setTaskStatus(SExecTaskInfo* pTaskInfo, int8_t status); int32_t createExecTaskInfoImpl(SSubplan* pPlan, SExecTaskInfo** pTaskInfo, SReadHandle* pHandle, uint64_t taskId, @@ -899,12 +810,13 @@ STimeWindow getActiveTimeWindow(SDiskbasedBuf* pBuf, SResultRowInfo* pResultRowI int32_t getNumOfRowsInTimeWindow(SDataBlockInfo* pDataBlockInfo, TSKEY* pPrimaryColumn, int32_t startPos, TSKEY ekey, __block_search_fn_t searchFn, STableQueryInfo* item, int32_t order); int32_t binarySearchForKey(char* pValue, int num, TSKEY key, int order); -int32_t initStreamAggSupporter(SStreamAggSupporter* pSup, const char* pKey, - SqlFunctionCtx* pCtx, int32_t numOfOutput, int32_t size); +int32_t initStreamAggSupporter(SStreamAggSupporter* pSup, const char* pKey, SqlFunctionCtx* pCtx, int32_t numOfOutput, + int32_t size); SResultRow* getNewResultRow(SDiskbasedBuf* pResultBuf, int64_t tableGroupId, int32_t interBufSize); SResultWindowInfo* getSessionTimeWindow(SStreamAggSupporter* pAggSup, TSKEY ts, uint64_t groupId, int64_t gap, int32_t* pIndex); -int32_t updateSessionWindowInfo(SResultWindowInfo* pWinInfo, TSKEY* pTs, int32_t rows, - int32_t start, int64_t gap, SHashObj* pStDeleted); +int32_t updateSessionWindowInfo(SResultWindowInfo* pWinInfo, TSKEY* pTs, int32_t rows, int32_t start, int64_t gap, + SHashObj* pStDeleted); + bool functionNeedToExecute(SqlFunctionCtx* pCtx); int32_t compareTimeWindow(const void* p1, const void* p2, const void* param); @@ -912,8 +824,6 @@ int32_t finalizeResultRowIntoResultDataBlock(SDiskbasedBuf* pBuf, SResultRowPosi SqlFunctionCtx* pCtx, SExprInfo* pExprInfo, int32_t numOfExprs, const int32_t* rowCellOffset, SSDataBlock* pBlock, SExecTaskInfo* pTaskInfo); -int32_t getTableList(void* metaHandle, int32_t tableType, uint64_t tableUid, STableListInfo* pListInfo, - SNode* pTagCond); int32_t createMultipleDataReaders(STableScanPhysiNode* pTableScanNode, SReadHandle* pHandle, STableListInfo* pTableListInfo, SArray* arrayReader, uint64_t queryId, uint64_t taskId, SNode* pTagCond); diff --git a/source/libs/executor/inc/tsort.h b/source/libs/executor/inc/tsort.h index 86ee841cc24d91ff124293e64a14a78196e53840..363f379ee471e9d9d604784f1fcd24880b4ae3e7 100644 --- a/source/libs/executor/inc/tsort.h +++ b/source/libs/executor/inc/tsort.h @@ -63,7 +63,7 @@ typedef int32_t (*_sort_merge_compar_fn_t)(const void* p1, const void* p2, void* * @param type * @return */ -SSortHandle* tsortCreateSortHandle(SArray* pOrderInfo, SArray* pIndexMap, int32_t type, int32_t pageSize, int32_t numOfPages, SSDataBlock* pBlock, const char* idstr); +SSortHandle* tsortCreateSortHandle(SArray* pOrderInfo, int32_t type, int32_t pageSize, int32_t numOfPages, SSDataBlock* pBlock, const char* idstr); /** * diff --git a/source/libs/executor/src/executil.c b/source/libs/executor/src/executil.c index 01ed30c1896c11c7f9d5291d289d97c0af3ffaff..99139be409ace233b565013a939e0c3cf066509c 100644 --- a/source/libs/executor/src/executil.c +++ b/source/libs/executor/src/executil.c @@ -14,24 +14,20 @@ */ #include "os.h" -#include "tmsg.h" +#include "index.h" +#include "function.h" +#include "functionMgt.h" +#include "tdatablock.h" #include "thash.h" +#include "tmsg.h" #include "executil.h" #include "executorimpl.h" #include "tcompression.h" -#include "tlosertree.h" -typedef struct SCompSupporter { - STableQueryInfo **pTableQueryInfo; - int32_t *rowIndex; - int32_t order; -} SCompSupporter; - -int32_t initResultRowInfo(SResultRowInfo *pResultRowInfo, int32_t size) { +void initResultRowInfo(SResultRowInfo *pResultRowInfo) { pResultRowInfo->size = 0; pResultRowInfo->cur.pageId = -1; - return TSDB_CODE_SUCCESS; } void cleanupResultRowInfo(SResultRowInfo *pResultRowInfo) { @@ -46,21 +42,6 @@ void cleanupResultRowInfo(SResultRowInfo *pResultRowInfo) { } } -void resetResultRowInfo(STaskRuntimeEnv *pRuntimeEnv, SResultRowInfo *pResultRowInfo) { - for (int32_t i = 0; i < pResultRowInfo->size; ++i) { -// SResultRow *pWindowRes = pResultRowInfo->pResult[i]; -// clearResultRow(pRuntimeEnv, pWindowRes); - - int32_t groupIndex = 0; - int64_t uid = 0; - - SET_RES_WINDOW_KEY(pRuntimeEnv->keyBuf, &groupIndex, sizeof(groupIndex), uid); - taosHashRemove(pRuntimeEnv->pResultRowHashTable, (const char *)pRuntimeEnv->keyBuf, GET_RES_WINDOW_KEY_LEN(sizeof(groupIndex))); - } - - pResultRowInfo->size = 0; -} - void closeAllResultRows(SResultRowInfo *pResultRowInfo) { // do nothing } @@ -74,7 +55,7 @@ void closeResultRow(SResultRow* pResultRow) { } // TODO refactor: use macro -SResultRowEntryInfo* getResultCell(const SResultRow* pRow, int32_t index, const int32_t* offset) { +SResultRowEntryInfo* getResultEntryInfo(const SResultRow* pRow, int32_t index, const int32_t* offset) { assert(index >= 0 && offset != NULL); return (SResultRowEntryInfo*)((char*) pRow->pEntryInfo + offset[index]); } @@ -160,7 +141,7 @@ void initMultiResInfoFromArrayList(SGroupResInfo* pGroupResInfo, SArray* pArrayL ASSERT(pGroupResInfo->index <= getNumOfTotalRes(pGroupResInfo)); } -bool hashRemainDataInGroupInfo(SGroupResInfo* pGroupResInfo) { +bool hasDataInGroupInfo(SGroupResInfo* pGroupResInfo) { if (pGroupResInfo->pRows == NULL) { return false; } @@ -177,86 +158,532 @@ int32_t getNumOfTotalRes(SGroupResInfo* pGroupResInfo) { return (int32_t) taosArrayGetSize(pGroupResInfo->pRows); } -static int32_t tableResultComparFn(const void *pLeft, const void *pRight, void *param) { - int32_t left = *(int32_t *)pLeft; - int32_t right = *(int32_t *)pRight; +SArray* createSortInfo(SNodeList* pNodeList) { + size_t numOfCols = LIST_LENGTH(pNodeList); + SArray* pList = taosArrayInit(numOfCols, sizeof(SBlockOrderInfo)); + if (pList == NULL) { + terrno = TSDB_CODE_OUT_OF_MEMORY; + return pList; + } + + for (int32_t i = 0; i < numOfCols; ++i) { + SOrderByExprNode* pSortKey = (SOrderByExprNode*)nodesListGetNode(pNodeList, i); + SBlockOrderInfo bi = {0}; + bi.order = (pSortKey->order == ORDER_ASC) ? TSDB_ORDER_ASC : TSDB_ORDER_DESC; + bi.nullFirst = (pSortKey->nullOrder == NULL_ORDER_FIRST); - SCompSupporter * supporter = (SCompSupporter *)param; + SColumnNode* pColNode = (SColumnNode*)pSortKey->pExpr; + bi.slotId = pColNode->slotId; + taosArrayPush(pList, &bi); + } + + return pList; +} - int32_t leftPos = supporter->rowIndex[left]; - int32_t rightPos = supporter->rowIndex[right]; +SSDataBlock* createResDataBlock(SDataBlockDescNode* pNode) { + int32_t numOfCols = LIST_LENGTH(pNode->pSlots); - /* left source is exhausted */ - if (leftPos == -1) { - return 1; + SSDataBlock* pBlock = taosMemoryCalloc(1, sizeof(SSDataBlock)); + pBlock->pDataBlock = taosArrayInit(numOfCols, sizeof(SColumnInfoData)); + + pBlock->info.blockId = pNode->dataBlockId; + pBlock->info.rowSize = pNode->totalRowSize; // todo ?? + pBlock->info.type = STREAM_INVALID; + + for (int32_t i = 0; i < numOfCols; ++i) { + SColumnInfoData idata = {{0}}; + SSlotDescNode* pDescNode = (SSlotDescNode*)nodesListGetNode(pNode->pSlots, i); + // if (!pDescNode->output) { // todo disable it temporarily + // continue; + // } + + idata.info.type = pDescNode->dataType.type; + idata.info.bytes = pDescNode->dataType.bytes; + idata.info.scale = pDescNode->dataType.scale; + idata.info.slotId = pDescNode->slotId; + idata.info.precision = pDescNode->dataType.precision; + + if (IS_VAR_DATA_TYPE(idata.info.type)) { + pBlock->info.hasVarCol = true; + } + + taosArrayPush(pBlock->pDataBlock, &idata); } - /* right source is exhausted*/ - if (rightPos == -1) { - return -1; + pBlock->info.numOfCols = taosArrayGetSize(pBlock->pDataBlock); + return pBlock; +} + +int32_t getTableList(void* metaHandle, SScanPhysiNode* pScanNode, STableListInfo* pListInfo, SNode* pTagCond) { + int32_t code = TSDB_CODE_SUCCESS; + pListInfo->pTableList = taosArrayInit(8, sizeof(STableKeyInfo)); + + uint64_t tableUid = pScanNode->uid; + + if (pScanNode->tableType == TSDB_SUPER_TABLE) { + if (pTagCond) { + SIndexMetaArg metaArg = { + .metaEx = metaHandle, .idx = tsdbGetIdx(metaHandle), .ivtIdx = tsdbGetIvtIdx(metaHandle), .suid = tableUid}; + + SArray* res = taosArrayInit(8, sizeof(uint64_t)); + code = doFilterTag(pTagCond, &metaArg, res); + if (code == TSDB_CODE_INDEX_REBUILDING) { // todo + // doFilter(); + } else if (code != TSDB_CODE_SUCCESS) { + qError("failed to get tableIds, reason: %s, suid: %" PRIu64 "", tstrerror(code), tableUid); + taosArrayDestroy(res); + terrno = code; + return code; + } else { + qDebug("sucess to get tableIds, size: %d, suid: %" PRIu64 "", (int)taosArrayGetSize(res), tableUid); + } + + for (int i = 0; i < taosArrayGetSize(res); i++) { + STableKeyInfo info = {.lastKey = TSKEY_INITIAL_VAL, .uid = *(uint64_t*)taosArrayGet(res, i)}; + taosArrayPush(pListInfo->pTableList, &info); + } + taosArrayDestroy(res); + } else { + code = tsdbGetAllTableList(metaHandle, tableUid, pListInfo->pTableList); + } + } else { // Create one table group. + STableKeyInfo info = {.lastKey = 0, .uid = tableUid}; + taosArrayPush(pListInfo->pTableList, &info); } - ASSERT(0); - STableQueryInfo** pList = supporter->pTableQueryInfo; -// SResultRow* pWindowRes1 = pList[left]->resInfo.pResult[leftPos]; -// SResultRow * pWindowRes1 = getResultRow(&(pList[left]->resInfo), leftPos); -// TSKEY leftTimestamp = pWindowRes1->win.skey; + return code; +} -// SResultRowInfo *pWindowResInfo2 = &(pList[right]->resInfo); -// SResultRow * pWindowRes2 = getResultRow(pWindowResInfo2, rightPos); -// SResultRow* pWindowRes2 = pList[right]->resInfo.pResult[rightPos]; -// TSKEY rightTimestamp = pWindowRes2->win.skey; +SArray* extractPartitionColInfo(SNodeList* pNodeList) { + if(!pNodeList) { + return NULL; + } -// if (leftTimestamp == rightTimestamp) { - return 0; -// } + size_t numOfCols = LIST_LENGTH(pNodeList); + SArray* pList = taosArrayInit(numOfCols, sizeof(SColumn)); + if (pList == NULL) { + terrno = TSDB_CODE_OUT_OF_MEMORY; + return NULL; + } -// if (supporter->order == TSDB_ORDER_ASC) { -// return (leftTimestamp > rightTimestamp)? 1:-1; -// } else { -// return (leftTimestamp < rightTimestamp)? 1:-1; -// } + for (int32_t i = 0; i < numOfCols; ++i) { + SColumnNode* pColNode = (SColumnNode*)nodesListGetNode(pNodeList, i); + + // todo extract method + SColumn c = {0}; + c.slotId = pColNode->slotId; + c.colId = pColNode->colId; + c.type = pColNode->node.resType.type; + c.bytes = pColNode->node.resType.bytes; + c.precision = pColNode->node.resType.precision; + c.scale = pColNode->node.resType.scale; + + taosArrayPush(pList, &c); + } + + return pList; } -int32_t tsAscOrder(const void* p1, const void* p2) { - SResultRowCell* pc1 = (SResultRowCell*) p1; - SResultRowCell* pc2 = (SResultRowCell*) p2; - if (pc1->groupId == pc2->groupId) { - ASSERT(0); -// if (pc1->pRow->win.skey == pc2->pRow->win.skey) { -// return 0; -// } else { -// return (pc1->pRow->win.skey < pc2->pRow->win.skey)? -1:1; -// } - } else { - return (pc1->groupId < pc2->groupId)? -1:1; +SArray* extractColMatchInfo(SNodeList* pNodeList, SDataBlockDescNode* pOutputNodeList, int32_t* numOfOutputCols, + int32_t type) { + size_t numOfCols = LIST_LENGTH(pNodeList); + SArray* pList = taosArrayInit(numOfCols, sizeof(SColMatchInfo)); + if (pList == NULL) { + terrno = TSDB_CODE_OUT_OF_MEMORY; + return NULL; + } + + for (int32_t i = 0; i < numOfCols; ++i) { + STargetNode* pNode = (STargetNode*)nodesListGetNode(pNodeList, i); + SColumnNode* pColNode = (SColumnNode*)pNode->pExpr; + + SColMatchInfo c = {0}; + c.output = true; + c.colId = pColNode->colId; + c.srcSlotId = pColNode->slotId; + c.matchType = type; + c.targetSlotId = pNode->slotId; + taosArrayPush(pList, &c); + } + + *numOfOutputCols = 0; + int32_t num = LIST_LENGTH(pOutputNodeList->pSlots); + for (int32_t i = 0; i < num; ++i) { + SSlotDescNode* pNode = (SSlotDescNode*)nodesListGetNode(pOutputNodeList->pSlots, i); + + // todo: add reserve flag check + // it is a column reserved for the arithmetic expression calculation + if (pNode->slotId >= numOfCols) { + (*numOfOutputCols) += 1; + continue; + } + + SColMatchInfo* info = taosArrayGet(pList, pNode->slotId); + if (pNode->output) { + (*numOfOutputCols) += 1; + } else { + info->output = false; + } } + + return pList; } -int32_t tsDescOrder(const void* p1, const void* p2) { - SResultRowCell* pc1 = (SResultRowCell*) p1; - SResultRowCell* pc2 = (SResultRowCell*) p2; +static SResSchema createResSchema(int32_t type, int32_t bytes, int32_t slotId, int32_t scale, int32_t precision, + const char* name) { + SResSchema s = {0}; + s.scale = scale; + s.type = type; + s.bytes = bytes; + s.slotId = slotId; + s.precision = precision; + strncpy(s.name, name, tListLen(s.name)); + + return s; +} - if (pc1->groupId == pc2->groupId) { - ASSERT(0); -// if (pc1->pRow->win.skey == pc2->pRow->win.skey) { -// return 0; -// } else { -// return (pc1->pRow->win.skey < pc2->pRow->win.skey)? 1:-1; -// } +static SColumn* createColumn(int32_t blockId, int32_t slotId, int32_t colId, SDataType* pType) { + SColumn* pCol = taosMemoryCalloc(1, sizeof(SColumn)); + if (pCol == NULL) { + terrno = TSDB_CODE_OUT_OF_MEMORY; + return NULL; + } + + pCol->slotId = slotId; + pCol->colId = colId; + pCol->bytes = pType->bytes; + pCol->type = pType->type; + pCol->scale = pType->scale; + pCol->precision = pType->precision; + pCol->dataBlockId = blockId; + + return pCol; +} + +SExprInfo* createExprInfo(SNodeList* pNodeList, SNodeList* pGroupKeys, int32_t* numOfExprs) { + int32_t numOfFuncs = LIST_LENGTH(pNodeList); + int32_t numOfGroupKeys = 0; + if (pGroupKeys != NULL) { + numOfGroupKeys = LIST_LENGTH(pGroupKeys); + } + + *numOfExprs = numOfFuncs + numOfGroupKeys; + SExprInfo* pExprs = taosMemoryCalloc(*numOfExprs, sizeof(SExprInfo)); + + for (int32_t i = 0; i < (*numOfExprs); ++i) { + STargetNode* pTargetNode = NULL; + if (i < numOfFuncs) { + pTargetNode = (STargetNode*)nodesListGetNode(pNodeList, i); + } else { + pTargetNode = (STargetNode*)nodesListGetNode(pGroupKeys, i - numOfFuncs); + } + + SExprInfo* pExp = &pExprs[i]; + + pExp->pExpr = taosMemoryCalloc(1, sizeof(tExprNode)); + pExp->pExpr->_function.num = 1; + pExp->pExpr->_function.functionId = -1; + + int32_t type = nodeType(pTargetNode->pExpr); + // it is a project query, or group by column + if (type == QUERY_NODE_COLUMN) { + pExp->pExpr->nodeType = QUERY_NODE_COLUMN; + SColumnNode* pColNode = (SColumnNode*)pTargetNode->pExpr; + + pExp->base.pParam = taosMemoryCalloc(1, sizeof(SFunctParam)); + pExp->base.numOfParams = 1; + + SDataType* pType = &pColNode->node.resType; + pExp->base.resSchema = createResSchema(pType->type, pType->bytes, pTargetNode->slotId, pType->scale, + pType->precision, pColNode->colName); + pExp->base.pParam[0].pCol = createColumn(pColNode->dataBlockId, pColNode->slotId, pColNode->colId, pType); + pExp->base.pParam[0].type = FUNC_PARAM_TYPE_COLUMN; + } else if (type == QUERY_NODE_VALUE) { + pExp->pExpr->nodeType = QUERY_NODE_VALUE; + SValueNode* pValNode = (SValueNode*)pTargetNode->pExpr; + + pExp->base.pParam = taosMemoryCalloc(1, sizeof(SFunctParam)); + pExp->base.numOfParams = 1; + + SDataType* pType = &pValNode->node.resType; + pExp->base.resSchema = createResSchema(pType->type, pType->bytes, pTargetNode->slotId, pType->scale, + pType->precision, pValNode->node.aliasName); + pExp->base.pParam[0].type = FUNC_PARAM_TYPE_VALUE; + nodesValueNodeToVariant(pValNode, &pExp->base.pParam[0].param); + } else if (type == QUERY_NODE_FUNCTION) { + pExp->pExpr->nodeType = QUERY_NODE_FUNCTION; + SFunctionNode* pFuncNode = (SFunctionNode*)pTargetNode->pExpr; + + SDataType* pType = &pFuncNode->node.resType; + pExp->base.resSchema = createResSchema(pType->type, pType->bytes, pTargetNode->slotId, pType->scale, + pType->precision, pFuncNode->node.aliasName); + + pExp->pExpr->_function.functionId = pFuncNode->funcId; + pExp->pExpr->_function.pFunctNode = pFuncNode; + + strncpy(pExp->pExpr->_function.functionName, pFuncNode->functionName, + tListLen(pExp->pExpr->_function.functionName)); +#if 1 + // todo refactor: add the parameter for tbname function + if (strcmp(pExp->pExpr->_function.functionName, "tbname") == 0) { + pFuncNode->pParameterList = nodesMakeList(); + ASSERT(LIST_LENGTH(pFuncNode->pParameterList) == 0); + SValueNode* res = (SValueNode*)nodesMakeNode(QUERY_NODE_VALUE); + if (NULL == res) { // todo handle error + } else { + res->node.resType = (SDataType){.bytes = sizeof(int64_t), .type = TSDB_DATA_TYPE_BIGINT}; + nodesListAppend(pFuncNode->pParameterList, (SNode*)res); + } + } +#endif + + int32_t numOfParam = LIST_LENGTH(pFuncNode->pParameterList); + + pExp->base.pParam = taosMemoryCalloc(numOfParam, sizeof(SFunctParam)); + pExp->base.numOfParams = numOfParam; + + for (int32_t j = 0; j < numOfParam; ++j) { + SNode* p1 = nodesListGetNode(pFuncNode->pParameterList, j); + if (p1->type == QUERY_NODE_COLUMN) { + SColumnNode* pcn = (SColumnNode*)p1; + + pExp->base.pParam[j].type = FUNC_PARAM_TYPE_COLUMN; + pExp->base.pParam[j].pCol = createColumn(pcn->dataBlockId, pcn->slotId, pcn->colId, &pcn->node.resType); + } else if (p1->type == QUERY_NODE_VALUE) { + SValueNode* pvn = (SValueNode*)p1; + pExp->base.pParam[j].type = FUNC_PARAM_TYPE_VALUE; + nodesValueNodeToVariant(pvn, &pExp->base.pParam[j].param); + } + } + } else if (type == QUERY_NODE_OPERATOR) { + pExp->pExpr->nodeType = QUERY_NODE_OPERATOR; + SOperatorNode* pNode = (SOperatorNode*)pTargetNode->pExpr; + + pExp->base.pParam = taosMemoryCalloc(1, sizeof(SFunctParam)); + pExp->base.numOfParams = 1; + + SDataType* pType = &pNode->node.resType; + pExp->base.resSchema = createResSchema(pType->type, pType->bytes, pTargetNode->slotId, pType->scale, + pType->precision, pNode->node.aliasName); + pExp->pExpr->_optrRoot.pRootNode = pTargetNode->pExpr; + } else { + ASSERT(0); + } + } + + return pExprs; +} + +// set the output buffer for the selectivity + tag query +static int32_t setSelectValueColumnInfo(SqlFunctionCtx* pCtx, int32_t numOfOutput) { + int32_t num = 0; + + SqlFunctionCtx* p = NULL; + SqlFunctionCtx** pValCtx = taosMemoryCalloc(numOfOutput, POINTER_BYTES); + if (pValCtx == NULL) { + return TSDB_CODE_QRY_OUT_OF_MEMORY; + } + + for (int32_t i = 0; i < numOfOutput; ++i) { + if (strcmp(pCtx[i].pExpr->pExpr->_function.functionName, "_select_value") == 0) { + pValCtx[num++] = &pCtx[i]; + } else if (fmIsSelectFunc(pCtx[i].functionId)) { + p = &pCtx[i]; + } + } + + if (p != NULL) { + p->subsidiaries.pCtx = pValCtx; + p->subsidiaries.num = num; } else { - return (pc1->groupId < pc2->groupId)? -1:1; + taosMemoryFreeClear(pValCtx); + } + + return TSDB_CODE_SUCCESS; +} + +SqlFunctionCtx* createSqlFunctionCtx(SExprInfo* pExprInfo, int32_t numOfOutput, int32_t** rowEntryInfoOffset) { + SqlFunctionCtx* pFuncCtx = (SqlFunctionCtx*)taosMemoryCalloc(numOfOutput, sizeof(SqlFunctionCtx)); + if (pFuncCtx == NULL) { + return NULL; + } + + *rowEntryInfoOffset = taosMemoryCalloc(numOfOutput, sizeof(int32_t)); + if (*rowEntryInfoOffset == 0) { + taosMemoryFreeClear(pFuncCtx); + return NULL; + } + + for (int32_t i = 0; i < numOfOutput; ++i) { + SExprInfo* pExpr = &pExprInfo[i]; + + SExprBasicInfo* pFunct = &pExpr->base; + SqlFunctionCtx* pCtx = &pFuncCtx[i]; + + pCtx->functionId = -1; + pCtx->curBufPage = -1; + pCtx->pExpr = pExpr; + + if (pExpr->pExpr->nodeType == QUERY_NODE_FUNCTION) { + SFuncExecEnv env = {0}; + pCtx->functionId = pExpr->pExpr->_function.pFunctNode->funcId; + + if (fmIsAggFunc(pCtx->functionId) || fmIsIndefiniteRowsFunc(pCtx->functionId)) { + bool isUdaf = fmIsUserDefinedFunc(pCtx->functionId); + if (!isUdaf) { + fmGetFuncExecFuncs(pCtx->functionId, &pCtx->fpSet); + } else { + char* udfName = pExpr->pExpr->_function.pFunctNode->functionName; + strncpy(pCtx->udfName, udfName, strlen(udfName)); + fmGetUdafExecFuncs(pCtx->functionId, &pCtx->fpSet); + } + pCtx->fpSet.getEnv(pExpr->pExpr->_function.pFunctNode, &env); + } else { + fmGetScalarFuncExecFuncs(pCtx->functionId, &pCtx->sfp); + if (pCtx->sfp.getEnv != NULL) { + pCtx->sfp.getEnv(pExpr->pExpr->_function.pFunctNode, &env); + } + } + pCtx->resDataInfo.interBufSize = env.calcMemSize; + } else if (pExpr->pExpr->nodeType == QUERY_NODE_COLUMN || pExpr->pExpr->nodeType == QUERY_NODE_OPERATOR || + pExpr->pExpr->nodeType == QUERY_NODE_VALUE) { + // for simple column, the result buffer needs to hold at least one element. + pCtx->resDataInfo.interBufSize = pFunct->resSchema.bytes; + } + + pCtx->input.numOfInputCols = pFunct->numOfParams; + pCtx->input.pData = taosMemoryCalloc(pFunct->numOfParams, POINTER_BYTES); + pCtx->input.pColumnDataAgg = taosMemoryCalloc(pFunct->numOfParams, POINTER_BYTES); + + pCtx->pTsOutput = NULL; + pCtx->resDataInfo.bytes = pFunct->resSchema.bytes; + pCtx->resDataInfo.type = pFunct->resSchema.type; + pCtx->order = TSDB_ORDER_ASC; + pCtx->start.key = INT64_MIN; + pCtx->end.key = INT64_MIN; + pCtx->numOfParams = pExpr->base.numOfParams; + pCtx->increase = false; + + pCtx->param = pFunct->pParam; + } + + for (int32_t i = 1; i < numOfOutput; ++i) { + (*rowEntryInfoOffset)[i] = + (int32_t)((*rowEntryInfoOffset)[i - 1] + sizeof(SResultRowEntryInfo) + pFuncCtx[i - 1].resDataInfo.interBufSize); } + + setSelectValueColumnInfo(pFuncCtx, numOfOutput); + return pFuncCtx; +} + +// NOTE: sources columns are more than the destination SSDatablock columns. +void relocateColumnData(SSDataBlock* pBlock, const SArray* pColMatchInfo, SArray* pCols) { + size_t numOfSrcCols = taosArrayGetSize(pCols); + + int32_t i = 0, j = 0; + while (i < numOfSrcCols && j < taosArrayGetSize(pColMatchInfo)) { + SColumnInfoData* p = taosArrayGet(pCols, i); + SColMatchInfo* pmInfo = taosArrayGet(pColMatchInfo, j); + if (!pmInfo->output) { + j++; + continue; + } + + if (p->info.colId == pmInfo->colId) { + SColumnInfoData* pDst = taosArrayGet(pBlock->pDataBlock, pmInfo->targetSlotId); + colDataAssign(pDst, p, pBlock->info.rows); + i++; + j++; + } else if (p->info.colId < pmInfo->colId) { + i++; + } else { + ASSERT(0); + } + } +} + +SInterval extractIntervalInfo(const STableScanPhysiNode* pTableScanNode) { + SInterval interval = { + .interval = pTableScanNode->interval, + .sliding = pTableScanNode->sliding, + .intervalUnit = pTableScanNode->intervalUnit, + .slidingUnit = pTableScanNode->slidingUnit, + .offset = pTableScanNode->offset, + }; + + return interval; } -void orderTheResultRows(STaskRuntimeEnv* pRuntimeEnv) { - __compar_fn_t fn = NULL; -// if (pRuntimeEnv->pQueryAttr->order.order == TSDB_ORDER_ASC) { -// fn = tsAscOrder; -// } else { -// fn = tsDescOrder; -// } +SColumn extractColumnFromColumnNode(SColumnNode* pColNode) { + SColumn c = {0}; + c.slotId = pColNode->slotId; + c.colId = pColNode->colId; + c.type = pColNode->node.resType.type; + c.bytes = pColNode->node.resType.bytes; + c.scale = pColNode->node.resType.scale; + c.precision = pColNode->node.resType.precision; + return c; +} + +int32_t initQueryTableDataCond(SQueryTableDataCond* pCond, const STableScanPhysiNode* pTableScanNode) { + pCond->loadExternalRows = false; + + pCond->order = pTableScanNode->scanSeq[0] > 0 ? TSDB_ORDER_ASC : TSDB_ORDER_DESC; + pCond->numOfCols = LIST_LENGTH(pTableScanNode->scan.pScanCols); + pCond->colList = taosMemoryCalloc(pCond->numOfCols, sizeof(SColumnInfo)); + if (pCond->colList == NULL) { + terrno = TSDB_CODE_QRY_OUT_OF_MEMORY; + return terrno; + } + + // pCond->twindow = pTableScanNode->scanRange; + // TODO: get it from stable scan node + pCond->numOfTWindows = 1; + pCond->twindows = taosMemoryCalloc(pCond->numOfTWindows, sizeof(STimeWindow)); + pCond->twindows[0] = pTableScanNode->scanRange; + pCond->suid = pTableScanNode->scan.suid; + +#if 1 + // todo work around a problem, remove it later + for (int32_t i = 0; i < pCond->numOfTWindows; ++i) { + if ((pCond->order == TSDB_ORDER_ASC && pCond->twindows[i].skey > pCond->twindows[i].ekey) || + (pCond->order == TSDB_ORDER_DESC && pCond->twindows[i].skey < pCond->twindows[i].ekey)) { + TSWAP(pCond->twindows[i].skey, pCond->twindows[i].ekey); + } + } +#endif + + for (int32_t i = 0; i < pCond->numOfTWindows; ++i) { + if ((pCond->order == TSDB_ORDER_ASC && pCond->twindows[i].skey > pCond->twindows[i].ekey) || + (pCond->order == TSDB_ORDER_DESC && pCond->twindows[i].skey < pCond->twindows[i].ekey)) { + TSWAP(pCond->twindows[i].skey, pCond->twindows[i].ekey); + } + } + taosqsort(pCond->twindows, pCond->numOfTWindows, sizeof(STimeWindow), pCond, compareTimeWindow); - taosArraySort(pRuntimeEnv->pResultRowArrayList, fn); + pCond->type = BLOCK_LOAD_OFFSET_SEQ_ORDER; + // pCond->type = pTableScanNode->scanFlag; + + int32_t j = 0; + for (int32_t i = 0; i < pCond->numOfCols; ++i) { + STargetNode* pNode = (STargetNode*)nodesListGetNode(pTableScanNode->scan.pScanCols, i); + SColumnNode* pColNode = (SColumnNode*)pNode->pExpr; + if (pColNode->colType == COLUMN_TYPE_TAG) { + continue; + } + + pCond->colList[j].type = pColNode->node.resType.type; + pCond->colList[j].bytes = pColNode->node.resType.bytes; + pCond->colList[j].colId = pColNode->colId; + j += 1; + } + + pCond->numOfCols = j; + return TSDB_CODE_SUCCESS; } + +void cleanupQueryTableDataCond(SQueryTableDataCond* pCond) { + taosMemoryFree(pCond->twindows); + taosMemoryFree(pCond->colList); +} \ No newline at end of file diff --git a/source/libs/executor/src/executorMain.c b/source/libs/executor/src/executorMain.c index 00158d702439518f60841ec6e3c0a8fd38315b70..ff281dacbdbae5d1cdf600fc6fb44ad2a4d2bdec 100644 --- a/source/libs/executor/src/executorMain.c +++ b/source/libs/executor/src/executorMain.c @@ -219,4 +219,23 @@ int32_t qGetExplainExecInfo(qTaskInfo_t tinfo, int32_t *resNum, SExplainExecInfo return getOperatorExplainExecInfo(pTaskInfo->pRoot, pRes, &capacity, resNum); } +int32_t qSerializeTaskStatus(qTaskInfo_t tinfo, char** pOutput, int32_t* len) { + SExecTaskInfo* pTaskInfo = (struct SExecTaskInfo*)tinfo; + if (pTaskInfo->pRoot == NULL) { + return TSDB_CODE_INVALID_PARA; + } + + return encodeOperator(pTaskInfo->pRoot, pOutput, len); +} + +int32_t qDeserializeTaskStatus(qTaskInfo_t tinfo, const char* pInput, int32_t len) { + SExecTaskInfo* pTaskInfo = (struct SExecTaskInfo*) tinfo; + + if (pTaskInfo == NULL || pInput == NULL || len == 0) { + return TSDB_CODE_INVALID_PARA; + } + + return decodeOperator(pTaskInfo->pRoot, pInput, len); +} + diff --git a/source/libs/executor/src/executorimpl.c b/source/libs/executor/src/executorimpl.c index 40b019eb5d7ea8ed38be08d2da372d97700d00b1..421fd56e1f60313f3da20a0de348692fbdb11e9a 100644 --- a/source/libs/executor/src/executorimpl.c +++ b/source/libs/executor/src/executorimpl.c @@ -90,8 +90,6 @@ static void doSetTagValueToResultBuf(char* output, const char* val, int16_t type static void setBlockStatisInfo(SqlFunctionCtx* pCtx, SExprInfo* pExpr, SSDataBlock* pSDataBlock); -static void destroyTableQueryInfoImpl(STableQueryInfo* pTableQueryInfo); - static void releaseQueryBuf(size_t numOfTables); static void destroySFillOperatorInfo(void* param, int32_t numOfOutput); @@ -143,71 +141,10 @@ static int32_t doCopyToSDataBlock(SExecTaskInfo* taskInfo, SSDataBlock* pBlock, SqlFunctionCtx* pCtx, int32_t numOfExprs); static void initCtxOutputBuffer(SqlFunctionCtx* pCtx, int32_t size); -static void setResultBufSize(STaskAttr* pQueryAttr, SResultInfo* pResultInfo); -static void doSetTableGroupOutputBuf(SAggOperatorInfo* pAggInfo, int32_t numOfOutput, uint64_t groupId, - SExecTaskInfo* pTaskInfo); - -SArray* getOrderCheckColumns(STaskAttr* pQuery); - -typedef struct SRowCompSupporter { - STaskRuntimeEnv* pRuntimeEnv; - int16_t dataOffset; - __compar_fn_t comFunc; -} SRowCompSupporter; - -static int compareRowData(const void* a, const void* b, const void* userData) { - const SResultRow* pRow1 = (const SResultRow*)a; - const SResultRow* pRow2 = (const SResultRow*)b; - - SRowCompSupporter* supporter = (SRowCompSupporter*)userData; - STaskRuntimeEnv* pRuntimeEnv = supporter->pRuntimeEnv; - - SFilePage* page1 = getBufPage(pRuntimeEnv->pResultBuf, pRow1->pageId); - SFilePage* page2 = getBufPage(pRuntimeEnv->pResultBuf, pRow2->pageId); - - int16_t offset = supporter->dataOffset; - return 0; - // char* in1 = getPosInResultPage(pRuntimeEnv->pQueryAttr, page1, pRow1->offset, offset); - // char* in2 = getPosInResultPage(pRuntimeEnv->pQueryAttr, page2, pRow2->offset, offset); +static void doSetTableGroupOutputBuf(SOperatorInfo* pOperator, SAggOperatorInfo* pAggInfo, int32_t numOfOutput, uint64_t groupId); - // return (in1 != NULL && in2 != NULL) ? supporter->comFunc(in1, in2) : 0; -} // setup the output buffer for each operator -SSDataBlock* createResDataBlock(SDataBlockDescNode* pNode) { - int32_t numOfCols = LIST_LENGTH(pNode->pSlots); - - SSDataBlock* pBlock = taosMemoryCalloc(1, sizeof(SSDataBlock)); - pBlock->pDataBlock = taosArrayInit(numOfCols, sizeof(SColumnInfoData)); - - pBlock->info.blockId = pNode->dataBlockId; - pBlock->info.rowSize = pNode->totalRowSize; // todo ?? - pBlock->info.type = STREAM_INVALID; - - for (int32_t i = 0; i < numOfCols; ++i) { - SColumnInfoData idata = {{0}}; - SSlotDescNode* pDescNode = (SSlotDescNode*)nodesListGetNode(pNode->pSlots, i); - // if (!pDescNode->output) { // todo disable it temporarily - // continue; - // } - - idata.info.type = pDescNode->dataType.type; - idata.info.bytes = pDescNode->dataType.bytes; - idata.info.scale = pDescNode->dataType.scale; - idata.info.slotId = pDescNode->slotId; - idata.info.precision = pDescNode->dataType.precision; - - if (IS_VAR_DATA_TYPE(idata.info.type)) { - pBlock->info.hasVarCol = true; - } - - taosArrayPush(pBlock->pDataBlock, &idata); - } - - pBlock->info.numOfCols = taosArrayGetSize(pBlock->pDataBlock); - return pBlock; -} - static bool hasNull(SColumn* pColumn, SColumnDataAgg* pStatis) { if (TSDB_COL_IS_TAG(pColumn->flag) || TSDB_COL_IS_UD_COL(pColumn->flag) || pColumn->colId == PRIMARYKEY_TIMESTAMP_COL_ID) { @@ -221,6 +158,7 @@ static bool hasNull(SColumn* pColumn, SColumnDataAgg* pStatis) { return true; } +#if 0 static bool chkResultRowFromKey(STaskRuntimeEnv* pRuntimeEnv, SResultRowInfo* pResultRowInfo, char* pData, int16_t bytes, bool masterscan, uint64_t uid) { bool existed = false; @@ -257,6 +195,7 @@ static bool chkResultRowFromKey(STaskRuntimeEnv* pRuntimeEnv, SResultRowInfo* pR return p1 != NULL; } +#endif SResultRow* getNewResultRow(SDiskbasedBuf* pResultBuf, int64_t tableGroupId, int32_t interBufSize) { SFilePage* pData = NULL; @@ -365,36 +304,6 @@ SResultRow* doSetResultOutBufByKey(SDiskbasedBuf* pResultBuf, SResultRowInfo* pR return pResult; } -// get the correct time window according to the handled timestamp -static STimeWindow getCurrentActiveTimeWindow(SResultRowInfo* pResultRowInfo, int64_t ts, STaskAttr* pQueryAttr) { - STimeWindow w = {0}; -#if 0 - if (pResultRowInfo->curPos == -1) { // the first window, from the previous stored value - // getInitialStartTimeWindow(pQueryAttr, ts, &w); - - if (pQueryAttr->interval.intervalUnit == 'n' || pQueryAttr->interval.intervalUnit == 'y') { - w.ekey = - taosTimeAdd(w.skey, pQueryAttr->interval.interval, pQueryAttr->interval.intervalUnit, pQueryAttr->precision) - - 1; - } else { - w.ekey = w.skey + pQueryAttr->interval.interval - 1; - } - } else { - w = pRow->win; - } - - /* - * query border check, skey should not be bounded by the query time range, since the value skey will - * be used as the time window index value. So we only change ekey of time window accordingly. - */ - if (w.ekey > pQueryAttr->window.ekey && QUERY_IS_ASC_QUERY(pQueryAttr)) { - w.ekey = pQueryAttr->window.ekey; - } -#endif - - return w; -} - // a new buffer page for each table. Needs to opt this design static int32_t addNewWindowResultBuf(SResultRow* pWindowRes, SDiskbasedBuf* pResultBuf, int32_t tid, uint32_t size) { if (pWindowRes->pageId != -1) { @@ -442,13 +351,6 @@ static int32_t addNewWindowResultBuf(SResultRow* pWindowRes, SDiskbasedBuf* pRes return 0; } -static bool chkWindowOutputBufByKey(STaskRuntimeEnv* pRuntimeEnv, SResultRowInfo* pResultRowInfo, STimeWindow* win, - bool masterscan, SResultRow** pResult, int64_t groupId, SqlFunctionCtx* pCtx, - int32_t numOfOutput, int32_t* rowCellInfoOffset) { - assert(win->skey <= win->ekey); - return chkResultRowFromKey(pRuntimeEnv, pResultRowInfo, (char*)&win->skey, TSDB_KEYSIZE, masterscan, groupId); -} - // query_range_start, query_range_end, window_duration, window_start, window_end void initExecTimeWindowInfo(SColumnInfoData* pColData, STimeWindow* pQueryWindow) { pColData->info.type = TSDB_DATA_TYPE_TIMESTAMP; @@ -517,33 +419,15 @@ void doApplyFunctions(SExecTaskInfo* taskInfo, SqlFunctionCtx* pCtx, STimeWindow } } -static FORCE_INLINE TSKEY reviseWindowEkey(STaskAttr* pQueryAttr, STimeWindow* pWindow) { - TSKEY ekey = -1; - int32_t order = TSDB_ORDER_ASC; - if (order == TSDB_ORDER_ASC) { - ekey = pWindow->ekey; - if (ekey > pQueryAttr->window.ekey) { - ekey = pQueryAttr->window.ekey; - } - } else { - ekey = pWindow->skey; - if (ekey < pQueryAttr->window.ekey) { - ekey = pQueryAttr->window.ekey; - } - } - - return ekey; -} - static int32_t doSetInputDataBlock(SOperatorInfo* pOperator, SqlFunctionCtx* pCtx, SSDataBlock* pBlock, int32_t order, int32_t scanFlag, bool createDummyCol); static void doSetInputDataBlockInfo(SOperatorInfo* pOperator, SqlFunctionCtx* pCtx, SSDataBlock* pBlock, int32_t order) { - for (int32_t i = 0; i < pOperator->numOfExprs; ++i) { + for (int32_t i = 0; i < pOperator->exprSupp.numOfExprs; ++i) { pCtx[i].order = order; pCtx[i].input.numOfRows = pBlock->info.rows; - setBlockStatisInfo(&pCtx[i], &pOperator->pExpr[i], pBlock); + setBlockStatisInfo(&pCtx[i], &pOperator->exprSupp.pExprInfo[i], pBlock); } } @@ -602,7 +486,7 @@ static int32_t doSetInputDataBlock(SOperatorInfo* pOperator, SqlFunctionCtx* pCt int32_t scanFlag, bool createDummyCol) { int32_t code = TSDB_CODE_SUCCESS; - for (int32_t i = 0; i < pOperator->numOfExprs; ++i) { + for (int32_t i = 0; i < pOperator->exprSupp.numOfExprs; ++i) { pCtx[i].order = order; pCtx[i].input.numOfRows = pBlock->info.rows; @@ -613,7 +497,7 @@ static int32_t doSetInputDataBlock(SOperatorInfo* pOperator, SqlFunctionCtx* pCt pInput->uid = pBlock->info.uid; pInput->colDataAggIsSet = false; - SExprInfo* pOneExpr = &pOperator->pExpr[i]; + SExprInfo* pOneExpr = &pOperator->exprSupp.pExprInfo[i]; for (int32_t j = 0; j < pOneExpr->base.numOfParams; ++j) { SFunctParam* pFuncParam = &pOneExpr->base.pParam[j]; if (pFuncParam->type == FUNC_PARAM_TYPE_COLUMN) { @@ -649,7 +533,7 @@ static int32_t doSetInputDataBlock(SOperatorInfo* pOperator, SqlFunctionCtx* pCt } static int32_t doAggregateImpl(SOperatorInfo* pOperator, TSKEY startTs, SqlFunctionCtx* pCtx) { - for (int32_t k = 0; k < pOperator->numOfExprs; ++k) { + for (int32_t k = 0; k < pOperator->exprSupp.numOfExprs; ++k) { if (functionNeedToExecute(&pCtx[k])) { // todo add a dummy funtion to avoid process check if (pCtx[k].fpSet.process == NULL) { @@ -802,20 +686,6 @@ static void setResultRowKey(SResultRow* pResultRow, char* pData, int16_t type) { } } -int32_t setGroupResultOutputBuf(SOptrBasicInfo* binfo, int32_t numOfCols, char* pData, int16_t type, int16_t bytes, - int32_t groupId, SDiskbasedBuf* pBuf, SExecTaskInfo* pTaskInfo, - SAggSupporter* pAggSup) { - SResultRowInfo* pResultRowInfo = &binfo->resultRowInfo; - SqlFunctionCtx* pCtx = binfo->pCtx; - - SResultRow* pResultRow = - doSetResultOutBufByKey(pBuf, pResultRowInfo, (char*)pData, bytes, true, groupId, pTaskInfo, false, pAggSup); - assert(pResultRow != NULL); - - setResultRowInitCtx(pResultRow, pCtx, numOfCols, binfo->rowCellInfoOffset); - return TSDB_CODE_SUCCESS; -} - bool functionNeedToExecute(SqlFunctionCtx* pCtx) { struct SResultRowEntryInfo* pResInfo = GET_RES_INFO(pCtx); @@ -927,137 +797,6 @@ void setBlockStatisInfo(SqlFunctionCtx* pCtx, SExprInfo* pExprInfo, SSDataBlock* // } } -// set the output buffer for the selectivity + tag query -static int32_t setSelectValueColumnInfo(SqlFunctionCtx* pCtx, int32_t numOfOutput) { - int32_t num = 0; - - SqlFunctionCtx* p = NULL; - SqlFunctionCtx** pValCtx = taosMemoryCalloc(numOfOutput, POINTER_BYTES); - if (pValCtx == NULL) { - return TSDB_CODE_QRY_OUT_OF_MEMORY; - } - - for (int32_t i = 0; i < numOfOutput; ++i) { - if (strcmp(pCtx[i].pExpr->pExpr->_function.functionName, "_select_value") == 0) { - pValCtx[num++] = &pCtx[i]; - } else if (fmIsSelectFunc(pCtx[i].functionId)) { - p = &pCtx[i]; - } - // if (functionId == FUNCTION_TAG_DUMMY || functionId == FUNCTION_TS_DUMMY) { - // tagLen += pCtx[i].resDataInfo.bytes; - // pTagCtx[num++] = &pCtx[i]; - // } else if (functionId == FUNCTION_TS || functionId == FUNCTION_TAG) { - // // tag function may be the group by tag column - // // ts may be the required primary timestamp column - // continue; - // } else { - // // the column may be the normal column, group by normal_column, the functionId is FUNCTION_PRJ - // } - } - - if (p != NULL) { - p->subsidiaries.pCtx = pValCtx; - p->subsidiaries.num = num; - } else { - taosMemoryFreeClear(pValCtx); - } - - return TSDB_CODE_SUCCESS; -} - -SqlFunctionCtx* createSqlFunctionCtx(SExprInfo* pExprInfo, int32_t numOfOutput, int32_t** rowCellInfoOffset) { - SqlFunctionCtx* pFuncCtx = (SqlFunctionCtx*)taosMemoryCalloc(numOfOutput, sizeof(SqlFunctionCtx)); - if (pFuncCtx == NULL) { - return NULL; - } - - *rowCellInfoOffset = taosMemoryCalloc(numOfOutput, sizeof(int32_t)); - if (*rowCellInfoOffset == 0) { - taosMemoryFreeClear(pFuncCtx); - return NULL; - } - - for (int32_t i = 0; i < numOfOutput; ++i) { - SExprInfo* pExpr = &pExprInfo[i]; - - SExprBasicInfo* pFunct = &pExpr->base; - SqlFunctionCtx* pCtx = &pFuncCtx[i]; - - pCtx->functionId = -1; - pCtx->curBufPage = -1; - pCtx->pExpr = pExpr; - - if (pExpr->pExpr->nodeType == QUERY_NODE_FUNCTION) { - SFuncExecEnv env = {0}; - pCtx->functionId = pExpr->pExpr->_function.pFunctNode->funcId; - - if (fmIsAggFunc(pCtx->functionId) || fmIsIndefiniteRowsFunc(pCtx->functionId)) { - bool isUdaf = fmIsUserDefinedFunc(pCtx->functionId); - if (!isUdaf) { - fmGetFuncExecFuncs(pCtx->functionId, &pCtx->fpSet); - } else { - char* udfName = pExpr->pExpr->_function.pFunctNode->functionName; - strncpy(pCtx->udfName, udfName, strlen(udfName)); - fmGetUdafExecFuncs(pCtx->functionId, &pCtx->fpSet); - } - pCtx->fpSet.getEnv(pExpr->pExpr->_function.pFunctNode, &env); - } else { - fmGetScalarFuncExecFuncs(pCtx->functionId, &pCtx->sfp); - if (pCtx->sfp.getEnv != NULL) { - pCtx->sfp.getEnv(pExpr->pExpr->_function.pFunctNode, &env); - } - } - pCtx->resDataInfo.interBufSize = env.calcMemSize; - } else if (pExpr->pExpr->nodeType == QUERY_NODE_COLUMN || pExpr->pExpr->nodeType == QUERY_NODE_OPERATOR || - pExpr->pExpr->nodeType == QUERY_NODE_VALUE) { - // for simple column, the result buffer needs to hold at least one element. - pCtx->resDataInfo.interBufSize = pFunct->resSchema.bytes; - } - - pCtx->input.numOfInputCols = pFunct->numOfParams; - pCtx->input.pData = taosMemoryCalloc(pFunct->numOfParams, POINTER_BYTES); - pCtx->input.pColumnDataAgg = taosMemoryCalloc(pFunct->numOfParams, POINTER_BYTES); - - pCtx->pTsOutput = NULL; - pCtx->resDataInfo.bytes = pFunct->resSchema.bytes; - pCtx->resDataInfo.type = pFunct->resSchema.type; - pCtx->order = TSDB_ORDER_ASC; - pCtx->start.key = INT64_MIN; - pCtx->end.key = INT64_MIN; - pCtx->numOfParams = pExpr->base.numOfParams; - pCtx->increase = false; - - pCtx->param = pFunct->pParam; - } - - for (int32_t i = 1; i < numOfOutput; ++i) { - (*rowCellInfoOffset)[i] = - (int32_t)((*rowCellInfoOffset)[i - 1] + sizeof(SResultRowEntryInfo) + pFuncCtx[i - 1].resDataInfo.interBufSize); - } - - setSelectValueColumnInfo(pFuncCtx, numOfOutput); - return pFuncCtx; -} - -static void* destroySqlFunctionCtx(SqlFunctionCtx* pCtx, int32_t numOfOutput) { - if (pCtx == NULL) { - return NULL; - } - - for (int32_t i = 0; i < numOfOutput; ++i) { - for (int32_t j = 0; j < pCtx[i].numOfParams; ++j) { - taosVariantDestroy(&pCtx[i].param[j].param); - } - - taosMemoryFreeClear(pCtx[i].subsidiaries.pCtx); - taosMemoryFree(pCtx[i].input.pData); - taosMemoryFree(pCtx[i].input.pColumnDataAgg); - } - - taosMemoryFreeClear(pCtx); - return NULL; -} - bool isTaskKilled(SExecTaskInfo* pTaskInfo) { // query has been executed more than tsShellActivityTimer, and the retrieve has not arrived // abort current query execution. @@ -1075,36 +814,6 @@ bool isTaskKilled(SExecTaskInfo* pTaskInfo) { void setTaskKilled(SExecTaskInfo* pTaskInfo) { pTaskInfo->code = TSDB_CODE_TSC_QUERY_CANCELLED; } -static bool isCachedLastQuery(STaskAttr* pQueryAttr) { - for (int32_t i = 0; i < pQueryAttr->numOfOutput; ++i) { - int32_t functionId = getExprFunctionId(&pQueryAttr->pExpr1[i]); - // if (functionId == FUNCTION_LAST || functionId == FUNCTION_LAST_DST) { - // continue; - // } - - return false; - } - - int32_t order = TSDB_ORDER_ASC; - if (order != TSDB_ORDER_DESC || !TSWINDOW_IS_EQUAL(pQueryAttr->window, TSWINDOW_DESC_INITIALIZER)) { - return false; - } - - if (pQueryAttr->groupbyColumn) { - return false; - } - - if (pQueryAttr->interval.interval > 0) { - return false; - } - - if (pQueryAttr->numOfFilterCols > 0 || pQueryAttr->havingNum > 0) { - return false; - } - - return true; -} - ///////////////////////////////////////////////////////////////////////////////////////////// // todo refactor : return window void getAlignQueryTimeWindow(SInterval* pInterval, int32_t precision, int64_t key, STimeWindow* win) { @@ -1120,7 +829,9 @@ void getAlignQueryTimeWindow(SInterval* pInterval, int32_t precision, int64_t ke } } +#if 0 static int32_t updateBlockLoadStatus(STaskAttr* pQuery, int32_t status) { + bool hasFirstLastFunc = false; bool hasOtherFunc = false; @@ -1130,7 +841,7 @@ static int32_t updateBlockLoadStatus(STaskAttr* pQuery, int32_t status) { for (int32_t i = 0; i < pQuery->numOfOutput; ++i) { int32_t functionId = getExprFunctionId(&pQuery->pExpr1[i]); -#if 0 + if (functionId == FUNCTION_TS || functionId == FUNCTION_TS_DUMMY || functionId == FUNCTION_TAG || functionId == FUNCTION_TAG_DUMMY) { continue; @@ -1141,7 +852,7 @@ static int32_t updateBlockLoadStatus(STaskAttr* pQuery, int32_t status) { } else { hasOtherFunc = true; } -#endif + } if (hasFirstLastFunc && status == BLK_DATA_NOT_LOAD) { @@ -1155,6 +866,8 @@ static int32_t updateBlockLoadStatus(STaskAttr* pQuery, int32_t status) { return status; } +#endif + // static void updateDataCheckOrder(SQInfo *pQInfo, SQueryTableReq* pQueryMsg, bool stableQuery) { // STaskAttr* pQueryAttr = pQInfo->runtimeEnv.pQueryAttr; // @@ -1266,7 +979,7 @@ static int32_t updateBlockLoadStatus(STaskAttr* pQuery, int32_t status) { // // return filterRangeExecute(pQueryAttr->pFilters, pDataStatis, pQueryAttr->numOfCols, numOfRows); // } - +#if 0 static bool overlapWithTimeWindow(STaskAttr* pQueryAttr, SDataBlockInfo* pBlockInfo) { STimeWindow w = {0}; @@ -1315,55 +1028,7 @@ static bool overlapWithTimeWindow(STaskAttr* pQueryAttr, SDataBlockInfo* pBlockI return false; } - -void doCompactSDataBlock(SSDataBlock* pBlock, int32_t numOfRows, int8_t* p) { - int32_t len = 0; - int32_t start = 0; - for (int32_t j = 0; j < numOfRows; ++j) { - if (p[j] == 1) { - len++; - } else { - if (len > 0) { - int32_t cstart = j - len; - for (int32_t i = 0; i < pBlock->info.numOfCols; ++i) { - SColumnInfoData* pColumnInfoData = taosArrayGet(pBlock->pDataBlock, i); - - int16_t bytes = pColumnInfoData->info.bytes; - memmove(((char*)pColumnInfoData->pData) + start * bytes, pColumnInfoData->pData + cstart * bytes, - len * bytes); - } - - start += len; - len = 0; - } - } - } - - if (len > 0) { - int32_t cstart = numOfRows - len; - for (int32_t i = 0; i < pBlock->info.numOfCols; ++i) { - SColumnInfoData* pColumnInfoData = taosArrayGet(pBlock->pDataBlock, i); - - int16_t bytes = pColumnInfoData->info.bytes; - memmove(pColumnInfoData->pData + start * bytes, pColumnInfoData->pData + cstart * bytes, len * bytes); - } - - start += len; - len = 0; - } - - pBlock->info.rows = start; - pBlock->pBlockAgg = NULL; // clean the block statistics info - - if (start > 0) { - SColumnInfoData* pColumnInfoData = taosArrayGet(pBlock->pDataBlock, 0); - if (pColumnInfoData->info.type == TSDB_DATA_TYPE_TIMESTAMP && - pColumnInfoData->info.colId == PRIMARYKEY_TIMESTAMP_COL_ID) { - pBlock->info.window.skey = *(int64_t*)pColumnInfoData->pData; - pBlock->info.window.ekey = *(int64_t*)(pColumnInfoData->pData + TSDB_KEYSIZE * (start - 1)); - } - } -} +#endif static uint32_t doFilterByBlockTimeWindow(STableScanInfo* pTableScanInfo, SSDataBlock* pBlock) { SqlFunctionCtx* pCtx = pTableScanInfo->pCtx; @@ -1427,17 +1092,17 @@ int32_t loadDataBlockOnDemand(SExecTaskInfo* pTaskInfo, STableScanInfo* pTableSc if (pQueryAttr->pointInterpQuery) { needFilter = chkWindowOutputBufByKey(pRuntimeEnv, pTableScanInfo->pResultRowInfo, &win, masterScan, &pResult, groupId, pTableScanInfo->pCtx, pTableScanInfo->numOfOutput, - pTableScanInfo->rowCellInfoOffset); + pTableScanInfo->rowEntryInfoOffset); } else { if (setResultOutputBufByKey(pRuntimeEnv, pTableScanInfo->pResultRowInfo, pBlock->info.uid, &win, masterScan, &pResult, groupId, pTableScanInfo->pCtx, pTableScanInfo->numOfOutput, - pTableScanInfo->rowCellInfoOffset) != TSDB_CODE_SUCCESS) { + pTableScanInfo->rowEntryInfoOffset) != TSDB_CODE_SUCCESS) { longjmp(pRuntimeEnv->env, TSDB_CODE_QRY_OUT_OF_MEMORY); } } } else if (pQueryAttr->stableQuery && (!pQueryAttr->tsCompQuery) && (!pQueryAttr->diffQuery)) { // stable aggregate, not interval aggregate or normal column aggregate doSetTableGroupOutputBuf(pRuntimeEnv, pTableScanInfo->pResultRowInfo, pTableScanInfo->pCtx, - pTableScanInfo->rowCellInfoOffset, pTableScanInfo->numOfOutput, + pTableScanInfo->rowEntryInfoOffset, pTableScanInfo->numOfOutput, pRuntimeEnv->current->groupIndex); } @@ -1482,7 +1147,7 @@ int32_t loadDataBlockOnDemand(SExecTaskInfo* pTaskInfo, STableScanInfo* pTableSc STimeWindow win = getActiveTimeWindow(pTableScanInfo->pResultRowInfo, k, pQueryAttr); if (setResultOutputBufByKey(pRuntimeEnv, pTableScanInfo->pResultRowInfo, pBlock->info.uid, &win, masterScan, &pResult, groupId, pTableScanInfo->pCtx, pTableScanInfo->numOfOutput, - pTableScanInfo->rowCellInfoOffset) != TSDB_CODE_SUCCESS) { + pTableScanInfo->rowEntryInfoOffset) != TSDB_CODE_SUCCESS) { longjmp(pRuntimeEnv->env, TSDB_CODE_QRY_OUT_OF_MEMORY); } } @@ -1565,14 +1230,13 @@ void initResultRow(SResultRow* pResultRow) { * offset[0] offset[1] offset[2] */ // TODO refactor: some function move away -void setFunctionResultOutput(SOptrBasicInfo* pInfo, SAggSupporter* pSup, int32_t stage, int32_t numOfExprs, - SExecTaskInfo* pTaskInfo) { - SqlFunctionCtx* pCtx = pInfo->pCtx; - SSDataBlock* pDataBlock = pInfo->pRes; - int32_t* rowCellInfoOffset = pInfo->rowCellInfoOffset; +void setFunctionResultOutput(SOperatorInfo *pOperator, SOptrBasicInfo *pInfo, SAggSupporter* pSup, int32_t stage, int32_t numOfExprs) { + SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo; + SqlFunctionCtx* pCtx = pOperator->exprSupp.pCtx; + int32_t* rowEntryInfoOffset = pOperator->exprSupp.rowEntryInfoOffset; SResultRowInfo* pResultRowInfo = &pInfo->resultRowInfo; - initResultRowInfo(pResultRowInfo, 16); + initResultRowInfo(pResultRowInfo); int64_t tid = 0; int64_t groupId = 0; @@ -1580,7 +1244,7 @@ void setFunctionResultOutput(SOptrBasicInfo* pInfo, SAggSupporter* pSup, int32_t pTaskInfo, false, pSup); for (int32_t i = 0; i < numOfExprs; ++i) { - struct SResultRowEntryInfo* pEntry = getResultCell(pRow, i, rowCellInfoOffset); + struct SResultRowEntryInfo* pEntry = getResultEntryInfo(pRow, i, rowEntryInfoOffset); cleanupResultRowEntry(pEntry); pCtx[i].resultInfo = pEntry; @@ -1590,42 +1254,6 @@ void setFunctionResultOutput(SOptrBasicInfo* pInfo, SAggSupporter* pSup, int32_t initCtxOutputBuffer(pCtx, numOfExprs); } -void updateOutputBuf(SOptrBasicInfo* pBInfo, int32_t* bufCapacity, int32_t numOfInputRows) { - SSDataBlock* pDataBlock = pBInfo->pRes; - - int32_t newSize = pDataBlock->info.rows + numOfInputRows + 5; // extra output buffer - if ((*bufCapacity) < newSize) { - for (int32_t i = 0; i < pDataBlock->info.numOfCols; ++i) { - SColumnInfoData* pColInfo = taosArrayGet(pDataBlock->pDataBlock, i); - - char* p = taosMemoryRealloc(pColInfo->pData, newSize * pColInfo->info.bytes); - if (p != NULL) { - pColInfo->pData = p; - - // it starts from the tail of the previously generated results. - pBInfo->pCtx[i].pOutput = pColInfo->pData; - (*bufCapacity) = newSize; - } else { - // longjmp - } - } - } - - for (int32_t i = 0; i < pDataBlock->info.numOfCols; ++i) { - SColumnInfoData* pColInfo = taosArrayGet(pDataBlock->pDataBlock, i); - pBInfo->pCtx[i].pOutput = pColInfo->pData + pColInfo->info.bytes * pDataBlock->info.rows; - - // set the correct pointer after the memory buffer reallocated. - int32_t functionId = pBInfo->pCtx[i].functionId; -#if 0 - if (functionId == FUNCTION_TOP || functionId == FUNCTION_BOTTOM || functionId == FUNCTION_DIFF || - functionId == FUNCTION_DERIVATIVE) { - // if (i > 0) pBInfo->pCtx[i].pTsOutput = pBInfo->pCtx[i - 1].pOutput; - } -#endif - } -} - void initCtxOutputBuffer(SqlFunctionCtx* pCtx, int32_t size) { for (int32_t j = 0; j < size; ++j) { struct SResultRowEntryInfo* pResInfo = GET_RES_INFO(&pCtx[j]); @@ -1657,9 +1285,9 @@ void destroyTableQueryInfoImpl(STableQueryInfo* pTableQueryInfo) { // cleanupResultRowInfo(&pTableQueryInfo->resInfo); } -void setResultRowInitCtx(SResultRow* pResult, SqlFunctionCtx* pCtx, int32_t numOfOutput, int32_t* rowCellInfoOffset) { +void setResultRowInitCtx(SResultRow* pResult, SqlFunctionCtx* pCtx, int32_t numOfOutput, int32_t* rowEntryInfoOffset) { for (int32_t i = 0; i < numOfOutput; ++i) { - pCtx[i].resultInfo = getResultCell(pResult, i, rowCellInfoOffset); + pCtx[i].resultInfo = getResultEntryInfo(pResult, i, rowEntryInfoOffset); struct SResultRowEntryInfo* pResInfo = pCtx[i].resultInfo; if (isRowEntryCompleted(pResInfo) && isRowEntryInitialized(pResInfo)) { @@ -1752,14 +1380,12 @@ void extractQualifiedTupleByFilterResult(SSDataBlock* pBlock, const int8_t* rowR } } -void doSetTableGroupOutputBuf(SAggOperatorInfo* pAggInfo, int32_t numOfOutput, uint64_t groupId, - SExecTaskInfo* pTaskInfo) { +void doSetTableGroupOutputBuf(SOperatorInfo* pOperator, SAggOperatorInfo* pAggInfo, int32_t numOfOutput, uint64_t groupId) { // for simple group by query without interval, all the tables belong to one group result. - int64_t uid = 0; - + SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo; SResultRowInfo* pResultRowInfo = &pAggInfo->binfo.resultRowInfo; - SqlFunctionCtx* pCtx = pAggInfo->binfo.pCtx; - int32_t* rowCellInfoOffset = pAggInfo->binfo.rowCellInfoOffset; + SqlFunctionCtx* pCtx = pOperator->exprSupp.pCtx; + int32_t* rowEntryInfoOffset = pOperator->exprSupp.rowEntryInfoOffset; SResultRow* pResultRow = doSetResultOutBufByKey(pAggInfo->aggSup.pResultBuf, pResultRowInfo, (char*)&groupId, sizeof(groupId), true, groupId, pTaskInfo, false, &pAggInfo->aggSup); @@ -1777,15 +1403,15 @@ void doSetTableGroupOutputBuf(SAggOperatorInfo* pAggInfo, int32_t numOfOutput, u } } - setResultRowInitCtx(pResultRow, pCtx, numOfOutput, rowCellInfoOffset); + setResultRowInitCtx(pResultRow, pCtx, numOfOutput, rowEntryInfoOffset); } -void setExecutionContext(int32_t numOfOutput, uint64_t groupId, SExecTaskInfo* pTaskInfo, SAggOperatorInfo* pAggInfo) { +void setExecutionContext(SOperatorInfo* pOperator, int32_t numOfOutput, uint64_t groupId, SAggOperatorInfo* pAggInfo) { if (pAggInfo->groupId != INT32_MIN && pAggInfo->groupId == groupId) { return; } - doSetTableGroupOutputBuf(pAggInfo, numOfOutput, groupId, pTaskInfo); + doSetTableGroupOutputBuf(pOperator, pAggInfo, numOfOutput, groupId); // record the current active group id pAggInfo->groupId = groupId; @@ -1793,7 +1419,7 @@ void setExecutionContext(int32_t numOfOutput, uint64_t groupId, SExecTaskInfo* p static void doUpdateNumOfRows(SResultRow* pRow, int32_t numOfExprs, const int32_t* rowCellOffset) { for (int32_t j = 0; j < numOfExprs; ++j) { - struct SResultRowEntryInfo* pResInfo = getResultCell(pRow, j, rowCellOffset); + struct SResultRowEntryInfo* pResInfo = getResultEntryInfo(pRow, j, rowCellOffset); if (!isRowEntryInitialized(pResInfo)) { continue; } @@ -1829,7 +1455,7 @@ int32_t finalizeResultRowIntoResultDataBlock(SDiskbasedBuf* pBuf, SResultRowPosi for (int32_t j = 0; j < numOfExprs; ++j) { int32_t slotId = pExprInfo[j].base.resSchema.slotId; - pCtx[j].resultInfo = getResultCell(pRow, j, rowCellOffset); + pCtx[j].resultInfo = getResultEntryInfo(pRow, j, rowCellOffset); if (pCtx[j].fpSet.finalize) { int32_t code = pCtx[j].fpSet.finalize(&pCtx[j], pBlock); if (TAOS_FAILED(code)) { @@ -1894,7 +1520,7 @@ int32_t doCopyToSDataBlock(SExecTaskInfo* pTaskInfo, SSDataBlock* pBlock, SExprI for (int32_t j = 0; j < numOfExprs; ++j) { int32_t slotId = pExprInfo[j].base.resSchema.slotId; - pCtx[j].resultInfo = getResultCell(pRow, j, rowCellOffset); + pCtx[j].resultInfo = getResultEntryInfo(pRow, j, rowCellOffset); if (pCtx[j].fpSet.finalize) { int32_t code = pCtx[j].fpSet.finalize(&pCtx[j], pBlock); if (TAOS_FAILED(code)) { @@ -1937,16 +1563,16 @@ int32_t doCopyToSDataBlock(SExecTaskInfo* pTaskInfo, SSDataBlock* pBlock, SExprI void doBuildResultDatablock(SOperatorInfo* pOperator, SOptrBasicInfo* pbInfo, SGroupResInfo* pGroupResInfo, SDiskbasedBuf* pBuf) { - SExprInfo* pExprInfo = pOperator->pExpr; - int32_t numOfExprs = pOperator->numOfExprs; + SExprInfo* pExprInfo = pOperator->exprSupp.pExprInfo; + int32_t numOfExprs = pOperator->exprSupp.numOfExprs; SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo; - int32_t* rowCellOffset = pbInfo->rowCellInfoOffset; + int32_t* rowCellOffset = pOperator->exprSupp.rowEntryInfoOffset; SSDataBlock* pBlock = pbInfo->pRes; - SqlFunctionCtx* pCtx = pbInfo->pCtx; + SqlFunctionCtx* pCtx = pOperator->exprSupp.pCtx; blockDataCleanup(pBlock); - if (!hashRemainDataInGroupInfo(pGroupResInfo)) { + if (!hasDataInGroupInfo(pGroupResInfo)) { return; } @@ -1956,7 +1582,7 @@ void doBuildResultDatablock(SOperatorInfo* pOperator, SOptrBasicInfo* pbInfo, SG } static void updateNumOfRowsInResultRows(SqlFunctionCtx* pCtx, int32_t numOfOutput, SResultRowInfo* pResultRowInfo, - int32_t* rowCellInfoOffset) { + int32_t* rowEntryInfoOffset) { // update the number of result for each, only update the number of rows for the corresponding window result. // if (QUERY_IS_INTERVAL_QUERY(pQueryAttr)) { // return; @@ -1971,7 +1597,7 @@ static void updateNumOfRowsInResultRows(SqlFunctionCtx* pCtx, int32_t numOfOutpu continue; } - SResultRowEntryInfo* pCell = getResultCell(pResult, j, rowCellInfoOffset); + SResultRowEntryInfo* pCell = getResultEntryInfo(pResult, j, rowEntryInfoOffset); pResult->numOfRows = (uint16_t)(TMAX(pResult->numOfRows, pCell->numOfRes)); } } @@ -2413,33 +2039,7 @@ static int32_t doSendFetchDataRequest(SExchangeInfo* pExchangeInfo, SExecTaskInf return TSDB_CODE_SUCCESS; } -// NOTE: sources columns are more than the destination SSDatablock columns. -void relocateColumnData(SSDataBlock* pBlock, const SArray* pColMatchInfo, SArray* pCols) { - size_t numOfSrcCols = taosArrayGetSize(pCols); - - int32_t i = 0, j = 0; - while (i < numOfSrcCols && j < taosArrayGetSize(pColMatchInfo)) { - SColumnInfoData* p = taosArrayGet(pCols, i); - SColMatchInfo* pmInfo = taosArrayGet(pColMatchInfo, j); - if (!pmInfo->output) { - j++; - continue; - } - - if (p->info.colId == pmInfo->colId) { - SColumnInfoData* pDst = taosArrayGet(pBlock->pDataBlock, pmInfo->targetSlotId); - colDataAssign(pDst, p, pBlock->info.rows); - i++; - j++; - } else if (p->info.colId < pmInfo->colId) { - i++; - } else { - ASSERT(0); - } - } -} - -int32_t setDataBlockFromFetchRsp(SSDataBlock* pRes, SLoadRemoteDataInfo* pLoadInfo, int32_t numOfRows, char* pData, +int32_t extractDataBlockFromFetchRsp(SSDataBlock* pRes, SLoadRemoteDataInfo* pLoadInfo, int32_t numOfRows, char* pData, int32_t compLen, int32_t numOfOutput, int64_t startTs, uint64_t* total, SArray* pColList) { if (pColList == NULL) { // data from other sources @@ -2565,7 +2165,7 @@ static SSDataBlock* concurrentlyLoadRemoteDataImpl(SOperatorInfo* pOperator, SEx } SRetrieveTableRsp* pTableRsp = pDataInfo->pRsp; - code = setDataBlockFromFetchRsp(pExchangeInfo->pResult, pLoadInfo, pTableRsp->numOfRows, pTableRsp->data, + code = extractDataBlockFromFetchRsp(pExchangeInfo->pResult, pLoadInfo, pTableRsp->numOfRows, pTableRsp->data, pTableRsp->compLen, pTableRsp->numOfCols, startTs, &pDataInfo->totalRows, NULL); if (code != 0) { taosMemoryFreeClear(pDataInfo->pRsp); @@ -2680,7 +2280,7 @@ static SSDataBlock* seqLoadRemoteData(SOperatorInfo* pOperator) { SSDataBlock* pRes = pExchangeInfo->pResult; SRetrieveTableRsp* pTableRsp = pDataInfo->pRsp; int32_t code = - setDataBlockFromFetchRsp(pExchangeInfo->pResult, pLoadInfo, pTableRsp->numOfRows, pTableRsp->data, + extractDataBlockFromFetchRsp(pExchangeInfo->pResult, pLoadInfo, pTableRsp->numOfRows, pTableRsp->data, pTableRsp->compLen, pTableRsp->numOfCols, startTs, &pDataInfo->totalRows, NULL); if (pRsp->completed == 1) { @@ -2817,7 +2417,7 @@ SOperatorInfo* createExchangeOperatorInfo(void* pTransporter, SExchangePhysiNode pOperator->blocking = false; pOperator->status = OP_NOT_OPENED; pOperator->info = pInfo; - pOperator->numOfExprs = pInfo->pResult->info.numOfCols; + pOperator->exprSupp.numOfExprs = pInfo->pResult->info.numOfCols; pOperator->pTaskInfo = pTaskInfo; pOperator->fpSet = createOperatorFpSet(prepareLoadRemoteData, doLoadRemoteData, NULL, NULL, @@ -2938,7 +2538,7 @@ static bool saveCurrentTuple(char** rowColData, SArray* pColumnList, SSDataBlock static void doMergeImpl(SOperatorInfo* pOperator, int32_t numOfExpr, SSDataBlock* pBlock) { SSortedMergeOperatorInfo* pInfo = pOperator->info; - SqlFunctionCtx* pCtx = pInfo->binfo.pCtx; + SqlFunctionCtx* pCtx = pOperator->exprSupp.pCtx; for (int32_t i = 0; i < pBlock->info.numOfCols; ++i) { // pCtx[i].size = 1; } @@ -2953,8 +2553,8 @@ static void doMergeImpl(SOperatorInfo* pOperator, int32_t numOfExpr, SSDataBlock doMergeResultImpl(pInfo, pCtx, numOfExpr, i); } else { doFinalizeResultImpl(pCtx, numOfExpr); - int32_t numOfRows = getNumOfResult(pInfo->binfo.pCtx, pOperator->numOfExprs, NULL); - // setTagValueForMultipleRows(pCtx, pOperator->numOfExprs, numOfRows); + int32_t numOfRows = getNumOfResult(pOperator->exprSupp.pCtx, pOperator->exprSupp.numOfExprs, NULL); + // setTagValueForMultipleRows(pCtx, pOperator->exprSupp.numOfExprs, numOfRows); // TODO check for available buffer; @@ -3001,16 +2601,16 @@ static SSDataBlock* doMerge(SOperatorInfo* pOperator) { break; } - setInputDataBlock(pOperator, pInfo->binfo.pCtx, pDataBlock, TSDB_ORDER_ASC, MAIN_SCAN, true); + setInputDataBlock(pOperator, pOperator->exprSupp.pCtx, pDataBlock, TSDB_ORDER_ASC, MAIN_SCAN, true); // updateOutputBuf(&pInfo->binfo, &pAggInfo->bufCapacity, pBlock->info.rows * pAggInfo->resultRowFactor, // pOperator->pRuntimeEnv, true); - doMergeImpl(pOperator, pOperator->numOfExprs, pDataBlock); + doMergeImpl(pOperator, pOperator->exprSupp.numOfExprs, pDataBlock); // flush to tuple store, and after all data have been handled, return to upstream node or sink node } - doFinalizeResultImpl(pInfo->binfo.pCtx, pOperator->numOfExprs); - int32_t numOfRows = getNumOfResult(pInfo->binfo.pCtx, pOperator->numOfExprs, NULL); - // setTagValueForMultipleRows(pCtx, pOperator->numOfExprs, numOfRows); + doFinalizeResultImpl(pOperator->exprSupp.pCtx, pOperator->exprSupp.numOfExprs); + int32_t numOfRows = getNumOfResult(pOperator->exprSupp.pCtx, pOperator->exprSupp.numOfExprs, NULL); + // setTagValueForMultipleRows(pCtx, pOperator->exprSupp.numOfExprs, numOfRows); // TODO check for available buffer; @@ -3073,7 +2673,7 @@ static SSDataBlock* doSortedMerge(SOperatorInfo* pOperator) { } int32_t numOfBufPage = pInfo->sortBufSize / pInfo->bufPageSize; - pInfo->pSortHandle = tsortCreateSortHandle(pInfo->pSortInfo, NULL, SORT_MULTISOURCE_MERGE, pInfo->bufPageSize, + pInfo->pSortHandle = tsortCreateSortHandle(pInfo->pSortInfo, SORT_MULTISOURCE_MERGE, pInfo->bufPageSize, numOfBufPage, pInfo->binfo.pRes, "GET_TASKID(pTaskInfo)"); tsortSetFetchRawDataFp(pInfo->pSortHandle, loadNextDataBlock, NULL, NULL); @@ -3151,20 +2751,20 @@ SOperatorInfo* createSortedMergeOperatorInfo(SOperatorInfo** downstream, int32_t goto _error; } - pInfo->binfo.pCtx = createSqlFunctionCtx(pExprInfo, num, &pInfo->binfo.rowCellInfoOffset); - initResultRowInfo(&pInfo->binfo.resultRowInfo, (int32_t)1); + pOperator->exprSupp.pCtx = createSqlFunctionCtx(pExprInfo, num, &pOperator->exprSupp.rowEntryInfoOffset); + initResultRowInfo(&pInfo->binfo.resultRowInfo); - if (pInfo->binfo.pCtx == NULL || pInfo->binfo.pRes == NULL) { + if (pOperator->exprSupp.pCtx == NULL || pInfo->binfo.pRes == NULL) { goto _error; } size_t keyBufSize = sizeof(int64_t) + sizeof(int64_t) + POINTER_BYTES; - int32_t code = doInitAggInfoSup(&pInfo->aggSup, pInfo->binfo.pCtx, num, keyBufSize, pTaskInfo->id.str); + int32_t code = doInitAggInfoSup(&pInfo->aggSup, pOperator->exprSupp.pCtx, num, keyBufSize, pTaskInfo->id.str); if (code != TSDB_CODE_SUCCESS) { goto _error; } - setFunctionResultOutput(&pInfo->binfo, &pInfo->aggSup, MAIN_SCAN, num, pTaskInfo); + setFunctionResultOutput(pOperator, &pInfo->binfo, &pInfo->aggSup, MAIN_SCAN, num); code = initGroupCol(pExprInfo, num, pGroupInfo, pInfo); if (code != TSDB_CODE_SUCCESS) { goto _error; @@ -3183,8 +2783,8 @@ SOperatorInfo* createSortedMergeOperatorInfo(SOperatorInfo** downstream, int32_t pOperator->blocking = true; pOperator->status = OP_NOT_OPENED; pOperator->info = pInfo; - pOperator->numOfExprs = num; - pOperator->pExpr = pExprInfo; + pOperator->exprSupp.numOfExprs = num; + pOperator->exprSupp.pExprInfo = pExprInfo; pOperator->pTaskInfo = pTaskInfo; @@ -3239,6 +2839,7 @@ static int32_t doOpenAggregateOptr(SOperatorInfo* pOperator) { SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo; SAggOperatorInfo* pAggInfo = pOperator->info; + SExprSupp* pSup = &pOperator->exprSupp; SOptrBasicInfo* pInfo = &pAggInfo->binfo; SOperatorInfo* downstream = pOperator->pDownstream[0]; @@ -3259,18 +2860,18 @@ static int32_t doOpenAggregateOptr(SOperatorInfo* pOperator) { } // there is an scalar expression that needs to be calculated before apply the group aggregation. - if (pAggInfo->pScalarExprInfo != NULL) { - code = projectApplyFunctions(pAggInfo->pScalarExprInfo, pBlock, pBlock, pAggInfo->pScalarCtx, - pAggInfo->numOfScalarExpr, NULL); + if (pAggInfo->scalarExprSup.pExprInfo != NULL) { + SExprSupp* pSup1 = &pAggInfo->scalarExprSup; + code = projectApplyFunctions(pSup1->pExprInfo, pBlock, pBlock, pSup1->pCtx, pSup1->numOfExprs, NULL); if (code != TSDB_CODE_SUCCESS) { longjmp(pTaskInfo->env, code); } } // the pDataBlock are always the same one, no need to call this again - setExecutionContext(pOperator->numOfExprs, pBlock->info.groupId, pTaskInfo, pAggInfo); - setInputDataBlock(pOperator, pInfo->pCtx, pBlock, order, scanFlag, true); - code = doAggregateImpl(pOperator, 0, pInfo->pCtx); + setExecutionContext(pOperator, pOperator->exprSupp.numOfExprs, pBlock->info.groupId, pAggInfo); + setInputDataBlock(pOperator, pSup->pCtx, pBlock, order, scanFlag, true); + code = doAggregateImpl(pOperator, 0, pSup->pCtx); if (code != 0) { longjmp(pTaskInfo->env, code); } @@ -3316,7 +2917,7 @@ static SSDataBlock* getAggregateResult(SOperatorInfo* pOperator) { blockDataEnsureCapacity(pInfo->pRes, pOperator->resultInfo.capacity); doBuildResultDatablock(pOperator, pInfo, &pAggInfo->groupResInfo, pAggInfo->aggSup.pResultBuf); - if (pInfo->pRes->info.rows == 0 || !hashRemainDataInGroupInfo(&pAggInfo->groupResInfo)) { + if (pInfo->pRes->info.rows == 0 || !hasDataInGroupInfo(&pAggInfo->groupResInfo)) { doSetOperatorCompleted(pOperator); } @@ -3529,6 +3130,7 @@ static SSDataBlock* doProjectOperation(SOperatorInfo* pOperator) { SProjectOperatorInfo* pProjectInfo = pOperator->info; SOptrBasicInfo* pInfo = &pProjectInfo->binfo; + SExprSupp* pSup = &pOperator->exprSupp; SSDataBlock* pRes = pInfo->pRes; blockDataCleanup(pRes); @@ -3546,10 +3148,10 @@ static SSDataBlock* doProjectOperation(SOperatorInfo* pOperator) { setInputDataBlock(pOperator, pInfo->pCtx, pBlock, TSDB_ORDER_ASC); blockDataEnsureCapacity(pInfo->pRes, pBlock->info.rows); - projectApplyFunctions(pOperator->pExpr, pInfo->pRes, pBlock, pInfo->pCtx, pOperator->numOfExprs); + projectApplyFunctions(pOperator->exprSupp.pExprInfo, pInfo->pRes, pBlock, pInfo->pCtx, pOperator->exprSupp.numOfExprs); if (pRes->info.rows >= pProjectInfo->binfo.capacity * 0.8) { - copyTsColoum(pRes, pInfo->pCtx, pOperator->numOfExprs); - resetResultRowEntryResult(pInfo->pCtx, pOperator->numOfExprs); + copyTsColoum(pRes, pInfo->pCtx, pOperator->exprSupp.numOfExprs); + resetResultRowEntryResult(pInfo->pCtx, pOperator->exprSupp.numOfExprs); return pRes; } } @@ -3581,10 +3183,10 @@ static SSDataBlock* doProjectOperation(SOperatorInfo* pOperator) { doFilter(pProjectInfo->pFilterNode, pBlock); - setInputDataBlock(pOperator, pInfo->pCtx, pBlock, order, scanFlag, false); + setInputDataBlock(pOperator, pSup->pCtx, pBlock, order, scanFlag, false); blockDataEnsureCapacity(pInfo->pRes, pInfo->pRes->info.rows + pBlock->info.rows); - code = projectApplyFunctions(pOperator->pExpr, pInfo->pRes, pBlock, pInfo->pCtx, pOperator->numOfExprs, + code = projectApplyFunctions(pSup->pExprInfo, pInfo->pRes, pBlock, pSup->pCtx, pSup->numOfExprs, pProjectInfo->pPseudoColInfo); if (code != TSDB_CODE_SUCCESS) { longjmp(pTaskInfo->env, code); @@ -3735,7 +3337,7 @@ static void destroyOperatorInfo(SOperatorInfo* pOperator) { } if (pOperator->fpSet.closeFn != NULL) { - pOperator->fpSet.closeFn(pOperator->info, pOperator->numOfExprs); + pOperator->fpSet.closeFn(pOperator->info, pOperator->exprSupp.numOfExprs); } if (pOperator->pDownstream != NULL) { @@ -3747,11 +3349,11 @@ static void destroyOperatorInfo(SOperatorInfo* pOperator) { pOperator->numOfDownstream = 0; } - if (pOperator->pExpr != NULL) { - destroyExprInfo(pOperator->pExpr, pOperator->numOfExprs); + if (pOperator->exprSupp.pExprInfo != NULL) { + destroyExprInfo(pOperator->exprSupp.pExprInfo, pOperator->exprSupp.numOfExprs); } - taosMemoryFreeClear(pOperator->pExpr); + taosMemoryFreeClear(pOperator->exprSupp.pExprInfo); taosMemoryFreeClear(pOperator->info); taosMemoryFreeClear(pOperator); } @@ -3801,15 +3403,12 @@ void cleanupAggSup(SAggSupporter* pAggSup) { destroyDiskbasedBuf(pAggSup->pResultBuf); } -int32_t initAggInfo(SOptrBasicInfo* pBasicInfo, SAggSupporter* pAggSup, SExprInfo* pExprInfo, int32_t numOfCols, - SSDataBlock* pResultBlock, size_t keyBufSize, const char* pkey) { - pBasicInfo->pCtx = createSqlFunctionCtx(pExprInfo, numOfCols, &pBasicInfo->rowCellInfoOffset); - pBasicInfo->pRes = pResultBlock; - - doInitAggInfoSup(pAggSup, pBasicInfo->pCtx, numOfCols, keyBufSize, pkey); - +int32_t initAggInfo(SExprSupp* pSup, SAggSupporter* pAggSup, SExprInfo* pExprInfo, int32_t numOfCols, + size_t keyBufSize, const char* pkey) { + initExprSupp(pSup, pExprInfo, numOfCols); + doInitAggInfoSup(pAggSup, pSup->pCtx, numOfCols, keyBufSize, pkey); for (int32_t i = 0; i < numOfCols; ++i) { - pBasicInfo->pCtx[i].pBuf = pAggSup->pResultBuf; + pSup->pCtx[i].pBuf = pAggSup->pResultBuf; } return TSDB_CODE_SUCCESS; @@ -3824,6 +3423,19 @@ void initResultSizeInfo(SOperatorInfo* pOperator, int32_t numOfRows) { } } +void initBasicInfo(SOptrBasicInfo* pInfo, SSDataBlock* pBlock) { + pInfo->pRes = pBlock; + initResultRowInfo(&pInfo->resultRowInfo); +} + +void initExprSupp(SExprSupp* pSup, SExprInfo* pExprInfo, int32_t numOfExpr) { + pSup->pExprInfo = pExprInfo; + pSup->numOfExprs = numOfExpr; + if (pSup->pExprInfo != NULL) { + pSup->pCtx = createSqlFunctionCtx(pExprInfo, numOfExpr, &pSup->rowEntryInfoOffset); + } +} + SOperatorInfo* createAggregateOperatorInfo(SOperatorInfo* downstream, SExprInfo* pExprInfo, int32_t numOfCols, SSDataBlock* pResultBlock, SExprInfo* pScalarExprInfo, int32_t numOfScalarExpr, SExecTaskInfo* pTaskInfo) { @@ -3837,29 +3449,20 @@ SOperatorInfo* createAggregateOperatorInfo(SOperatorInfo* downstream, SExprInfo* size_t keyBufSize = sizeof(int64_t) + sizeof(int64_t) + POINTER_BYTES; initResultSizeInfo(pOperator, numOfRows); - int32_t code = - initAggInfo(&pInfo->binfo, &pInfo->aggSup, pExprInfo, numOfCols, pResultBlock, keyBufSize, pTaskInfo->id.str); + int32_t code = initAggInfo(&pOperator->exprSupp, &pInfo->aggSup, pExprInfo, numOfCols, keyBufSize, pTaskInfo->id.str); if (code != TSDB_CODE_SUCCESS) { goto _error; } - int32_t numOfGroup = 10; // todo replaced with true value - pInfo->groupId = INT32_MIN; - initResultRowInfo(&pInfo->binfo.resultRowInfo, numOfGroup); + initBasicInfo(&pInfo->binfo, pResultBlock); + initExprSupp(&pInfo->scalarExprSup, pScalarExprInfo, numOfScalarExpr); - pInfo->pScalarExprInfo = pScalarExprInfo; - pInfo->numOfScalarExpr = numOfScalarExpr; - if (pInfo->pScalarExprInfo != NULL) { - pInfo->pScalarCtx = createSqlFunctionCtx(pScalarExprInfo, numOfScalarExpr, &pInfo->rowCellInfoOffset); - } - - pOperator->name = "TableAggregate"; + pInfo->groupId = INT32_MIN; + pOperator->name = "TableAggregate"; pOperator->operatorType = QUERY_NODE_PHYSICAL_PLAN_HASH_AGG; - pOperator->blocking = true; - pOperator->status = OP_NOT_OPENED; - pOperator->info = pInfo; - pOperator->pExpr = pExprInfo; - pOperator->numOfExprs = numOfCols; + pOperator->blocking = true; + pOperator->status = OP_NOT_OPENED; + pOperator->info = pInfo; pOperator->pTaskInfo = pTaskInfo; pOperator->fpSet = createOperatorFpSet(doOpenAggregateOptr, getAggregateResult, NULL, NULL, destroyAggOperatorInfo, @@ -3879,24 +3482,39 @@ _error: return NULL; } -void doDestroyBasicInfo(SOptrBasicInfo* pInfo, int32_t numOfOutput) { - assert(pInfo != NULL); +static void* destroySqlFunctionCtx(SqlFunctionCtx* pCtx, int32_t numOfOutput) { + if (pCtx == NULL) { + return NULL; + } + + for (int32_t i = 0; i < numOfOutput; ++i) { + for (int32_t j = 0; j < pCtx[i].numOfParams; ++j) { + taosVariantDestroy(&pCtx[i].param[j].param); + } - destroySqlFunctionCtx(pInfo->pCtx, numOfOutput); - taosMemoryFreeClear(pInfo->rowCellInfoOffset); + taosMemoryFreeClear(pCtx[i].subsidiaries.pCtx); + taosMemoryFree(pCtx[i].input.pData); + taosMemoryFree(pCtx[i].input.pColumnDataAgg); + } + taosMemoryFreeClear(pCtx); + return NULL; +} + +void cleanupBasicInfo(SOptrBasicInfo* pInfo) { + assert(pInfo != NULL); cleanupResultRowInfo(&pInfo->resultRowInfo); pInfo->pRes = blockDataDestroy(pInfo->pRes); } void destroyBasicOperatorInfo(void* param, int32_t numOfOutput) { SOptrBasicInfo* pInfo = (SOptrBasicInfo*)param; - doDestroyBasicInfo(pInfo, numOfOutput); + cleanupBasicInfo(pInfo); } void destroyAggOperatorInfo(void* param, int32_t numOfOutput) { SAggOperatorInfo* pInfo = (SAggOperatorInfo*)param; - doDestroyBasicInfo(&pInfo->binfo, numOfOutput); + cleanupBasicInfo(&pInfo->binfo); } void destroySFillOperatorInfo(void* param, int32_t numOfOutput) { @@ -3911,22 +3529,25 @@ static void destroyProjectOperatorInfo(void* param, int32_t numOfOutput) { return; } SProjectOperatorInfo* pInfo = (SProjectOperatorInfo*)param; - doDestroyBasicInfo(&pInfo->binfo, numOfOutput); + cleanupBasicInfo(&pInfo->binfo); cleanupAggSup(&pInfo->aggSup); taosArrayDestroy(pInfo->pPseudoColInfo); } +void cleanupExecSupp(SExprSupp* pSupp) { + destroySqlFunctionCtx(pSupp->pCtx, pSupp->numOfExprs); + destroyExprInfo(pSupp->pExprInfo, pSupp->numOfExprs); + + taosMemoryFree(pSupp->rowEntryInfoOffset); +} + static void destroyIndefinitOperatorInfo(void* param, int32_t numOfOutput) { SIndefOperatorInfo* pInfo = (SIndefOperatorInfo*)param; - doDestroyBasicInfo(&pInfo->binfo, numOfOutput); + cleanupBasicInfo(&pInfo->binfo); taosArrayDestroy(pInfo->pPseudoColInfo); cleanupAggSup(&pInfo->aggSup); - - destroySqlFunctionCtx(pInfo->pScalarCtx, numOfOutput); - destroyExprInfo(pInfo->pScalarExpr, pInfo->numOfScalarExpr); - - taosMemoryFree(pInfo->rowCellInfoOffset); + cleanupExecSupp(&pInfo->scalarSup); } void destroyExchangeOperatorInfo(void* param, int32_t numOfOutput) { @@ -3957,23 +3578,27 @@ static SArray* setRowTsColumnOutputInfo(SqlFunctionCtx* pCtx, int32_t numOfCols) return pList; } -SOperatorInfo* createProjectOperatorInfo(SOperatorInfo* downstream, SExprInfo* pExprInfo, int32_t num, - SSDataBlock* pResBlock, SLimit* pLimit, SLimit* pSlimit, SNode* pCondition, - SExecTaskInfo* pTaskInfo) { +SOperatorInfo* createProjectOperatorInfo(SOperatorInfo* downstream, SProjectPhysiNode* pProjPhyNode, SExecTaskInfo* pTaskInfo) { SProjectOperatorInfo* pInfo = taosMemoryCalloc(1, sizeof(SProjectOperatorInfo)); SOperatorInfo* pOperator = taosMemoryCalloc(1, sizeof(SOperatorInfo)); if (pInfo == NULL || pOperator == NULL) { goto _error; } - pInfo->limit = *pLimit; - pInfo->slimit = *pSlimit; - pInfo->curOffset = pLimit->offset; - pInfo->curSOffset = pSlimit->offset; - pInfo->binfo.pRes = pResBlock; - pInfo->pFilterNode = pCondition; + int32_t numOfCols = 0; + SExprInfo* pExprInfo = createExprInfo(pProjPhyNode->pProjections, NULL, &numOfCols); + + SSDataBlock* pResBlock = createResDataBlock(pProjPhyNode->node.pOutputDataBlockDesc); + SLimit limit = {.limit = pProjPhyNode->limit, .offset = pProjPhyNode->offset}; + SLimit slimit = {.limit = pProjPhyNode->slimit, .offset = pProjPhyNode->soffset}; + + pInfo->limit = limit; + pInfo->slimit = slimit; + pInfo->curOffset = limit.offset; + pInfo->curSOffset = slimit.offset; + pInfo->binfo.pRes = pResBlock; + pInfo->pFilterNode = pProjPhyNode->node.pConditions; - int32_t numOfCols = num; int32_t numOfRows = 4096; size_t keyBufSize = sizeof(int64_t) + sizeof(int64_t) + POINTER_BYTES; @@ -3984,18 +3609,17 @@ SOperatorInfo* createProjectOperatorInfo(SOperatorInfo* downstream, SExprInfo* p } initResultSizeInfo(pOperator, numOfRows); - initAggInfo(&pInfo->binfo, &pInfo->aggSup, pExprInfo, numOfCols, pResBlock, keyBufSize, pTaskInfo->id.str); - setFunctionResultOutput(&pInfo->binfo, &pInfo->aggSup, MAIN_SCAN, numOfCols, pTaskInfo); + initAggInfo(&pOperator->exprSupp, &pInfo->aggSup, pExprInfo, numOfCols, keyBufSize, pTaskInfo->id.str); + initBasicInfo(&pInfo->binfo, pResBlock); + setFunctionResultOutput(pOperator, &pInfo->binfo, &pInfo->aggSup, MAIN_SCAN, numOfCols); - pInfo->pPseudoColInfo = setRowTsColumnOutputInfo(pInfo->binfo.pCtx, numOfCols); - pOperator->name = "ProjectOperator"; + pInfo->pPseudoColInfo = setRowTsColumnOutputInfo(pOperator->exprSupp.pCtx, numOfCols); + pOperator->name = "ProjectOperator"; pOperator->operatorType = QUERY_NODE_PHYSICAL_PLAN_PROJECT; - pOperator->blocking = false; - pOperator->status = OP_NOT_OPENED; - pOperator->info = pInfo; - pOperator->pExpr = pExprInfo; - pOperator->numOfExprs = num; - pOperator->pTaskInfo = pTaskInfo; + pOperator->blocking = false; + pOperator->status = OP_NOT_OPENED; + pOperator->info = pInfo; + pOperator->pTaskInfo = pTaskInfo; pOperator->fpSet = createOperatorFpSet(operatorDummyOpenFn, doProjectOperation, NULL, NULL, destroyProjectOperatorInfo, NULL, NULL, NULL); @@ -4015,6 +3639,7 @@ _error: static SSDataBlock* doApplyIndefinitFunction(SOperatorInfo* pOperator) { SIndefOperatorInfo* pIndefInfo = pOperator->info; SOptrBasicInfo* pInfo = &pIndefInfo->binfo; + SExprSupp* pSup = &pOperator->exprSupp; SSDataBlock* pRes = pInfo->pRes; blockDataCleanup(pRes); @@ -4049,18 +3674,19 @@ static SSDataBlock* doApplyIndefinitFunction(SOperatorInfo* pOperator) { } // there is an scalar expression that needs to be calculated before apply the group aggregation. - if (pIndefInfo->pScalarExpr != NULL) { - code = projectApplyFunctions(pIndefInfo->pScalarExpr, pBlock, pBlock, pIndefInfo->pScalarCtx, - pIndefInfo->numOfScalarExpr, pIndefInfo->pPseudoColInfo); + SExprSupp* pScalarSup = &pIndefInfo->scalarSup; + if (pScalarSup->pExprInfo != NULL) { + code = projectApplyFunctions(pScalarSup->pExprInfo, pBlock, pBlock, pScalarSup->pCtx, pScalarSup->numOfExprs, + pIndefInfo->pPseudoColInfo); if (code != TSDB_CODE_SUCCESS) { longjmp(pTaskInfo->env, code); } } - setInputDataBlock(pOperator, pInfo->pCtx, pBlock, order, scanFlag, false); + setInputDataBlock(pOperator, pSup->pCtx, pBlock, order, scanFlag, false); blockDataEnsureCapacity(pInfo->pRes, pInfo->pRes->info.rows + pBlock->info.rows); - code = projectApplyFunctions(pOperator->pExpr, pInfo->pRes, pBlock, pInfo->pCtx, pOperator->numOfExprs, + code = projectApplyFunctions(pOperator->exprSupp.pExprInfo, pInfo->pRes, pBlock, pSup->pCtx, pOperator->exprSupp.numOfExprs, pIndefInfo->pPseudoColInfo); if (code != TSDB_CODE_SUCCESS) { longjmp(pTaskInfo->env, code); @@ -4085,15 +3711,17 @@ SOperatorInfo* createIndefinitOutputOperatorInfo(SOperatorInfo* downstream, SPhy goto _error; } + SExprSupp* pSup = &pOperator->exprSupp; + SIndefRowsFuncPhysiNode* pPhyNode = (SIndefRowsFuncPhysiNode*)pNode; int32_t numOfExpr = 0; SExprInfo* pExprInfo = createExprInfo(pPhyNode->pVectorFuncs, NULL, &numOfExpr); - int32_t numOfScalarExpr = 0; if (pPhyNode->pExprs != NULL) { - pInfo->pScalarExpr = createExprInfo(pPhyNode->pExprs, NULL, &numOfScalarExpr); - pInfo->pScalarCtx = createSqlFunctionCtx(pInfo->pScalarExpr, numOfScalarExpr, &pInfo->rowCellInfoOffset); + SExprSupp* pSup1 = &pInfo->scalarSup; + pSup1->pExprInfo = createExprInfo(pPhyNode->pExprs, NULL, &pSup1->numOfExprs); + pSup1->pCtx = createSqlFunctionCtx(pSup1->pExprInfo, pSup1->numOfExprs, &pSup1->rowEntryInfoOffset); } SSDataBlock* pResBlock = createResDataBlock(pPhyNode->node.pOutputDataBlockDesc); @@ -4109,21 +3737,22 @@ SOperatorInfo* createIndefinitOutputOperatorInfo(SOperatorInfo* downstream, SPhy } initResultSizeInfo(pOperator, numOfRows); - initAggInfo(&pInfo->binfo, &pInfo->aggSup, pExprInfo, numOfExpr, pResBlock, keyBufSize, pTaskInfo->id.str); - setFunctionResultOutput(&pInfo->binfo, &pInfo->aggSup, MAIN_SCAN, numOfExpr, pTaskInfo); + initAggInfo(&pOperator->exprSupp, &pInfo->aggSup, pExprInfo, numOfExpr, keyBufSize, pTaskInfo->id.str); + initBasicInfo(&pInfo->binfo, pResBlock); + + setFunctionResultOutput(pOperator, &pInfo->binfo, &pInfo->aggSup, MAIN_SCAN, numOfExpr); pInfo->binfo.pRes = pResBlock; - pInfo->numOfScalarExpr = numOfScalarExpr; - pInfo->pPseudoColInfo = setRowTsColumnOutputInfo(pInfo->binfo.pCtx, numOfExpr); + pInfo->pPseudoColInfo = setRowTsColumnOutputInfo(pSup->pCtx, numOfExpr); - pOperator->name = "IndefinitOperator"; + pOperator->name = "IndefinitOperator"; pOperator->operatorType = QUERY_NODE_PHYSICAL_PLAN_PROJECT; - pOperator->blocking = false; - pOperator->status = OP_NOT_OPENED; - pOperator->info = pInfo; - pOperator->pExpr = pExprInfo; - pOperator->numOfExprs = numOfExpr; - pOperator->pTaskInfo = pTaskInfo; + pOperator->blocking = false; + pOperator->status = OP_NOT_OPENED; + pOperator->info = pInfo; + pOperator->exprSupp.pExprInfo = pExprInfo; + pOperator->exprSupp.numOfExprs = numOfExpr; + pOperator->pTaskInfo = pTaskInfo; pOperator->fpSet = createOperatorFpSet(operatorDummyOpenFn, doApplyIndefinitFunction, NULL, NULL, destroyIndefinitOperatorInfo, NULL, NULL, NULL); @@ -4162,18 +3791,9 @@ static int32_t initFillInfo(SFillOperatorInfo* pInfo, SExprInfo* pExpr, int32_t } } -SOperatorInfo* createFillOperatorInfo(SOperatorInfo* downstream, SExprInfo* pExpr, int32_t numOfCols, - SInterval* pInterval, STimeWindow* pWindow, SSDataBlock* pResBlock, - int32_t fillType, SNodeListNode* pValueNode, bool multigroupResult, - SExecTaskInfo* pTaskInfo) { - SFillOperatorInfo* pInfo = taosMemoryCalloc(1, sizeof(SFillOperatorInfo)); - SOperatorInfo* pOperator = taosMemoryCalloc(1, sizeof(SOperatorInfo)); - - pInfo->pRes = pResBlock; - pInfo->multigroupResult = multigroupResult; - +static int32_t convertFillType(int32_t mode) { int32_t type = TSDB_FILL_NONE; - switch (fillType) { + switch (mode) { case FILL_MODE_PREV: type = TSDB_FILL_PREV; break; @@ -4196,26 +3816,46 @@ SOperatorInfo* createFillOperatorInfo(SOperatorInfo* downstream, SExprInfo* pExp type = TSDB_FILL_NONE; } + return type; +} + +SOperatorInfo* createFillOperatorInfo(SOperatorInfo* downstream, SFillPhysiNode* pPhyFillNode, bool multigroupResult, + SExecTaskInfo* pTaskInfo) { + SFillOperatorInfo* pInfo = taosMemoryCalloc(1, sizeof(SFillOperatorInfo)); + SOperatorInfo* pOperator = taosMemoryCalloc(1, sizeof(SOperatorInfo)); + if (pInfo == NULL || pOperator == NULL) { + goto _error; + } + + int32_t num = 0; + SSDataBlock* pResBlock = createResDataBlock(pPhyFillNode->node.pOutputDataBlockDesc); + SExprInfo* pExprInfo = createExprInfo(pPhyFillNode->pTargets, NULL, &num); + SInterval* pInterval = &((SIntervalAggOperatorInfo*)downstream->info)->interval; + int32_t type = convertFillType(pPhyFillNode->mode); + SResultInfo* pResultInfo = &pOperator->resultInfo; initResultSizeInfo(pOperator, 4096); - int32_t code = initFillInfo(pInfo, pExpr, numOfCols, pValueNode, *pWindow, pResultInfo->capacity, pTaskInfo->id.str, - pInterval, type); + int32_t code = initFillInfo(pInfo, pExprInfo, num, (SNodeListNode*)pPhyFillNode->pValues, pPhyFillNode->timeRange, + pResultInfo->capacity, pTaskInfo->id.str, pInterval, type); if (code != TSDB_CODE_SUCCESS) { goto _error; } - pOperator->name = "FillOperator"; - pOperator->blocking = false; - pOperator->status = OP_NOT_OPENED; + pInfo->pRes = pResBlock; + pInfo->multigroupResult = multigroupResult; + pOperator->name = "FillOperator"; + pOperator->blocking = false; + pOperator->status = OP_NOT_OPENED; pOperator->operatorType = QUERY_NODE_PHYSICAL_PLAN_FILL; - pOperator->pExpr = pExpr; - pOperator->numOfExprs = numOfCols; - pOperator->info = pInfo; + pOperator->exprSupp.pExprInfo = pExprInfo; + pOperator->exprSupp.numOfExprs = num; + pOperator->info = pInfo; + pOperator->pTaskInfo = pTaskInfo; pOperator->fpSet = createOperatorFpSet(operatorDummyOpenFn, doFill, NULL, NULL, destroySFillOperatorInfo, NULL, NULL, NULL); - pOperator->pTaskInfo = pTaskInfo; + code = appendDownstream(pOperator, &downstream, 1); return pOperator; @@ -4225,151 +3865,6 @@ _error: return NULL; } -static SResSchema createResSchema(int32_t type, int32_t bytes, int32_t slotId, int32_t scale, int32_t precision, - const char* name) { - SResSchema s = {0}; - s.scale = scale; - s.type = type; - s.bytes = bytes; - s.slotId = slotId; - s.precision = precision; - strncpy(s.name, name, tListLen(s.name)); - - return s; -} - -static SColumn* createColumn(int32_t blockId, int32_t slotId, int32_t colId, SDataType* pType) { - SColumn* pCol = taosMemoryCalloc(1, sizeof(SColumn)); - if (pCol == NULL) { - terrno = TSDB_CODE_OUT_OF_MEMORY; - return NULL; - } - - pCol->slotId = slotId; - pCol->colId = colId; - pCol->bytes = pType->bytes; - pCol->type = pType->type; - pCol->scale = pType->scale; - pCol->precision = pType->precision; - pCol->dataBlockId = blockId; - - return pCol; -} - -SExprInfo* createExprInfo(SNodeList* pNodeList, SNodeList* pGroupKeys, int32_t* numOfExprs) { - int32_t numOfFuncs = LIST_LENGTH(pNodeList); - int32_t numOfGroupKeys = 0; - if (pGroupKeys != NULL) { - numOfGroupKeys = LIST_LENGTH(pGroupKeys); - } - - *numOfExprs = numOfFuncs + numOfGroupKeys; - SExprInfo* pExprs = taosMemoryCalloc(*numOfExprs, sizeof(SExprInfo)); - - for (int32_t i = 0; i < (*numOfExprs); ++i) { - STargetNode* pTargetNode = NULL; - if (i < numOfFuncs) { - pTargetNode = (STargetNode*)nodesListGetNode(pNodeList, i); - } else { - pTargetNode = (STargetNode*)nodesListGetNode(pGroupKeys, i - numOfFuncs); - } - - SExprInfo* pExp = &pExprs[i]; - - pExp->pExpr = taosMemoryCalloc(1, sizeof(tExprNode)); - pExp->pExpr->_function.num = 1; - pExp->pExpr->_function.functionId = -1; - - int32_t type = nodeType(pTargetNode->pExpr); - // it is a project query, or group by column - if (type == QUERY_NODE_COLUMN) { - pExp->pExpr->nodeType = QUERY_NODE_COLUMN; - SColumnNode* pColNode = (SColumnNode*)pTargetNode->pExpr; - - pExp->base.pParam = taosMemoryCalloc(1, sizeof(SFunctParam)); - pExp->base.numOfParams = 1; - - SDataType* pType = &pColNode->node.resType; - pExp->base.resSchema = createResSchema(pType->type, pType->bytes, pTargetNode->slotId, pType->scale, - pType->precision, pColNode->colName); - pExp->base.pParam[0].pCol = createColumn(pColNode->dataBlockId, pColNode->slotId, pColNode->colId, pType); - pExp->base.pParam[0].type = FUNC_PARAM_TYPE_COLUMN; - } else if (type == QUERY_NODE_VALUE) { - pExp->pExpr->nodeType = QUERY_NODE_VALUE; - SValueNode* pValNode = (SValueNode*)pTargetNode->pExpr; - - pExp->base.pParam = taosMemoryCalloc(1, sizeof(SFunctParam)); - pExp->base.numOfParams = 1; - - SDataType* pType = &pValNode->node.resType; - pExp->base.resSchema = createResSchema(pType->type, pType->bytes, pTargetNode->slotId, pType->scale, - pType->precision, pValNode->node.aliasName); - pExp->base.pParam[0].type = FUNC_PARAM_TYPE_VALUE; - nodesValueNodeToVariant(pValNode, &pExp->base.pParam[0].param); - } else if (type == QUERY_NODE_FUNCTION) { - pExp->pExpr->nodeType = QUERY_NODE_FUNCTION; - SFunctionNode* pFuncNode = (SFunctionNode*)pTargetNode->pExpr; - - SDataType* pType = &pFuncNode->node.resType; - pExp->base.resSchema = createResSchema(pType->type, pType->bytes, pTargetNode->slotId, pType->scale, - pType->precision, pFuncNode->node.aliasName); - - pExp->pExpr->_function.functionId = pFuncNode->funcId; - pExp->pExpr->_function.pFunctNode = pFuncNode; - - strncpy(pExp->pExpr->_function.functionName, pFuncNode->functionName, - tListLen(pExp->pExpr->_function.functionName)); -#if 1 - // todo refactor: add the parameter for tbname function - if (strcmp(pExp->pExpr->_function.functionName, "tbname") == 0) { - pFuncNode->pParameterList = nodesMakeList(); - ASSERT(LIST_LENGTH(pFuncNode->pParameterList) == 0); - SValueNode* res = (SValueNode*)nodesMakeNode(QUERY_NODE_VALUE); - if (NULL == res) { // todo handle error - } else { - res->node.resType = (SDataType){.bytes = sizeof(int64_t), .type = TSDB_DATA_TYPE_BIGINT}; - nodesListAppend(pFuncNode->pParameterList, (SNode*)res); - } - } -#endif - - int32_t numOfParam = LIST_LENGTH(pFuncNode->pParameterList); - - pExp->base.pParam = taosMemoryCalloc(numOfParam, sizeof(SFunctParam)); - pExp->base.numOfParams = numOfParam; - - for (int32_t j = 0; j < numOfParam; ++j) { - SNode* p1 = nodesListGetNode(pFuncNode->pParameterList, j); - if (p1->type == QUERY_NODE_COLUMN) { - SColumnNode* pcn = (SColumnNode*)p1; - - pExp->base.pParam[j].type = FUNC_PARAM_TYPE_COLUMN; - pExp->base.pParam[j].pCol = createColumn(pcn->dataBlockId, pcn->slotId, pcn->colId, &pcn->node.resType); - } else if (p1->type == QUERY_NODE_VALUE) { - SValueNode* pvn = (SValueNode*)p1; - pExp->base.pParam[j].type = FUNC_PARAM_TYPE_VALUE; - nodesValueNodeToVariant(pvn, &pExp->base.pParam[j].param); - } - } - } else if (type == QUERY_NODE_OPERATOR) { - pExp->pExpr->nodeType = QUERY_NODE_OPERATOR; - SOperatorNode* pNode = (SOperatorNode*)pTargetNode->pExpr; - - pExp->base.pParam = taosMemoryCalloc(1, sizeof(SFunctParam)); - pExp->base.numOfParams = 1; - - SDataType* pType = &pNode->node.resType; - pExp->base.resSchema = createResSchema(pType->type, pType->bytes, pTargetNode->slotId, pType->scale, - pType->precision, pNode->node.aliasName); - pExp->pExpr->_optrRoot.pRootNode = pTargetNode->pExpr; - } else { - ASSERT(0); - } - } - - return pExprs; -} - static SExecTaskInfo* createExecTaskInfo(uint64_t queryId, uint64_t taskId, EOPTR_EXEC_MODEL model, char* dbFName) { SExecTaskInfo* pTaskInfo = taosMemoryCalloc(1, sizeof(SExecTaskInfo)); setTaskStatus(pTaskInfo, TASK_NOT_COMPLETED); @@ -4392,8 +3887,6 @@ static tsdbReaderT doCreateDataReader(STableScanPhysiNode* pTableScanNode, SRead static SArray* extractColumnInfo(SNodeList* pNodeList); -static SArray* createSortInfo(SNodeList* pNodeList); - int32_t extractTableSchemaVersion(SReadHandle* pHandle, uint64_t uid, SExecTaskInfo* pTaskInfo) { SMetaReader mr = {0}; metaReaderInit(&mr, pHandle->meta, 0); @@ -4560,7 +4053,7 @@ SOperatorInfo* createOperatorTree(SPhysiNode* pPhyNode, SExecTaskInfo* pTaskInfo if (pHandle->vnode) { pDataReader = doCreateDataReader(pTableScanNode, pHandle, pTableListInfo, (uint64_t)queryId, taskId, pTagCond); } else { - getTableList(pHandle->meta, pScanPhyNode->tableType, pScanPhyNode->uid, pTableListInfo, pTagCond); + getTableList(pHandle->meta, pScanPhyNode, pTableListInfo, pTagCond); } if (pDataReader == NULL && terrno != 0) { @@ -4587,9 +4080,9 @@ SOperatorInfo* createOperatorTree(SPhysiNode* pPhyNode, SExecTaskInfo* pTaskInfo } else if (QUERY_NODE_PHYSICAL_PLAN_TAG_SCAN == type) { STagScanPhysiNode* pScanPhyNode = (STagScanPhysiNode*)pPhyNode; - int32_t code = getTableList(pHandle->meta, pScanPhyNode->tableType, pScanPhyNode->uid, pTableListInfo, - pScanPhyNode->node.pConditions); + int32_t code = getTableList(pHandle->meta, pScanPhyNode, pTableListInfo, pScanPhyNode->node.pConditions); if (code != TSDB_CODE_SUCCESS) { + pTaskInfo->code = terrno; return NULL; } @@ -4613,14 +4106,7 @@ SOperatorInfo* createOperatorTree(SPhysiNode* pPhyNode, SExecTaskInfo* pTaskInfo SOperatorInfo* pOptr = NULL; if (QUERY_NODE_PHYSICAL_PLAN_PROJECT == type) { - SProjectPhysiNode* pProjPhyNode = (SProjectPhysiNode*)pPhyNode; - SExprInfo* pExprInfo = createExprInfo(pProjPhyNode->pProjections, NULL, &num); - - SSDataBlock* pResBlock = createResDataBlock(pPhyNode->pOutputDataBlockDesc); - SLimit limit = {.limit = pProjPhyNode->limit, .offset = pProjPhyNode->offset}; - SLimit slimit = {.limit = pProjPhyNode->slimit, .offset = pProjPhyNode->soffset}; - pOptr = createProjectOperatorInfo(ops[0], pExprInfo, num, pResBlock, &limit, &slimit, - pProjPhyNode->node.pConditions, pTaskInfo); + pOptr = createProjectOperatorInfo(ops[0], (SProjectPhysiNode*)pPhyNode, pTaskInfo); } else if (QUERY_NODE_PHYSICAL_PLAN_HASH_AGG == type) { SAggPhysiNode* pAggNode = (SAggPhysiNode*)pPhyNode; SExprInfo* pExprInfo = createExprInfo(pAggNode->pAggFuncs, pAggNode->pGroupKeys, &num); @@ -4687,21 +4173,7 @@ SOperatorInfo* createOperatorTree(SPhysiNode* pPhyNode, SExecTaskInfo* pTaskInfo int32_t children = 1; pOptr = createStreamFinalIntervalOperatorInfo(ops[0], pPhyNode, pTaskInfo, children); } else if (QUERY_NODE_PHYSICAL_PLAN_SORT == type) { - SSortPhysiNode* pSortPhyNode = (SSortPhysiNode*)pPhyNode; - - SDataBlockDescNode* pDescNode = pPhyNode->pOutputDataBlockDesc; - - SSDataBlock* pResBlock = createResDataBlock(pDescNode); - SArray* info = createSortInfo(pSortPhyNode->pSortKeys); - - int32_t numOfCols = 0; - SExprInfo* pExprInfo = createExprInfo(pSortPhyNode->pExprs, NULL, &numOfCols); - - int32_t numOfOutputCols = 0; - SArray* pColList = - extractColMatchInfo(pSortPhyNode->pTargets, pDescNode, &numOfOutputCols, pTaskInfo, COL_MATCH_FROM_SLOT_ID); - - pOptr = createSortOperatorInfo(ops[0], pResBlock, info, pExprInfo, numOfCols, pColList, pTaskInfo); + pOptr = createSortOperatorInfo(ops[0], (SSortPhysiNode*)pPhyNode, pTaskInfo); } else if (QUERY_NODE_PHYSICAL_PLAN_MERGE == type) { SMergePhysiNode* pMergePhyNode = (SMergePhysiNode*)pPhyNode; @@ -4711,7 +4183,7 @@ SOperatorInfo* createOperatorTree(SPhysiNode* pPhyNode, SExecTaskInfo* pTaskInfo SArray* sortInfo = createSortInfo(pMergePhyNode->pMergeKeys); int32_t numOfOutputCols = 0; SArray* pColList = - extractColMatchInfo(pMergePhyNode->pTargets, pDescNode, &numOfOutputCols, pTaskInfo, COL_MATCH_FROM_SLOT_ID); + extractColMatchInfo(pMergePhyNode->pTargets, pDescNode, &numOfOutputCols, COL_MATCH_FROM_SLOT_ID); SPhysiNode* pChildNode = (SPhysiNode*)nodesListGetNode(pPhyNode->pChildren, 0); SSDataBlock* pInputDataBlock = createResDataBlock(pChildNode->pOutputDataBlockDesc); pOptr = createMultiwaySortMergeOperatorInfo(ops, size, pInputDataBlock, pResBlock, sortInfo, pColList, pTaskInfo); @@ -4728,18 +4200,13 @@ SOperatorInfo* createOperatorTree(SPhysiNode* pPhyNode, SExecTaskInfo* pTaskInfo pOptr = createSessionAggOperatorInfo(ops[0], pExprInfo, num, pResBlock, pSessionNode->gap, tsSlotId, &as, pTaskInfo); } else if (QUERY_NODE_PHYSICAL_PLAN_STREAM_SESSION == type) { - SSessionWinodwPhysiNode* pSessionNode = (SSessionWinodwPhysiNode*)pPhyNode; - - STimeWindowAggSupp as = {.waterMark = pSessionNode->window.watermark, - .calTrigger = pSessionNode->window.triggerType}; - - SExprInfo* pExprInfo = createExprInfo(pSessionNode->window.pFuncs, NULL, &num); - SSDataBlock* pResBlock = createResDataBlock(pPhyNode->pOutputDataBlockDesc); - int32_t tsSlotId = ((SColumnNode*)pSessionNode->window.pTspk)->slotId; - - pOptr = createStreamSessionAggOperatorInfo(ops[0], pExprInfo, num, pResBlock, pSessionNode->gap, tsSlotId, &as, - pTaskInfo); - + pOptr = createStreamSessionAggOperatorInfo(ops[0], pPhyNode, pTaskInfo); + } else if (QUERY_NODE_PHYSICAL_PLAN_STREAM_SEMI_SESSION == type) { + int32_t children = 0; + pOptr = createStreamFinalSessionAggOperatorInfo(ops[0], pPhyNode, pTaskInfo, children); + } else if (QUERY_NODE_PHYSICAL_PLAN_STREAM_FINAL_SESSION == type) { + int32_t children = 1; + pOptr = createStreamFinalSessionAggOperatorInfo(ops[0], pPhyNode, pTaskInfo, children); } else if (QUERY_NODE_PHYSICAL_PLAN_PARTITION == type) { pOptr = createPartitionOperatorInfo(ops[0], (SPartitionPhysiNode*)pPhyNode, pTaskInfo); } else if (QUERY_NODE_PHYSICAL_PLAN_MERGE_STATE == type) { @@ -4757,19 +4224,9 @@ SOperatorInfo* createOperatorTree(SPhysiNode* pPhyNode, SExecTaskInfo* pTaskInfo } else if (QUERY_NODE_PHYSICAL_PLAN_STREAM_STATE == type) { pOptr = createStreamStateAggOperatorInfo(ops[0], pPhyNode, pTaskInfo); } else if (QUERY_NODE_PHYSICAL_PLAN_MERGE_JOIN == type) { - SJoinPhysiNode* pJoinNode = (SJoinPhysiNode*)pPhyNode; - SSDataBlock* pResBlock = createResDataBlock(pPhyNode->pOutputDataBlockDesc); - - SExprInfo* pExprInfo = createExprInfo(pJoinNode->pTargets, NULL, &num); - pOptr = createMergeJoinOperatorInfo(ops, size, pExprInfo, num, pResBlock, pJoinNode->pOnConditions, pTaskInfo); + pOptr = createMergeJoinOperatorInfo(ops, size, (SJoinPhysiNode*)pPhyNode, pTaskInfo); } else if (QUERY_NODE_PHYSICAL_PLAN_FILL == type) { - SFillPhysiNode* pFillNode = (SFillPhysiNode*)pPhyNode; - SSDataBlock* pResBlock = createResDataBlock(pPhyNode->pOutputDataBlockDesc); - SExprInfo* pExprInfo = createExprInfo(pFillNode->pTargets, NULL, &num); - - SInterval* pInterval = &((SIntervalAggOperatorInfo*)ops[0]->info)->interval; - pOptr = createFillOperatorInfo(ops[0], pExprInfo, num, pInterval, &pFillNode->timeRange, pResBlock, pFillNode->mode, - (SNodeListNode*)pFillNode->pValues, false, pTaskInfo); + pOptr = createFillOperatorInfo(ops[0], (SFillPhysiNode*)pPhyNode, false, pTaskInfo); } else if (QUERY_NODE_PHYSICAL_PLAN_INDEF_ROWS_FUNC == type) { pOptr = createIndefinitOutputOperatorInfo(ops[0], pPhyNode, pTaskInfo); } else { @@ -4792,79 +4249,6 @@ int32_t compareTimeWindow(const void* p1, const void* p2, const void* param) { return 0; } -int32_t initQueryTableDataCond(SQueryTableDataCond* pCond, const STableScanPhysiNode* pTableScanNode) { - pCond->loadExternalRows = false; - - pCond->order = pTableScanNode->scanSeq[0] > 0 ? TSDB_ORDER_ASC : TSDB_ORDER_DESC; - pCond->numOfCols = LIST_LENGTH(pTableScanNode->scan.pScanCols); - pCond->colList = taosMemoryCalloc(pCond->numOfCols, sizeof(SColumnInfo)); - if (pCond->colList == NULL) { - terrno = TSDB_CODE_QRY_OUT_OF_MEMORY; - return terrno; - } - - // pCond->twindow = pTableScanNode->scanRange; - // TODO: get it from stable scan node - pCond->numOfTWindows = 1; - pCond->twindows = taosMemoryCalloc(pCond->numOfTWindows, sizeof(STimeWindow)); - pCond->twindows[0] = pTableScanNode->scanRange; - pCond->suid = pTableScanNode->scan.suid; - -#if 1 - // todo work around a problem, remove it later - for (int32_t i = 0; i < pCond->numOfTWindows; ++i) { - if ((pCond->order == TSDB_ORDER_ASC && pCond->twindows[i].skey > pCond->twindows[i].ekey) || - (pCond->order == TSDB_ORDER_DESC && pCond->twindows[i].skey < pCond->twindows[i].ekey)) { - TSWAP(pCond->twindows[i].skey, pCond->twindows[i].ekey); - } - } -#endif - - for (int32_t i = 0; i < pCond->numOfTWindows; ++i) { - if ((pCond->order == TSDB_ORDER_ASC && pCond->twindows[i].skey > pCond->twindows[i].ekey) || - (pCond->order == TSDB_ORDER_DESC && pCond->twindows[i].skey < pCond->twindows[i].ekey)) { - TSWAP(pCond->twindows[i].skey, pCond->twindows[i].ekey); - } - } - taosqsort(pCond->twindows, pCond->numOfTWindows, sizeof(STimeWindow), pCond, compareTimeWindow); - - pCond->type = BLOCK_LOAD_OFFSET_SEQ_ORDER; - // pCond->type = pTableScanNode->scanFlag; - - int32_t j = 0; - for (int32_t i = 0; i < pCond->numOfCols; ++i) { - STargetNode* pNode = (STargetNode*)nodesListGetNode(pTableScanNode->scan.pScanCols, i); - SColumnNode* pColNode = (SColumnNode*)pNode->pExpr; - if (pColNode->colType == COLUMN_TYPE_TAG) { - continue; - } - - pCond->colList[j].type = pColNode->node.resType.type; - pCond->colList[j].bytes = pColNode->node.resType.bytes; - pCond->colList[j].colId = pColNode->colId; - j += 1; - } - - pCond->numOfCols = j; - return TSDB_CODE_SUCCESS; -} - -void clearupQueryTableDataCond(SQueryTableDataCond* pCond) { - taosMemoryFree(pCond->twindows); - taosMemoryFree(pCond->colList); -} - -SColumn extractColumnFromColumnNode(SColumnNode* pColNode) { - SColumn c = {0}; - c.slotId = pColNode->slotId; - c.colId = pColNode->colId; - c.type = pColNode->node.resType.type; - c.bytes = pColNode->node.resType.bytes; - c.scale = pColNode->node.resType.scale; - c.precision = pColNode->node.resType.precision; - return c; -} - SArray* extractColumnInfo(SNodeList* pNodeList) { size_t numOfCols = LIST_LENGTH(pNodeList); SArray* pList = taosArrayInit(numOfCols, sizeof(SColumn)); @@ -4886,7 +4270,7 @@ SArray* extractColumnInfo(SNodeList* pNodeList) { SColumn c = {0}; c.slotId = pNode->slotId; c.colId = pNode->slotId; - c.type = pValNode->node.type; + c.type = pValNode->node.type; c.bytes = pValNode->node.resType.bytes; c.scale = pValNode->node.resType.scale; c.precision = pValNode->node.resType.precision; @@ -4898,146 +4282,10 @@ SArray* extractColumnInfo(SNodeList* pNodeList) { return pList; } -SArray* extractPartitionColInfo(SNodeList* pNodeList) { - if(!pNodeList) { - return NULL; - } - - size_t numOfCols = LIST_LENGTH(pNodeList); - SArray* pList = taosArrayInit(numOfCols, sizeof(SColumn)); - if (pList == NULL) { - terrno = TSDB_CODE_OUT_OF_MEMORY; - return NULL; - } - - for (int32_t i = 0; i < numOfCols; ++i) { - SColumnNode* pColNode = (SColumnNode*)nodesListGetNode(pNodeList, i); - - // todo extract method - SColumn c = {0}; - c.slotId = pColNode->slotId; - c.colId = pColNode->colId; - c.type = pColNode->node.resType.type; - c.bytes = pColNode->node.resType.bytes; - c.precision = pColNode->node.resType.precision; - c.scale = pColNode->node.resType.scale; - - taosArrayPush(pList, &c); - } - - return pList; -} - -SArray* createSortInfo(SNodeList* pNodeList) { - size_t numOfCols = LIST_LENGTH(pNodeList); - SArray* pList = taosArrayInit(numOfCols, sizeof(SBlockOrderInfo)); - if (pList == NULL) { - terrno = TSDB_CODE_OUT_OF_MEMORY; - return pList; - } - - for (int32_t i = 0; i < numOfCols; ++i) { - SOrderByExprNode* pSortKey = (SOrderByExprNode*)nodesListGetNode(pNodeList, i); - SBlockOrderInfo bi = {0}; - bi.order = (pSortKey->order == ORDER_ASC) ? TSDB_ORDER_ASC : TSDB_ORDER_DESC; - bi.nullFirst = (pSortKey->nullOrder == NULL_ORDER_FIRST); - - SColumnNode* pColNode = (SColumnNode*)pSortKey->pExpr; - bi.slotId = pColNode->slotId; - taosArrayPush(pList, &bi); - } - - return pList; -} - -SArray* extractColMatchInfo(SNodeList* pNodeList, SDataBlockDescNode* pOutputNodeList, int32_t* numOfOutputCols, - SExecTaskInfo* pTaskInfo, int32_t type) { - size_t numOfCols = LIST_LENGTH(pNodeList); - SArray* pList = taosArrayInit(numOfCols, sizeof(SColMatchInfo)); - if (pList == NULL) { - terrno = TSDB_CODE_OUT_OF_MEMORY; - return NULL; - } - - for (int32_t i = 0; i < numOfCols; ++i) { - STargetNode* pNode = (STargetNode*)nodesListGetNode(pNodeList, i); - SColumnNode* pColNode = (SColumnNode*)pNode->pExpr; - - SColMatchInfo c = {0}; - c.output = true; - c.colId = pColNode->colId; - c.srcSlotId = pColNode->slotId; - c.matchType = type; - c.targetSlotId = pNode->slotId; - taosArrayPush(pList, &c); - } - - *numOfOutputCols = 0; - int32_t num = LIST_LENGTH(pOutputNodeList->pSlots); - for (int32_t i = 0; i < num; ++i) { - SSlotDescNode* pNode = (SSlotDescNode*)nodesListGetNode(pOutputNodeList->pSlots, i); - - // todo: add reserve flag check - // it is a column reserved for the arithmetic expression calculation - if (pNode->slotId >= numOfCols) { - (*numOfOutputCols) += 1; - continue; - } - - SColMatchInfo* info = taosArrayGet(pList, pNode->slotId); - if (pNode->output) { - (*numOfOutputCols) += 1; - } else { - info->output = false; - } - } - - return pList; -} - -int32_t getTableList(void* metaHandle, int32_t tableType, uint64_t tableUid, STableListInfo* pListInfo, - SNode* pTagCond) { - int32_t code = TSDB_CODE_SUCCESS; - pListInfo->pTableList = taosArrayInit(8, sizeof(STableKeyInfo)); - - if (tableType == TSDB_SUPER_TABLE) { - if (pTagCond) { - SIndexMetaArg metaArg = { - .metaEx = metaHandle, .idx = tsdbGetIdx(metaHandle), .ivtIdx = tsdbGetIvtIdx(metaHandle), .suid = tableUid}; - - SArray* res = taosArrayInit(8, sizeof(uint64_t)); - code = doFilterTag(pTagCond, &metaArg, res); - if (code == TSDB_CODE_INDEX_REBUILDING) { // todo - // doFilter(); - } else if (code != TSDB_CODE_SUCCESS) { - qError("failed to get tableIds, reason: %s, suid: %" PRIu64 "", tstrerror(code), tableUid); - taosArrayDestroy(res); - terrno = code; - return code; - } else { - qDebug("sucess to get tableIds, size: %d, suid: %" PRIu64 "", (int)taosArrayGetSize(res), tableUid); - } - - for (int i = 0; i < taosArrayGetSize(res); i++) { - STableKeyInfo info = {.lastKey = TSKEY_INITIAL_VAL, .uid = *(uint64_t*)taosArrayGet(res, i)}; - taosArrayPush(pListInfo->pTableList, &info); - } - taosArrayDestroy(res); - } else { - code = tsdbGetAllTableList(metaHandle, tableUid, pListInfo->pTableList); - } - } else { // Create one table group. - STableKeyInfo info = {.lastKey = 0, .uid = tableUid}; - taosArrayPush(pListInfo->pTableList, &info); - } - - return code; -} - tsdbReaderT doCreateDataReader(STableScanPhysiNode* pTableScanNode, SReadHandle* pHandle, STableListInfo* pTableListInfo, uint64_t queryId, uint64_t taskId, SNode* pTagCond) { int32_t code = - getTableList(pHandle->meta, pTableScanNode->scan.tableType, pTableScanNode->scan.uid, pTableListInfo, pTagCond); + getTableList(pHandle->meta, &pTableScanNode->scan, pTableListInfo, pTagCond); if (code != TSDB_CODE_SUCCESS) { goto _error; } @@ -5055,7 +4303,7 @@ tsdbReaderT doCreateDataReader(STableScanPhysiNode* pTableScanNode, SReadHandle* } tsdbReaderT* pReader = tsdbReaderOpen(pHandle->vnode, &cond, pTableListInfo, queryId, taskId); - clearupQueryTableDataCond(&cond); + cleanupQueryTableDataCond(&cond); return pReader; @@ -5116,15 +4364,17 @@ int32_t encodeOperator(SOperatorInfo* ops, char** result, int32_t* length) { return TDB_CODE_SUCCESS; } -int32_t decodeOperator(SOperatorInfo* ops, char* result, int32_t length) { +int32_t decodeOperator(SOperatorInfo* ops, const char* result, int32_t length) { int32_t code = TDB_CODE_SUCCESS; if (ops->fpSet.decodeResultRow) { if (result == NULL) { return TSDB_CODE_TSC_INVALID_INPUT; } + ASSERT(length == *(int32_t*)result); - char* data = result + sizeof(int32_t); - code = ops->fpSet.decodeResultRow(ops, data); + + const char* data = result + sizeof(int32_t); + code = ops->fpSet.decodeResultRow(ops, (char*) data); if (code != TDB_CODE_SUCCESS) { return code; } diff --git a/source/libs/executor/src/groupoperator.c b/source/libs/executor/src/groupoperator.c index 4fc25688c467effa96da8035a5225be07ce39198..527f4520bf72c06e1cceaf2d6e7f9105d3acd2e9 100644 --- a/source/libs/executor/src/groupoperator.c +++ b/source/libs/executor/src/groupoperator.c @@ -26,15 +26,18 @@ #include "ttypes.h" #include "executorInt.h" +static void* getCurrentDataGroupInfo(const SPartitionOperatorInfo* pInfo, SDataGroupInfo** pGroupInfo, int32_t len); static int32_t* setupColumnOffset(const SSDataBlock* pBlock, int32_t rowCapacity); -static void* getCurrentDataGroupInfo(const SPartitionOperatorInfo* pInfo, SDataGroupInfo** pGroupInfo, int32_t len); +static int32_t setGroupResultOutputBuf(SOperatorInfo* pOperator, SOptrBasicInfo* binfo, int32_t numOfCols, char* pData, int16_t bytes, + int32_t groupId, SDiskbasedBuf* pBuf, SAggSupporter* pAggSup); static void destroyGroupOperatorInfo(void* param, int32_t numOfOutput) { SGroupbyOperatorInfo* pInfo = (SGroupbyOperatorInfo*)param; - doDestroyBasicInfo(&pInfo->binfo, numOfOutput); + cleanupBasicInfo(&pInfo->binfo); taosMemoryFreeClear(pInfo->keyBuf); taosArrayDestroy(pInfo->pGroupCols); taosArrayDestroy(pInfo->pGroupColVals); + cleanupExecSupp(&pInfo->scalarSup); } static int32_t initGroupOptrInfo(SArray** pGroupColVals, int32_t* keyLen, char** keyBuf, const SArray* pGroupColList) { @@ -214,7 +217,7 @@ static void doHashGroupbyAgg(SOperatorInfo* pOperator, SSDataBlock* pBlock) { SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo; SGroupbyOperatorInfo* pInfo = pOperator->info; - SqlFunctionCtx* pCtx = pInfo->binfo.pCtx; + SqlFunctionCtx* pCtx = pOperator->exprSupp.pCtx; int32_t numOfGroupCols = taosArrayGetSize(pInfo->pGroupCols); // if (type == TSDB_DATA_TYPE_FLOAT || type == TSDB_DATA_TYPE_DOUBLE) { // qError("QInfo:0x%"PRIx64" group by not supported on double/float columns, abort", GET_TASKID(pRuntimeEnv)); @@ -248,16 +251,16 @@ static void doHashGroupbyAgg(SOperatorInfo* pOperator, SSDataBlock* pBlock) { } len = buildGroupKeys(pInfo->keyBuf, pInfo->pGroupColVals); - int32_t ret = setGroupResultOutputBuf(&(pInfo->binfo), pOperator->numOfExprs, pInfo->keyBuf, TSDB_DATA_TYPE_VARCHAR, len, 0, pInfo->aggSup.pResultBuf, pTaskInfo, &pInfo->aggSup); + int32_t ret = setGroupResultOutputBuf(pOperator, &(pInfo->binfo), pOperator->exprSupp.numOfExprs, pInfo->keyBuf, len, 0, pInfo->aggSup.pResultBuf, &pInfo->aggSup); if (ret != TSDB_CODE_SUCCESS) { // null data, too many state code longjmp(pTaskInfo->env, TSDB_CODE_QRY_APP_ERROR); } int32_t rowIndex = j - num; - doApplyFunctions(pTaskInfo, pCtx, &w, NULL, rowIndex, num, NULL, pBlock->info.rows, pOperator->numOfExprs, TSDB_ORDER_ASC); + doApplyFunctions(pTaskInfo, pCtx, &w, NULL, rowIndex, num, NULL, pBlock->info.rows, pOperator->exprSupp.numOfExprs, TSDB_ORDER_ASC); // assign the group keys or user input constant values if required - doAssignGroupKeys(pCtx, pOperator->numOfExprs, pBlock->info.rows, rowIndex); + doAssignGroupKeys(pCtx, pOperator->exprSupp.numOfExprs, pBlock->info.rows, rowIndex); recordNewGroupKeys(pInfo->pGroupCols, pInfo->pGroupColVals, pBlock, j); num = 1; } @@ -265,15 +268,15 @@ static void doHashGroupbyAgg(SOperatorInfo* pOperator, SSDataBlock* pBlock) { if (num > 0) { len = buildGroupKeys(pInfo->keyBuf, pInfo->pGroupColVals); int32_t ret = - setGroupResultOutputBuf(&(pInfo->binfo), pOperator->numOfExprs, pInfo->keyBuf, TSDB_DATA_TYPE_VARCHAR, len, - 0, pInfo->aggSup.pResultBuf, pTaskInfo, &pInfo->aggSup); + setGroupResultOutputBuf(pOperator, &(pInfo->binfo), pOperator->exprSupp.numOfExprs, pInfo->keyBuf, len, + 0, pInfo->aggSup.pResultBuf, &pInfo->aggSup); if (ret != TSDB_CODE_SUCCESS) { longjmp(pTaskInfo->env, TSDB_CODE_QRY_APP_ERROR); } int32_t rowIndex = pBlock->info.rows - num; - doApplyFunctions(pTaskInfo, pCtx, &w, NULL, rowIndex, num, NULL, pBlock->info.rows, pOperator->numOfExprs, TSDB_ORDER_ASC); - doAssignGroupKeys(pCtx, pOperator->numOfExprs, pBlock->info.rows, rowIndex); + doApplyFunctions(pTaskInfo, pCtx, &w, NULL, rowIndex, num, NULL, pBlock->info.rows, pOperator->exprSupp.numOfExprs, TSDB_ORDER_ASC); + doAssignGroupKeys(pCtx, pOperator->exprSupp.numOfExprs, pBlock->info.rows, rowIndex); } } @@ -291,7 +294,7 @@ static SSDataBlock* hashGroupbyAggregate(SOperatorInfo* pOperator) { doBuildResultDatablock(pOperator, &pInfo->binfo, &pInfo->groupResInfo, pInfo->aggSup.pResultBuf); size_t rows = pRes->info.rows; - if (rows == 0 || !hashRemainDataInGroupInfo(&pInfo->groupResInfo)) { + if (rows == 0 || !hasDataInGroupInfo(&pInfo->groupResInfo)) { doSetOperatorCompleted(pOperator); } @@ -317,11 +320,11 @@ static SSDataBlock* hashGroupbyAggregate(SOperatorInfo* pOperator) { } // the pDataBlock are always the same one, no need to call this again - setInputDataBlock(pOperator, pInfo->binfo.pCtx, pBlock, order, scanFlag, true); + setInputDataBlock(pOperator, pOperator->exprSupp.pCtx, pBlock, order, scanFlag, true); // there is an scalar expression that needs to be calculated right before apply the group aggregation. - if (pInfo->scalarSup.pScalarExprInfo != NULL) { - pTaskInfo->code = projectApplyFunctions(pInfo->scalarSup.pScalarExprInfo, pBlock, pBlock, pInfo->scalarSup.pScalarFuncCtx, pInfo->scalarSup.numOfScalarExpr, NULL); + if (pInfo->scalarSup.pExprInfo != NULL) { + pTaskInfo->code = projectApplyFunctions(pInfo->scalarSup.pExprInfo, pBlock, pBlock, pInfo->scalarSup.pCtx, pInfo->scalarSup.numOfExprs, NULL); if (pTaskInfo->code != TSDB_CODE_SUCCESS) { longjmp(pTaskInfo->env, pTaskInfo->code); } @@ -355,7 +358,7 @@ static SSDataBlock* hashGroupbyAggregate(SOperatorInfo* pOperator) { doBuildResultDatablock(pOperator, &pInfo->binfo, &pInfo->groupResInfo, pInfo->aggSup.pResultBuf); doFilter(pInfo->pCondition, pRes); - bool hasRemain = hashRemainDataInGroupInfo(&pInfo->groupResInfo); + bool hasRemain = hasDataInGroupInfo(&pInfo->groupResInfo); if (!hasRemain) { doSetOperatorCompleted(pOperator); break; @@ -384,9 +387,9 @@ SOperatorInfo* createGroupOperatorInfo(SOperatorInfo* downstream, SExprInfo* pEx pInfo->pGroupCols = pGroupColList; pInfo->pCondition = pCondition; - pInfo->scalarSup.pScalarExprInfo = pScalarExprInfo; - pInfo->scalarSup.numOfScalarExpr = numOfScalarExpr; - pInfo->scalarSup.pScalarFuncCtx = createSqlFunctionCtx(pScalarExprInfo, numOfScalarExpr, &pInfo->scalarSup.rowCellInfoOffset); + pInfo->scalarSup.pExprInfo = pScalarExprInfo; + pInfo->scalarSup.numOfExprs = numOfScalarExpr; + pInfo->scalarSup.pCtx = createSqlFunctionCtx(pScalarExprInfo, numOfScalarExpr, &pInfo->scalarSup.rowEntryInfoOffset); int32_t code = initGroupOptrInfo(&pInfo->pGroupColVals, &pInfo->groupKeyLen, &pInfo->keyBuf, pGroupColList); if (code != TSDB_CODE_SUCCESS) { @@ -394,15 +397,16 @@ SOperatorInfo* createGroupOperatorInfo(SOperatorInfo* downstream, SExprInfo* pEx } initResultSizeInfo(pOperator, 4096); - initAggInfo(&pInfo->binfo, &pInfo->aggSup, pExprInfo, numOfCols, pResultBlock, pInfo->groupKeyLen, pTaskInfo->id.str); - initResultRowInfo(&pInfo->binfo.resultRowInfo, 8); + initAggInfo(&pOperator->exprSupp, &pInfo->aggSup, pExprInfo, numOfCols, pInfo->groupKeyLen, pTaskInfo->id.str); + initBasicInfo(&pInfo->binfo, pResultBlock); + initResultRowInfo(&pInfo->binfo.resultRowInfo); pOperator->name = "GroupbyAggOperator"; pOperator->blocking = true; pOperator->status = OP_NOT_OPENED; // pOperator->operatorType = OP_Groupby; - pOperator->pExpr = pExprInfo; - pOperator->numOfExprs = numOfCols; + pOperator->exprSupp.pExprInfo = pExprInfo; + pOperator->exprSupp.numOfExprs = numOfCols; pOperator->info = pInfo; pOperator->pTaskInfo = pTaskInfo; @@ -439,9 +443,9 @@ static void doHashPartition(SOperatorInfo* pOperator, SSDataBlock* pBlock) { // group id - size_t numOfCols = pOperator->numOfExprs; + size_t numOfCols = pOperator->exprSupp.numOfExprs; for(int32_t i = 0; i < numOfCols; ++i) { - SExprInfo* pExpr = &pOperator->pExpr[i]; + SExprInfo* pExpr = &pOperator->exprSupp.pExprInfo[i]; int32_t slotId = pExpr->base.pParam[0].pCol->slotId; SColumnInfoData* pColInfoData = taosArrayGet(pBlock->pDataBlock, slotId); @@ -643,8 +647,8 @@ static SSDataBlock* hashPartition(SOperatorInfo* pOperator) { } // there is an scalar expression that needs to be calculated right before apply the group aggregation. - if (pInfo->scalarSupp.pScalarExprInfo != NULL) { - pTaskInfo->code = projectApplyFunctions(pInfo->scalarSupp.pScalarExprInfo, pBlock, pBlock, pInfo->scalarSupp.pScalarFuncCtx, pInfo->scalarSupp.numOfScalarExpr, NULL); + if (pInfo->scalarSup.pExprInfo != NULL) { + pTaskInfo->code = projectApplyFunctions(pInfo->scalarSup.pExprInfo, pBlock, pBlock, pInfo->scalarSup.pCtx, pInfo->scalarSup.numOfExprs, NULL); if (pTaskInfo->code != TSDB_CODE_SUCCESS) { longjmp(pTaskInfo->env, pTaskInfo->code); } @@ -662,16 +666,20 @@ static SSDataBlock* hashPartition(SOperatorInfo* pOperator) { static void destroyPartitionOperatorInfo(void* param, int32_t numOfOutput) { SPartitionOperatorInfo* pInfo = (SPartitionOperatorInfo*)param; - doDestroyBasicInfo(&pInfo->binfo, numOfOutput); + cleanupBasicInfo(&pInfo->binfo); taosArrayDestroy(pInfo->pGroupCols); + for(int i = 0; i < taosArrayGetSize(pInfo->pGroupColVals); i++){ SGroupKeys key = *(SGroupKeys*)taosArrayGet(pInfo->pGroupColVals, i); taosMemoryFree(key.pData); } + taosArrayDestroy(pInfo->pGroupColVals); taosMemoryFree(pInfo->keyBuf); taosHashCleanup(pInfo->pGroupSet); taosMemoryFree(pInfo->columnOffset); + + cleanupExecSupp(&pInfo->scalarSup); } SOperatorInfo* createPartitionOperatorInfo(SOperatorInfo* downstream, SPartitionPhysiNode* pPartNode, SExecTaskInfo* pTaskInfo) { @@ -689,10 +697,10 @@ SOperatorInfo* createPartitionOperatorInfo(SOperatorInfo* downstream, SPartition pInfo->pGroupCols = extractPartitionColInfo(pPartNode->pPartitionKeys); if (pPartNode->pExprs != NULL) { - pInfo->scalarSupp.numOfScalarExpr = 0; - pInfo->scalarSupp.pScalarExprInfo = createExprInfo(pPartNode->pExprs, NULL, &pInfo->scalarSupp.numOfScalarExpr); - pInfo->scalarSupp.pScalarFuncCtx = createSqlFunctionCtx( - pInfo->scalarSupp.pScalarExprInfo, pInfo->scalarSupp.numOfScalarExpr, &pInfo->scalarSupp.rowCellInfoOffset); + pInfo->scalarSup.numOfExprs = 0; + pInfo->scalarSup.pExprInfo = createExprInfo(pPartNode->pExprs, NULL, &pInfo->scalarSup.numOfExprs); + pInfo->scalarSup.pCtx = createSqlFunctionCtx( + pInfo->scalarSup.pExprInfo, pInfo->scalarSup.numOfExprs, &pInfo->scalarSup.rowEntryInfoOffset); } _hash_fn_t hashFn = taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY); @@ -722,8 +730,8 @@ SOperatorInfo* createPartitionOperatorInfo(SOperatorInfo* downstream, SPartition pOperator->status = OP_NOT_OPENED; pOperator->operatorType = QUERY_NODE_PHYSICAL_PLAN_PARTITION; pInfo->binfo.pRes = pResBlock; - pOperator->numOfExprs = numOfCols; - pOperator->pExpr = pExprInfo; + pOperator->exprSupp.numOfExprs = numOfCols; + pOperator->exprSupp.pExprInfo = pExprInfo; pOperator->info = pInfo; pOperator->pTaskInfo = pTaskInfo; @@ -738,4 +746,18 @@ SOperatorInfo* createPartitionOperatorInfo(SOperatorInfo* downstream, SPartition taosMemoryFreeClear(pInfo); taosMemoryFreeClear(pOperator); return NULL; +} + +int32_t setGroupResultOutputBuf(SOperatorInfo* pOperator, SOptrBasicInfo* binfo, int32_t numOfCols, char* pData, int16_t bytes, + int32_t groupId, SDiskbasedBuf* pBuf, SAggSupporter* pAggSup) { + SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo; + SResultRowInfo* pResultRowInfo = &binfo->resultRowInfo; + SqlFunctionCtx* pCtx = pOperator->exprSupp.pCtx; + + SResultRow* pResultRow = + doSetResultOutBufByKey(pBuf, pResultRowInfo, (char*)pData, bytes, true, groupId, pTaskInfo, false, pAggSup); + assert(pResultRow != NULL); + + setResultRowInitCtx(pResultRow, pCtx, numOfCols, pOperator->exprSupp.rowEntryInfoOffset); + return TSDB_CODE_SUCCESS; } \ No newline at end of file diff --git a/source/libs/executor/src/joinoperator.c b/source/libs/executor/src/joinoperator.c index 7c8ab244a1d6cfc9a5e358ab1fbc174d7ea1ce06..6fbda778087b345c7aeef19e068f582451eab006 100644 --- a/source/libs/executor/src/joinoperator.c +++ b/source/libs/executor/src/joinoperator.c @@ -28,27 +28,32 @@ static SSDataBlock* doMergeJoin(struct SOperatorInfo* pOperator); static void destroyMergeJoinOperator(void* param, int32_t numOfOutput); static void extractTimeCondition(SJoinOperatorInfo* Info, SLogicConditionNode* pLogicConditionNode); -SOperatorInfo* createMergeJoinOperatorInfo(SOperatorInfo** pDownstream, int32_t numOfDownstream, SExprInfo* pExprInfo, - int32_t numOfCols, SSDataBlock* pResBlock, SNode* pOnCondition, - SExecTaskInfo* pTaskInfo) { +SOperatorInfo* createMergeJoinOperatorInfo(SOperatorInfo** pDownstream, int32_t numOfDownstream, SJoinPhysiNode* pJoinNode, + SExecTaskInfo* pTaskInfo) { SJoinOperatorInfo* pInfo = taosMemoryCalloc(1, sizeof(SJoinOperatorInfo)); SOperatorInfo* pOperator = taosMemoryCalloc(1, sizeof(SOperatorInfo)); if (pOperator == NULL || pInfo == NULL) { goto _error; } + SSDataBlock* pResBlock = createResDataBlock(pJoinNode->node.pOutputDataBlockDesc); + + int32_t numOfCols = 0; + SExprInfo* pExprInfo = createExprInfo(pJoinNode->pTargets, NULL, &numOfCols); + initResultSizeInfo(pOperator, 4096); - pInfo->pRes = pResBlock; - pOperator->name = "MergeJoinOperator"; + pInfo->pRes = pResBlock; + pOperator->name = "MergeJoinOperator"; pOperator->operatorType = QUERY_NODE_PHYSICAL_PLAN_MERGE_JOIN; - pOperator->blocking = false; - pOperator->status = OP_NOT_OPENED; - pOperator->pExpr = pExprInfo; - pOperator->numOfExprs = numOfCols; - pOperator->info = pInfo; - pOperator->pTaskInfo = pTaskInfo; - + pOperator->blocking = false; + pOperator->status = OP_NOT_OPENED; + pOperator->exprSupp.pExprInfo = pExprInfo; + pOperator->exprSupp.numOfExprs = numOfCols; + pOperator->info = pInfo; + pOperator->pTaskInfo = pTaskInfo; + + SNode* pOnCondition = pJoinNode->pOnConditions; if (nodeType(pOnCondition) == QUERY_NODE_OPERATOR) { SOperatorNode* pNode = (SOperatorNode*)pOnCondition; setJoinColumnInfo(&pInfo->leftCol, (SColumnNode*)pNode->pLeft); @@ -127,10 +132,10 @@ SSDataBlock* doMergeJoin(struct SOperatorInfo* pOperator) { // only the timestamp match support for ordinary table ASSERT(pLeftCol->info.type == TSDB_DATA_TYPE_TIMESTAMP); if (*(int64_t*)pLeftVal == *(int64_t*)pRightVal) { - for (int32_t i = 0; i < pOperator->numOfExprs; ++i) { + for (int32_t i = 0; i < pOperator->exprSupp.numOfExprs; ++i) { SColumnInfoData* pDst = taosArrayGet(pRes->pDataBlock, i); - SExprInfo* pExprInfo = &pOperator->pExpr[i]; + SExprInfo* pExprInfo = &pOperator->exprSupp.pExprInfo[i]; int32_t blockId = pExprInfo->base.pParam[0].pCol->dataBlockId; int32_t slotId = pExprInfo->base.pParam[0].pCol->slotId; diff --git a/source/libs/executor/src/scanoperator.c b/source/libs/executor/src/scanoperator.c index a8aad8fe688fc4b363674a1dd35c58ecddfe98c6..812272c5596c727255a8006378ac055b516b198b 100644 --- a/source/libs/executor/src/scanoperator.c +++ b/source/libs/executor/src/scanoperator.c @@ -261,9 +261,9 @@ static int32_t loadDataBlock(SOperatorInfo* pOperator, STableScanInfo* pTableSca relocateColumnData(pBlock, pTableScanInfo->pColMatchInfo, pCols); // currently only the tbname pseudo column - if (pTableScanInfo->numOfPseudoExpr > 0) { - addTagPseudoColumnData(&pTableScanInfo->readHandle, pTableScanInfo->pPseudoExpr, pTableScanInfo->numOfPseudoExpr, - pBlock); + if (pTableScanInfo->pseudoSup.numOfExprs > 0) { + SExprSupp* pSup = &pTableScanInfo->pseudoSup; + addTagPseudoColumnData(&pTableScanInfo->readHandle, pSup->pExprInfo, pSup->numOfExprs, pBlock); } int64_t st = taosGetTimestampMs(); @@ -496,18 +496,6 @@ static SSDataBlock* doTableScan(SOperatorInfo* pOperator) { return NULL; } -SInterval extractIntervalInfo(const STableScanPhysiNode* pTableScanNode) { - SInterval interval = { - .interval = pTableScanNode->interval, - .sliding = pTableScanNode->sliding, - .intervalUnit = pTableScanNode->intervalUnit, - .slidingUnit = pTableScanNode->slidingUnit, - .offset = pTableScanNode->offset, - }; - - return interval; -} - static int32_t getTableScannerExecInfo(struct SOperatorInfo* pOptr, void** pOptrExplain, uint32_t* len) { SFileBlockLoadRecorder* pRecorder = taosMemoryCalloc(1, sizeof(SFileBlockLoadRecorder)); STableScanInfo* pTableScanInfo = pOptr->info; @@ -520,7 +508,7 @@ static int32_t getTableScannerExecInfo(struct SOperatorInfo* pOptr, void** pOptr static void destroyTableScanOperatorInfo(void* param, int32_t numOfOutput) { STableScanInfo* pTableScanInfo = (STableScanInfo*)param; blockDataDestroy(pTableScanInfo->pResBlock); - clearupQueryTableDataCond(&pTableScanInfo->cond); + cleanupQueryTableDataCond(&pTableScanInfo->cond); tsdbCleanupReadHandle(pTableScanInfo->dataReader); @@ -537,11 +525,12 @@ SOperatorInfo* createTableScanOperatorInfo(STableScanPhysiNode* pTableScanNode, goto _error; } + //taosSsleep(20); + SDataBlockDescNode* pDescNode = pTableScanNode->scan.node.pOutputDataBlockDesc; int32_t numOfCols = 0; - SArray* pColList = - extractColMatchInfo(pTableScanNode->scan.pScanCols, pDescNode, &numOfCols, pTaskInfo, COL_MATCH_FROM_COL_ID); + SArray* pColList = extractColMatchInfo(pTableScanNode->scan.pScanCols, pDescNode, &numOfCols, COL_MATCH_FROM_COL_ID); int32_t code = initQueryTableDataCond(&pInfo->cond, pTableScanNode); if (code != TSDB_CODE_SUCCESS) { @@ -549,8 +538,9 @@ SOperatorInfo* createTableScanOperatorInfo(STableScanPhysiNode* pTableScanNode, } if (pTableScanNode->scan.pScanPseudoCols != NULL) { - pInfo->pPseudoExpr = createExprInfo(pTableScanNode->scan.pScanPseudoCols, NULL, &pInfo->numOfPseudoExpr); - pInfo->pPseudoCtx = createSqlFunctionCtx(pInfo->pPseudoExpr, pInfo->numOfPseudoExpr, &pInfo->rowCellInfoOffset); + SExprSupp* pSup = &pInfo->pseudoSup; + pSup->pExprInfo = createExprInfo(pTableScanNode->scan.pScanPseudoCols, NULL, &pSup->numOfExprs); + pSup->pCtx = createSqlFunctionCtx(pSup->pExprInfo, pSup->numOfExprs, &pSup->rowEntryInfoOffset); } pInfo->scanInfo = (SScanInfo){.numOfAsc = pTableScanNode->scanSeq[0], .numOfDesc = pTableScanNode->scanSeq[1]}; @@ -574,7 +564,7 @@ SOperatorInfo* createTableScanOperatorInfo(STableScanPhysiNode* pTableScanNode, pOperator->blocking = false; pOperator->status = OP_NOT_OPENED; pOperator->info = pInfo; - pOperator->numOfExprs = numOfCols; + pOperator->exprSupp.numOfExprs = numOfCols; pOperator->pTaskInfo = pTaskInfo; pOperator->fpSet = createOperatorFpSet(operatorDummyOpenFn, doTableScan, NULL, NULL, destroyTableScanOperatorInfo, @@ -1064,8 +1054,7 @@ SOperatorInfo* createStreamScanOperatorInfo(void* pDataReader, SReadHandle* pHan STableScanInfo* pSTInfo = (STableScanInfo*)pTableScanDummy->info; int32_t numOfCols = 0; - pInfo->pColMatchInfo = - extractColMatchInfo(pScanPhyNode->pScanCols, pDescNode, &numOfCols, pTaskInfo, COL_MATCH_FROM_COL_ID); + pInfo->pColMatchInfo = extractColMatchInfo(pScanPhyNode->pScanCols, pDescNode, &numOfCols, COL_MATCH_FROM_COL_ID); int32_t numOfOutput = taosArrayGetSize(pInfo->pColMatchInfo); SArray* pColIds = taosArrayInit(numOfOutput, sizeof(int16_t)); @@ -1129,7 +1118,7 @@ SOperatorInfo* createStreamScanOperatorInfo(void* pDataReader, SReadHandle* pHan pOperator->blocking = false; pOperator->status = OP_NOT_OPENED; pOperator->info = pInfo; - pOperator->numOfExprs = pInfo->pRes->info.numOfCols; + pOperator->exprSupp.numOfExprs = pInfo->pRes->info.numOfCols; pOperator->pTaskInfo = pTaskInfo; pOperator->fpSet = @@ -1523,8 +1512,8 @@ static SSDataBlock* doSysTableScan(SOperatorInfo* pOperator) { } } - setDataBlockFromFetchRsp(pInfo->pRes, &pInfo->loadInfo, pRsp->numOfRows, pRsp->data, pRsp->compLen, - pOperator->numOfExprs, startTs, NULL, pInfo->scanCols); + extractDataBlockFromFetchRsp(pInfo->pRes, &pInfo->loadInfo, pRsp->numOfRows, pRsp->data, pRsp->compLen, + pOperator->exprSupp.numOfExprs, startTs, NULL, pInfo->scanCols); // todo log the filter info doFilterResult(pInfo); @@ -1612,7 +1601,7 @@ SOperatorInfo* createSysTableScanOperatorInfo(void* readHandle, SSystemTableScan SSDataBlock* pResBlock = createResDataBlock(pDescNode); int32_t num = 0; - SArray* colList = extractColMatchInfo(pScanNode->pScanCols, pDescNode, &num, pTaskInfo, COL_MATCH_FROM_COL_ID); + SArray* colList = extractColMatchInfo(pScanNode->pScanCols, pDescNode, &num, COL_MATCH_FROM_COL_ID); pInfo->accountId = pScanPhyNode->accountId; pInfo->showRewrite = pScanPhyNode->showRewrite; @@ -1639,7 +1628,7 @@ SOperatorInfo* createSysTableScanOperatorInfo(void* readHandle, SSystemTableScan pOperator->blocking = false; pOperator->status = OP_NOT_OPENED; pOperator->info = pInfo; - pOperator->numOfExprs = pResBlock->info.numOfCols; + pOperator->exprSupp.numOfExprs = pResBlock->info.numOfCols; pOperator->pTaskInfo = pTaskInfo; pOperator->fpSet = @@ -1670,11 +1659,11 @@ static SSDataBlock* doTagScan(SOperatorInfo* pOperator) { int32_t count = 0; SArray* pa = GET_TABLEGROUP(pRuntimeEnv, 0); - int32_t functionId = getExprFunctionId(&pOperator->pExpr[0]); + int32_t functionId = getExprFunctionId(&pOperator->exprSupp.pExprInfo[0]); if (functionId == FUNCTION_TID_TAG) { // return the tags & table Id assert(pQueryAttr->numOfOutput == 1); - SExprInfo* pExprInfo = &pOperator->pExpr[0]; + SExprInfo* pExprInfo = &pOperator->exprSupp.pExprInfo[0]; int32_t rsize = pExprInfo->base.resSchema.bytes; count = 0; @@ -1737,7 +1726,7 @@ static SSDataBlock* doTagScan(SOperatorInfo* pOperator) { #endif STagScanInfo* pInfo = pOperator->info; - SExprInfo* pExprInfo = &pOperator->pExpr[0]; + SExprInfo* pExprInfo = &pOperator->exprSupp.pExprInfo[0]; SSDataBlock* pRes = pInfo->pRes; int32_t size = taosArrayGetSize(pInfo->pTableList->pTableList); @@ -1755,7 +1744,7 @@ static SSDataBlock* doTagScan(SOperatorInfo* pOperator) { STableKeyInfo* item = taosArrayGet(pInfo->pTableList->pTableList, pInfo->curPos); metaGetTableEntryByUid(&mr, item->uid); - for (int32_t j = 0; j < pOperator->numOfExprs; ++j) { + for (int32_t j = 0; j < pOperator->exprSupp.numOfExprs; ++j) { SColumnInfoData* pDst = taosArrayGet(pRes->pDataBlock, pExprInfo[j].base.resSchema.slotId); // refactor later @@ -1818,27 +1807,26 @@ SOperatorInfo* createTagScanOperatorInfo(SReadHandle* pReadHandle, STagScanPhysi SDataBlockDescNode* pDescNode = pPhyNode->node.pOutputDataBlockDesc; + int32_t num = 0; int32_t numOfExprs = 0; SExprInfo* pExprInfo = createExprInfo(pPhyNode->pScanPseudoCols, NULL, &numOfExprs); + SArray* colList = extractColMatchInfo(pPhyNode->pScanPseudoCols, pDescNode, &num, COL_MATCH_FROM_COL_ID); - int32_t num = 0; - SArray* colList = extractColMatchInfo(pPhyNode->pScanPseudoCols, pDescNode, &num, pTaskInfo, COL_MATCH_FROM_COL_ID); + pInfo->pTableList = pTableListInfo; + pInfo->pColMatchInfo = colList; + pInfo->pRes = createResDataBlock(pDescNode); + pInfo->readHandle = *pReadHandle; + pInfo->curPos = 0; + pInfo->pFilterNode = pPhyNode->node.pConditions; - pInfo->pTableList = pTableListInfo; - pInfo->pColMatchInfo = colList; - pInfo->pRes = createResDataBlock(pDescNode); - ; - pInfo->readHandle = *pReadHandle; - pInfo->curPos = 0; - pInfo->pFilterNode = pPhyNode->node.pConditions; - pOperator->name = "TagScanOperator"; + pOperator->name = "TagScanOperator"; pOperator->operatorType = QUERY_NODE_PHYSICAL_PLAN_TAG_SCAN; - pOperator->blocking = false; - pOperator->status = OP_NOT_OPENED; - pOperator->info = pInfo; - pOperator->pExpr = pExprInfo; - pOperator->numOfExprs = numOfExprs; - pOperator->pTaskInfo = pTaskInfo; + pOperator->blocking = false; + pOperator->status = OP_NOT_OPENED; + pOperator->info = pInfo; + pOperator->exprSupp.pExprInfo = pExprInfo; + pOperator->exprSupp.numOfExprs = numOfExprs; + pOperator->pTaskInfo = pTaskInfo; initResultSizeInfo(pOperator, 4096); blockDataEnsureCapacity(pInfo->pRes, pOperator->resultInfo.capacity); @@ -1882,7 +1870,7 @@ typedef struct STableMergeScanInfo { SNode* pFilterNode; // filter info, which is push down by optimizer SqlFunctionCtx* pCtx; // which belongs to the direct upstream operator operator query context SResultRowInfo* pResultRowInfo; - int32_t* rowCellInfoOffset; + int32_t* rowEntryInfoOffset; SExprInfo* pExpr; SSDataBlock* pResBlock; SArray* pColMatchInfo; @@ -1891,7 +1879,7 @@ typedef struct STableMergeScanInfo { SExprInfo* pPseudoExpr; int32_t numOfPseudoExpr; SqlFunctionCtx* pPseudoCtx; - // int32_t* rowCellInfoOffset; + // int32_t* rowEntryInfoOffset; SQueryTableDataCond cond; int32_t scanFlag; // table scan flag to denote if it is a repeat/reverse/main scan @@ -1908,7 +1896,7 @@ int32_t createMultipleDataReaders(STableScanPhysiNode* pTableScanNode, SReadHand STableListInfo* pTableListInfo, SArray* arrayReader, uint64_t queryId, uint64_t taskId, SNode* pTagCond) { int32_t code = - getTableList(pHandle->meta, pTableScanNode->scan.tableType, pTableScanNode->scan.uid, pTableListInfo, pTagCond); + getTableList(pHandle->meta, &pTableScanNode->scan, pTableListInfo, pTagCond); if (code != TSDB_CODE_SUCCESS) { goto _error; } @@ -1935,7 +1923,7 @@ int32_t createMultipleDataReaders(STableScanPhysiNode* pTableScanNode, SReadHand taosArrayDestroy(subListInfo->pTableList); taosMemoryFree(subListInfo); } - clearupQueryTableDataCond(&cond); + cleanupQueryTableDataCond(&cond); return 0; @@ -2133,7 +2121,7 @@ int32_t doOpenTableMergeScanOperator(SOperatorInfo* pOperator) { int32_t numOfBufPage = pInfo->sortBufSize / pInfo->bufPageSize; pInfo->pSortHandle = - tsortCreateSortHandle(pInfo->pSortInfo, pInfo->pColMatchInfo, SORT_MULTISOURCE_MERGE, pInfo->bufPageSize, + tsortCreateSortHandle(pInfo->pSortInfo, SORT_MULTISOURCE_MERGE, pInfo->bufPageSize, numOfBufPage, pInfo->pSortInputBlock, pTaskInfo->id.str); tsortSetFetchRawDataFp(pInfo->pSortHandle, getTableDataBlock, NULL, NULL); @@ -2170,25 +2158,17 @@ SSDataBlock* getSortedTableMergeScanBlockData(SSortHandle* pHandle, int32_t capa blockDataEnsureCapacity(p, capacity); while (1) { - STupleHandle* pTupleHandle = NULL; - if (pInfo->prefetchedTuple == NULL) { - pTupleHandle = tsortNextTuple(pHandle); - } else { - pTupleHandle = pInfo->prefetchedTuple; - } - + STupleHandle* pTupleHandle = tsortNextTuple(pHandle); if (pTupleHandle == NULL) { break; } appendOneRowToDataBlock(p, pTupleHandle); - if (p->info.rows >= capacity) { break; } } - qDebug("%s get sorted row blocks, rows:%d", GET_TASKID(pTaskInfo), p->info.rows); return (p->info.rows > 0) ? p : NULL; } @@ -2219,7 +2199,7 @@ SSDataBlock* doTableMergeScan(SOperatorInfo* pOperator) { void destroyTableMergeScanOperatorInfo(void* param, int32_t numOfOutput) { STableMergeScanInfo* pTableScanInfo = (STableMergeScanInfo*)param; - clearupQueryTableDataCond(&pTableScanInfo->cond); + cleanupQueryTableDataCond(&pTableScanInfo->cond); for (int32_t i = 0; i < taosArrayGetSize(pTableScanInfo->dataReaders); ++i) { tsdbReaderT* reader = taosArrayGetP(pTableScanInfo->dataReaders, i); @@ -2261,7 +2241,6 @@ SOperatorInfo* createTableMergeScanOperatorInfo(STableScanPhysiNode* pTableScanN SReadHandle* readHandle, SExecTaskInfo* pTaskInfo) { STableMergeScanInfo* pInfo = taosMemoryCalloc(1, sizeof(STableMergeScanInfo)); SOperatorInfo* pOperator = taosMemoryCalloc(1, sizeof(SOperatorInfo)); - if (pInfo == NULL || pOperator == NULL) { goto _error; } @@ -2270,7 +2249,7 @@ SOperatorInfo* createTableMergeScanOperatorInfo(STableScanPhysiNode* pTableScanN int32_t numOfCols = 0; SArray* pColList = - extractColMatchInfo(pTableScanNode->scan.pScanCols, pDescNode, &numOfCols, pTaskInfo, COL_MATCH_FROM_COL_ID); + extractColMatchInfo(pTableScanNode->scan.pScanCols, pDescNode, &numOfCols, COL_MATCH_FROM_COL_ID); int32_t code = initQueryTableDataCond(&pInfo->cond, pTableScanNode); if (code != TSDB_CODE_SUCCESS) { @@ -2279,22 +2258,21 @@ SOperatorInfo* createTableMergeScanOperatorInfo(STableScanPhysiNode* pTableScanN if (pTableScanNode->scan.pScanPseudoCols != NULL) { pInfo->pPseudoExpr = createExprInfo(pTableScanNode->scan.pScanPseudoCols, NULL, &pInfo->numOfPseudoExpr); - pInfo->pPseudoCtx = createSqlFunctionCtx(pInfo->pPseudoExpr, pInfo->numOfPseudoExpr, &pInfo->rowCellInfoOffset); + pInfo->pPseudoCtx = createSqlFunctionCtx(pInfo->pPseudoExpr, pInfo->numOfPseudoExpr, &pInfo->rowEntryInfoOffset); } pInfo->scanInfo = (SScanInfo){.numOfAsc = pTableScanNode->scanSeq[0], .numOfDesc = pTableScanNode->scanSeq[1]}; - pInfo->readHandle = *readHandle; - pInfo->interval = extractIntervalInfo(pTableScanNode); + pInfo->readHandle = *readHandle; + pInfo->interval = extractIntervalInfo(pTableScanNode); pInfo->sample.sampleRatio = pTableScanNode->ratio; - pInfo->sample.seed = taosGetTimestampSec(); - - pInfo->dataBlockLoadFlag = pTableScanNode->dataRequired; - pInfo->pFilterNode = pTableScanNode->scan.node.pConditions; - pInfo->dataReaders = dataReaders; - pInfo->scanFlag = MAIN_SCAN; - pInfo->pColMatchInfo = pColList; - pInfo->curTWinIdx = 0; + pInfo->sample.seed = taosGetTimestampSec(); + pInfo->dataBlockLoadFlag = pTableScanNode->dataRequired; + pInfo->pFilterNode = pTableScanNode->scan.node.pConditions; + pInfo->dataReaders = dataReaders; + pInfo->scanFlag = MAIN_SCAN; + pInfo->pColMatchInfo = pColList; + pInfo->curTWinIdx = 0; pInfo->pResBlock = createResDataBlock(pDescNode); @@ -2307,22 +2285,27 @@ SOperatorInfo* createTableMergeScanOperatorInfo(STableScanPhysiNode* pTableScanN taosArrayPush(pInfo->sortSourceParams, param); taosMemoryFree(param); } + pInfo->pSortInfo = generateSortByTsInfo(pInfo->cond.order); pInfo->pSortInputBlock = createOneDataBlock(pInfo->pResBlock, false); + int32_t rowSize = pInfo->pResBlock->info.rowSize; - int32_t blockMetaSize = (int32_t)blockDataGetSerialMetaSize(pInfo->pResBlock->info.numOfCols); - pInfo->bufPageSize = (rowSize * 2 + blockMetaSize) < 1024 ? 1024 : (rowSize * 2 + blockMetaSize); - pInfo->sortBufSize = pInfo->bufPageSize * 16; - pInfo->hasGroupId = false; + pInfo->bufPageSize = getProperSortPageSize(rowSize); + + // todo the total available buffer should be determined by total capacity of buffer of this task. + // the additional one is reserved for merge result + pInfo->sortBufSize = pInfo->bufPageSize * (taosArrayGetSize(dataReaders) + 1); + pInfo->hasGroupId = false; pInfo->prefetchedTuple = NULL; - pOperator->name = "TableMergeScanOperator"; + pOperator->name = "TableMergeScanOperator"; + // TODO : change it pOperator->operatorType = QUERY_NODE_PHYSICAL_PLAN_TABLE_MERGE_SCAN; - pOperator->blocking = false; - pOperator->status = OP_NOT_OPENED; - pOperator->info = pInfo; - pOperator->numOfExprs = numOfCols; - pOperator->pTaskInfo = pTaskInfo; + pOperator->blocking = false; + pOperator->status = OP_NOT_OPENED; + pOperator->info = pInfo; + pOperator->exprSupp.numOfExprs = numOfCols; + pOperator->pTaskInfo = pTaskInfo; initResultSizeInfo(pOperator, 1024); pOperator->fpSet = diff --git a/source/libs/executor/src/sortoperator.c b/source/libs/executor/src/sortoperator.c index 81899b68cd51a4c11a5f12728145b07090e56839..9d13276e6d68d2aaf380d0ac3111c2214c6f0e75 100644 --- a/source/libs/executor/src/sortoperator.c +++ b/source/libs/executor/src/sortoperator.c @@ -22,41 +22,52 @@ static int32_t getExplainExecInfo(SOperatorInfo* pOptr, void** pOptrExplain static void destroyOrderOperatorInfo(void* param, int32_t numOfOutput); -SOperatorInfo* createSortOperatorInfo(SOperatorInfo* downstream, SSDataBlock* pResBlock, SArray* pSortInfo, - SExprInfo* pExprInfo, int32_t numOfCols, SArray* pColMatchColInfo, - SExecTaskInfo* pTaskInfo) { +SOperatorInfo* createSortOperatorInfo(SOperatorInfo* downstream, SSortPhysiNode* pSortPhyNode, SExecTaskInfo* pTaskInfo) { SSortOperatorInfo* pInfo = taosMemoryCalloc(1, sizeof(SSortOperatorInfo)); SOperatorInfo* pOperator = taosMemoryCalloc(1, sizeof(SOperatorInfo)); - int32_t rowSize = pResBlock->info.rowSize; - - if (pInfo == NULL || pOperator == NULL || rowSize > 100 * 1024 * 1024) { + if (pInfo == NULL || pOperator == NULL/* || rowSize > 100 * 1024 * 1024*/) { goto _error; } - pOperator->pExpr = pExprInfo; - pOperator->numOfExprs = numOfCols; - pInfo->binfo.pCtx = createSqlFunctionCtx(pExprInfo, numOfCols, &pInfo->binfo.rowCellInfoOffset); + SDataBlockDescNode* pDescNode = pSortPhyNode->node.pOutputDataBlockDesc; + + int32_t numOfCols = 0; + SSDataBlock* pResBlock = createResDataBlock(pDescNode); + SExprInfo* pExprInfo = createExprInfo(pSortPhyNode->pExprs, NULL, &numOfCols); + + int32_t numOfOutputCols = 0; + SArray* pColMatchColInfo = + extractColMatchInfo(pSortPhyNode->pTargets, pDescNode, &numOfOutputCols, COL_MATCH_FROM_SLOT_ID); + + pOperator->exprSupp.pCtx = createSqlFunctionCtx(pExprInfo, numOfCols, &pOperator->exprSupp.rowEntryInfoOffset); pInfo->binfo.pRes = pResBlock; initResultSizeInfo(pOperator, 1024); - pInfo->pSortInfo = pSortInfo; + pInfo->pSortInfo = createSortInfo(pSortPhyNode->pSortKeys);; pInfo->pColMatchInfo = pColMatchColInfo; pOperator->name = "SortOperator"; pOperator->operatorType = QUERY_NODE_PHYSICAL_PLAN_SORT; pOperator->blocking = true; pOperator->status = OP_NOT_OPENED; pOperator->info = pInfo; + pOperator->exprSupp.pExprInfo = pExprInfo; + pOperator->exprSupp.numOfExprs = numOfCols; + pOperator->pTaskInfo = pTaskInfo; // lazy evaluation for the following parameter since the input datablock is not known till now. - // pInfo->bufPageSize = rowSize < 1024 ? 1024 * 2 : rowSize * 2; // there are headers, so pageSize = rowSize + - // header pInfo->sortBufSize = pInfo->bufPageSize * 16; // TODO dynamic set the available sort buffer + // pInfo->bufPageSize = rowSize < 1024 ? 1024 * 2 : rowSize * 2; + // there are headers, so pageSize = rowSize + header pInfo->sortBufSize = pInfo->bufPageSize * 16; + // TODO dynamic set the available sort buffer - pOperator->pTaskInfo = pTaskInfo; pOperator->fpSet = createOperatorFpSet(doOpenSortOperator, doSort, NULL, NULL, destroyOrderOperatorInfo, NULL, NULL, getExplainExecInfo); int32_t code = appendDownstream(pOperator, &downstream, 1); + if (code != TSDB_CODE_SUCCESS) { + goto _error; + } + return pOperator; _error: @@ -134,9 +145,9 @@ SSDataBlock* loadNextDataBlock(void* param) { void applyScalarFunction(SSDataBlock* pBlock, void* param) { SOperatorInfo* pOperator = param; SSortOperatorInfo* pSort = pOperator->info; - if (pOperator->pExpr != NULL) { + if (pOperator->exprSupp.pExprInfo != NULL) { int32_t code = - projectApplyFunctions(pOperator->pExpr, pBlock, pBlock, pSort->binfo.pCtx, pOperator->numOfExprs, NULL); + projectApplyFunctions(pOperator->exprSupp.pExprInfo, pBlock, pBlock, pOperator->exprSupp.pCtx, pOperator->exprSupp.numOfExprs, NULL); if (code != TSDB_CODE_SUCCESS) { longjmp(pOperator->pTaskInfo->env, code); } @@ -154,7 +165,7 @@ int32_t doOpenSortOperator(SOperatorInfo* pOperator) { pInfo->startTs = taosGetTimestampUs(); // pInfo->binfo.pRes is not equalled to the input datablock. - pInfo->pSortHandle = tsortCreateSortHandle(pInfo->pSortInfo, pInfo->pColMatchInfo, SORT_SINGLESOURCE_SORT, -1, -1, + pInfo->pSortHandle = tsortCreateSortHandle(pInfo->pSortInfo, SORT_SINGLESOURCE_SORT, -1, -1, NULL, pTaskInfo->id.str); tsortSetFetchRawDataFp(pInfo->pSortHandle, loadNextDataBlock, applyScalarFunction, pOperator); @@ -248,7 +259,7 @@ int32_t doOpenMultiwaySortMergeOperator(SOperatorInfo* pOperator) { int32_t numOfBufPage = pInfo->sortBufSize / pInfo->bufPageSize; - pInfo->pSortHandle = tsortCreateSortHandle(pInfo->pSortInfo, pInfo->pColMatchInfo, SORT_MULTISOURCE_MERGE, + pInfo->pSortHandle = tsortCreateSortHandle(pInfo->pSortInfo, SORT_MULTISOURCE_MERGE, pInfo->bufPageSize, numOfBufPage, pInfo->pInputBlock, pTaskInfo->id.str); tsortSetFetchRawDataFp(pInfo->pSortHandle, loadNextDataBlock, NULL, NULL); @@ -392,10 +403,8 @@ SOperatorInfo* createMultiwaySortMergeOperatorInfo(SOperatorInfo** downStreams, pInfo->bufPageSize = getProperSortPageSize(rowSize); - uint32_t numOfSources = taosArrayGetSize(pSortInfo); - numOfSources = TMAX(4, numOfSources); - - pInfo->sortBufSize = numOfSources * pInfo->bufPageSize; + // one additional is reserved for merged result. + pInfo->sortBufSize = pInfo->bufPageSize * (numStreams + 1); pOperator->fpSet = createOperatorFpSet(doOpenMultiwaySortMergeOperator, doMultiwaySortMerge, NULL, NULL, diff --git a/source/libs/executor/src/timewindowoperator.c b/source/libs/executor/src/timewindowoperator.c index d7ae82352235295c327f4437192732fb8f3c39f1..a31bfab62d1d696486769a6255ff60459eeea314 100644 --- a/source/libs/executor/src/timewindowoperator.c +++ b/source/libs/executor/src/timewindowoperator.c @@ -103,7 +103,7 @@ STimeWindow getActiveTimeWindow(SDiskbasedBuf* pBuf, SResultRowInfo* pResultRowI static int32_t setTimeWindowOutputBuf(SResultRowInfo* pResultRowInfo, STimeWindow* win, bool masterscan, SResultRow** pResult, int64_t tableGroupId, SqlFunctionCtx* pCtx, - int32_t numOfOutput, int32_t* rowCellInfoOffset, SAggSupporter* pAggSup, + int32_t numOfOutput, int32_t* rowEntryInfoOffset, SAggSupporter* pAggSup, SExecTaskInfo* pTaskInfo) { assert(win->skey <= win->ekey); SResultRow* pResultRow = doSetResultOutBufByKey(pAggSup->pResultBuf, pResultRowInfo, (char*)&win->skey, TSDB_KEYSIZE, @@ -118,7 +118,7 @@ static int32_t setTimeWindowOutputBuf(SResultRowInfo* pResultRowInfo, STimeWindo pResultRow->win = (*win); *pResult = pResultRow; - setResultRowInitCtx(pResultRow, pCtx, numOfOutput, rowCellInfoOffset); + setResultRowInitCtx(pResultRow, pCtx, numOfOutput, rowEntryInfoOffset); return TSDB_CODE_SUCCESS; } @@ -322,12 +322,12 @@ static void getNextTimeWindow(SInterval* pInterval, int32_t precision, int32_t o tw->ekey -= 1; } -void doTimeWindowInterpolation(SIntervalAggOperatorInfo* pInfo, int32_t numOfExprs, SArray* pDataBlock, TSKEY prevTs, - int32_t prevRowIndex, TSKEY curTs, int32_t curRowIndex, TSKEY windowKey, int32_t type) { - SqlFunctionCtx* pCtx = pInfo->binfo.pCtx; +void doTimeWindowInterpolation(SArray* pPrevValues, SArray* pDataBlock, TSKEY prevTs, int32_t prevRowIndex, + TSKEY curTs, int32_t curRowIndex, TSKEY windowKey, int32_t type, SExprSupp* pSup) { + SqlFunctionCtx* pCtx = pSup->pCtx; int32_t index = 1; - for (int32_t k = 0; k < numOfExprs; ++k) { + for (int32_t k = 0; k < pSup->numOfExprs; ++k) { if (!fmIsIntervalInterpoFunc(pCtx[k].functionId)) { pCtx[k].start.key = INT64_MIN; continue; @@ -340,7 +340,7 @@ void doTimeWindowInterpolation(SIntervalAggOperatorInfo* pInfo, int32_t numOfExp double v1 = 0, v2 = 0, v = 0; if (prevRowIndex == -1) { - SGroupKeys* p = taosArrayGet(pInfo->pPrevValues, index); + SGroupKeys* p = taosArrayGet(pPrevValues, index); GET_TYPED_DATA(v1, double, pColInfo->info.type, p->pData); } else { GET_TYPED_DATA(v1, double, pColInfo->info.type, colDataGetData(pColInfo, prevRowIndex)); @@ -403,8 +403,8 @@ static void setNotInterpoWindowKey(SqlFunctionCtx* pCtx, int32_t numOfOutput, in } } -static bool setTimeWindowInterpolationStartTs(SIntervalAggOperatorInfo* pInfo, SqlFunctionCtx* pCtx, int32_t numOfExprs, - int32_t pos, SSDataBlock* pBlock, const TSKEY* tsCols, STimeWindow* win) { +static bool setTimeWindowInterpolationStartTs(SIntervalAggOperatorInfo* pInfo, int32_t pos, SSDataBlock* pBlock, const TSKEY* tsCols, + STimeWindow* win, SExprSupp* pSup) { bool ascQuery = (pInfo->order == TSDB_ORDER_ASC); TSKEY curTs = tsCols[pos]; @@ -416,23 +416,23 @@ static bool setTimeWindowInterpolationStartTs(SIntervalAggOperatorInfo* pInfo, S // start exactly from this point, no need to do interpolation TSKEY key = ascQuery ? win->skey : win->ekey; if (key == curTs) { - setNotInterpoWindowKey(pCtx, numOfExprs, RESULT_ROW_START_INTERP); + setNotInterpoWindowKey(pSup->pCtx, pSup->numOfExprs, RESULT_ROW_START_INTERP); return true; } // it is the first time window, no need to do interpolation if (pTsKey->isNull && pos == 0) { - setNotInterpoWindowKey(pCtx, numOfExprs, RESULT_ROW_START_INTERP); + setNotInterpoWindowKey(pSup->pCtx, pSup->numOfExprs, RESULT_ROW_START_INTERP); } else { TSKEY prevTs = ((pos == 0) ? lastTs : tsCols[pos - 1]); - doTimeWindowInterpolation(pInfo, numOfExprs, pBlock->pDataBlock, prevTs, pos - 1, curTs, pos, key, - RESULT_ROW_START_INTERP); + doTimeWindowInterpolation(pInfo->pPrevValues, pBlock->pDataBlock, prevTs, pos - 1, curTs, pos, key, + RESULT_ROW_START_INTERP, pSup); } return true; } -static bool setTimeWindowInterpolationEndTs(SIntervalAggOperatorInfo* pInfo, SqlFunctionCtx* pCtx, int32_t numOfExprs, +static bool setTimeWindowInterpolationEndTs(SIntervalAggOperatorInfo* pInfo, SExprSupp* pSup, int32_t endRowIndex, SArray* pDataBlock, const TSKEY* tsCols, TSKEY blockEkey, STimeWindow* win) { int32_t order = pInfo->order; @@ -442,13 +442,13 @@ static bool setTimeWindowInterpolationEndTs(SIntervalAggOperatorInfo* pInfo, Sql // not ended in current data block, do not invoke interpolation if ((key > blockEkey && (order == TSDB_ORDER_ASC)) || (key < blockEkey && (order == TSDB_ORDER_DESC))) { - setNotInterpoWindowKey(pCtx, numOfExprs, RESULT_ROW_END_INTERP); + setNotInterpoWindowKey(pSup->pCtx, pSup->numOfExprs, RESULT_ROW_END_INTERP); return false; } // there is actual end point of current time window, no interpolation needs if (key == actualEndKey) { - setNotInterpoWindowKey(pCtx, numOfExprs, RESULT_ROW_END_INTERP); + setNotInterpoWindowKey(pSup->pCtx, pSup->numOfExprs, RESULT_ROW_END_INTERP); return true; } @@ -456,8 +456,8 @@ static bool setTimeWindowInterpolationEndTs(SIntervalAggOperatorInfo* pInfo, Sql assert(nextRowIndex >= 0); TSKEY nextKey = tsCols[nextRowIndex]; - doTimeWindowInterpolation(pInfo, numOfExprs, pDataBlock, actualEndKey, endRowIndex, nextKey, nextRowIndex, key, - RESULT_ROW_END_INTERP); + doTimeWindowInterpolation(pInfo->pPrevValues, pDataBlock, actualEndKey, endRowIndex, nextKey, nextRowIndex, key, + RESULT_ROW_END_INTERP, pSup); return true; } @@ -546,9 +546,8 @@ static void setResultRowInterpo(SResultRow* pResult, SResultTsInterpType type) { } } -static void doWindowBorderInterpolation(SIntervalAggOperatorInfo* pInfo, SSDataBlock* pBlock, int32_t numOfExprs, - SqlFunctionCtx* pCtx, SResultRow* pResult, STimeWindow* win, int32_t startPos, - int32_t forwardRows) { +static void doWindowBorderInterpolation(SIntervalAggOperatorInfo* pInfo, SSDataBlock* pBlock, SResultRow* pResult, STimeWindow* win, int32_t startPos, + int32_t forwardRows, SExprSupp* pSup) { if (!pInfo->timeWindowInterpo) { return; } @@ -564,12 +563,12 @@ static void doWindowBorderInterpolation(SIntervalAggOperatorInfo* pInfo, SSDataB TSKEY* tsCols = (TSKEY*)(pColInfo->pData); bool done = isResultRowInterpolated(pResult, RESULT_ROW_START_INTERP); if (!done) { // it is not interpolated, now start to generated the interpolated value - bool interp = setTimeWindowInterpolationStartTs(pInfo, pCtx, numOfExprs, startPos, pBlock, tsCols, win); + bool interp = setTimeWindowInterpolationStartTs(pInfo, startPos, pBlock, tsCols, win, pSup); if (interp) { setResultRowInterpo(pResult, RESULT_ROW_START_INTERP); } } else { - setNotInterpoWindowKey(pCtx, numOfExprs, RESULT_ROW_START_INTERP); + setNotInterpoWindowKey(pSup->pCtx, pSup->numOfExprs, RESULT_ROW_START_INTERP); } // point interpolation does not require the end key time window interpolation. @@ -583,13 +582,12 @@ static void doWindowBorderInterpolation(SIntervalAggOperatorInfo* pInfo, SSDataB int32_t endRowIndex = startPos + forwardRows - 1; TSKEY endKey = (pInfo->order == TSDB_ORDER_ASC) ? pBlock->info.window.ekey : pBlock->info.window.skey; - bool interp = - setTimeWindowInterpolationEndTs(pInfo, pCtx, numOfExprs, endRowIndex, pBlock->pDataBlock, tsCols, endKey, win); + bool interp = setTimeWindowInterpolationEndTs(pInfo, pSup, endRowIndex, pBlock->pDataBlock, tsCols, endKey, win); if (interp) { setResultRowInterpo(pResult, RESULT_ROW_END_INTERP); } } else { - setNotInterpoWindowKey(pCtx, numOfExprs, RESULT_ROW_END_INTERP); + setNotInterpoWindowKey(pSup->pCtx, pSup->numOfExprs, RESULT_ROW_END_INTERP); } } @@ -628,9 +626,10 @@ static void doInterpUnclosedTimeWindow(SOperatorInfo* pOperatorInfo, int32_t num SExecTaskInfo* pTaskInfo = pOperatorInfo->pTaskInfo; SIntervalAggOperatorInfo* pInfo = (SIntervalAggOperatorInfo*)pOperatorInfo->info; + SExprSupp* pSup = &pOperatorInfo->exprSupp; int32_t startPos = 0; - int32_t numOfOutput = pOperatorInfo->numOfExprs; + int32_t numOfOutput = pSup->numOfExprs; uint64_t groupId = pBlock->info.groupId; SResultRow* pResult = NULL; @@ -655,8 +654,8 @@ static void doInterpUnclosedTimeWindow(SOperatorInfo* pOperatorInfo, int32_t num STimeWindow w = pr->win; int32_t ret = - setTimeWindowOutputBuf(pResultRowInfo, &w, (scanFlag == MAIN_SCAN), &pResult, groupId, pInfo->binfo.pCtx, - numOfOutput, pInfo->binfo.rowCellInfoOffset, &pInfo->aggSup, pTaskInfo); + setTimeWindowOutputBuf(pResultRowInfo, &w, (scanFlag == MAIN_SCAN), &pResult, groupId, pSup->pCtx, + numOfOutput, pSup->rowEntryInfoOffset, &pInfo->aggSup, pTaskInfo); if (ret != TSDB_CODE_SUCCESS) { longjmp(pTaskInfo->env, TSDB_CODE_QRY_OUT_OF_MEMORY); } @@ -665,13 +664,13 @@ static void doInterpUnclosedTimeWindow(SOperatorInfo* pOperatorInfo, int32_t num SGroupKeys* pTsKey = taosArrayGet(pInfo->pPrevValues, 0); int64_t prevTs = *(int64_t*)pTsKey->pData; - doTimeWindowInterpolation(pInfo, numOfOutput, pBlock->pDataBlock, prevTs, -1, tsCols[startPos], startPos, w.ekey, - RESULT_ROW_END_INTERP); + doTimeWindowInterpolation(pInfo->pPrevValues, pBlock->pDataBlock, prevTs, -1, tsCols[startPos], startPos, w.ekey, + RESULT_ROW_END_INTERP, pSup); setResultRowInterpo(pResult, RESULT_ROW_END_INTERP); - setNotInterpoWindowKey(pInfo->binfo.pCtx, numOfExprs, RESULT_ROW_START_INTERP); + setNotInterpoWindowKey(pSup->pCtx, numOfExprs, RESULT_ROW_START_INTERP); - doApplyFunctions(pTaskInfo, pInfo->binfo.pCtx, &w, &pInfo->twAggSup.timeWindowData, startPos, 0, tsCols, + doApplyFunctions(pTaskInfo, pSup->pCtx, &w, &pInfo->twAggSup.timeWindowData, startPos, 0, tsCols, pBlock->info.rows, numOfExprs, pInfo->order); if (isResultRowInterpolated(pResult, RESULT_ROW_END_INTERP)) { @@ -798,9 +797,10 @@ static void hashIntervalAgg(SOperatorInfo* pOperatorInfo, SResultRowInfo* pResul SIntervalAggOperatorInfo* pInfo = (SIntervalAggOperatorInfo*)pOperatorInfo->info; SExecTaskInfo* pTaskInfo = pOperatorInfo->pTaskInfo; + SExprSupp* pSup = &pOperatorInfo->exprSupp; int32_t startPos = 0; - int32_t numOfOutput = pOperatorInfo->numOfExprs; + int32_t numOfOutput = pSup->numOfExprs; int64_t* tsCols = extractTsCol(pBlock, pInfo); uint64_t tableGroupId = pBlock->info.groupId; bool ascScan = (pInfo->order == TSDB_ORDER_ASC); @@ -811,8 +811,8 @@ static void hashIntervalAgg(SOperatorInfo* pOperatorInfo, SResultRowInfo* pResul pInfo->interval.precision, &pInfo->win); int32_t ret = - setTimeWindowOutputBuf(pResultRowInfo, &win, (scanFlag == MAIN_SCAN), &pResult, tableGroupId, pInfo->binfo.pCtx, - numOfOutput, pInfo->binfo.rowCellInfoOffset, &pInfo->aggSup, pTaskInfo); + setTimeWindowOutputBuf(pResultRowInfo, &win, (scanFlag == MAIN_SCAN), &pResult, tableGroupId, pSup->pCtx, + numOfOutput, pSup->rowEntryInfoOffset, &pInfo->aggSup, pTaskInfo); if (ret != TSDB_CODE_SUCCESS || pResult == NULL) { longjmp(pTaskInfo->env, TSDB_CODE_QRY_OUT_OF_MEMORY); } @@ -835,18 +835,18 @@ static void hashIntervalAgg(SOperatorInfo* pOperatorInfo, SResultRowInfo* pResul // restore current time window ret = - setTimeWindowOutputBuf(pResultRowInfo, &win, (scanFlag == MAIN_SCAN), &pResult, tableGroupId, pInfo->binfo.pCtx, - numOfOutput, pInfo->binfo.rowCellInfoOffset, &pInfo->aggSup, pTaskInfo); + setTimeWindowOutputBuf(pResultRowInfo, &win, (scanFlag == MAIN_SCAN), &pResult, tableGroupId, pSup->pCtx, + numOfOutput, pSup->rowEntryInfoOffset, &pInfo->aggSup, pTaskInfo); if (ret != TSDB_CODE_SUCCESS) { longjmp(pTaskInfo->env, TSDB_CODE_QRY_OUT_OF_MEMORY); } // window start key interpolation - doWindowBorderInterpolation(pInfo, pBlock, numOfOutput, pInfo->binfo.pCtx, pResult, &win, startPos, forwardRows); + doWindowBorderInterpolation(pInfo, pBlock, pResult, &win, startPos, forwardRows, pSup); } updateTimeWindowInfo(&pInfo->twAggSup.timeWindowData, &win, true); - doApplyFunctions(pTaskInfo, pInfo->binfo.pCtx, &win, &pInfo->twAggSup.timeWindowData, startPos, forwardRows, tsCols, + doApplyFunctions(pTaskInfo, pSup->pCtx, &win, &pInfo->twAggSup.timeWindowData, startPos, forwardRows, tsCols, pBlock->info.rows, numOfOutput, pInfo->order); doCloseWindow(pResultRowInfo, pInfo, pResult); @@ -861,7 +861,7 @@ static void hashIntervalAgg(SOperatorInfo* pOperatorInfo, SResultRowInfo* pResul // null data, failed to allocate more memory buffer int32_t code = setTimeWindowOutputBuf(pResultRowInfo, &nextWin, (scanFlag == MAIN_SCAN), &pResult, tableGroupId, - pInfo->binfo.pCtx, numOfOutput, pInfo->binfo.rowCellInfoOffset, + pSup->pCtx, numOfOutput, pSup->rowEntryInfoOffset, &pInfo->aggSup, pTaskInfo); if (code != TSDB_CODE_SUCCESS || pResult == NULL) { longjmp(pTaskInfo->env, TSDB_CODE_QRY_OUT_OF_MEMORY); @@ -878,11 +878,10 @@ static void hashIntervalAgg(SOperatorInfo* pOperatorInfo, SResultRowInfo* pResul getNumOfRowsInTimeWindow(&pBlock->info, tsCols, startPos, ekey, binarySearchForKey, NULL, pInfo->order); // window start(end) key interpolation - doWindowBorderInterpolation(pInfo, pBlock, numOfOutput, pInfo->binfo.pCtx, pResult, &nextWin, startPos, - forwardRows); + doWindowBorderInterpolation(pInfo, pBlock, pResult, &nextWin, startPos, forwardRows, pSup); updateTimeWindowInfo(&pInfo->twAggSup.timeWindowData, &nextWin, true); - doApplyFunctions(pTaskInfo, pInfo->binfo.pCtx, &nextWin, &pInfo->twAggSup.timeWindowData, startPos, forwardRows, + doApplyFunctions(pTaskInfo, pSup->pCtx, &nextWin, &pInfo->twAggSup.timeWindowData, startPos, forwardRows, tsCols, pBlock->info.rows, numOfOutput, pInfo->order); doCloseWindow(pResultRowInfo, pInfo, pResult); } @@ -937,6 +936,7 @@ static int32_t doOpenIntervalAgg(SOperatorInfo* pOperator) { SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo; SIntervalAggOperatorInfo* pInfo = pOperator->info; + SExprSupp* pSup = &pOperator->exprSupp; int32_t scanFlag = MAIN_SCAN; @@ -952,7 +952,7 @@ static int32_t doOpenIntervalAgg(SOperatorInfo* pOperator) { getTableScanInfo(pOperator, &pInfo->order, &scanFlag); // the pDataBlock are always the same one, no need to call this again - setInputDataBlock(pOperator, pInfo->binfo.pCtx, pBlock, pInfo->order, scanFlag, true); + setInputDataBlock(pOperator, pSup->pCtx, pBlock, pInfo->order, scanFlag, true); hashIntervalAgg(pOperator, &pInfo->binfo.resultRowInfo, pBlock, scanFlag, NULL); #if 0 // test for encode/decode result info @@ -993,12 +993,13 @@ static bool compareVal(const char* v, const SStateKeys* pKey) { static void doStateWindowAggImpl(SOperatorInfo* pOperator, SStateWindowOperatorInfo* pInfo, SSDataBlock* pBlock) { SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo; + SExprSupp* pSup = &pOperator->exprSupp; SColumnInfoData* pStateColInfoData = taosArrayGet(pBlock->pDataBlock, pInfo->stateCol.slotId); int64_t gid = pBlock->info.groupId; bool masterScan = true; - int32_t numOfOutput = pOperator->numOfExprs; + int32_t numOfOutput = pOperator->exprSupp.numOfExprs; int16_t bytes = pStateColInfoData->info.bytes; SColumnInfoData* pColInfoData = taosArrayGet(pBlock->pDataBlock, pInfo->tsSlotId); @@ -1041,14 +1042,14 @@ static void doStateWindowAggImpl(SOperatorInfo* pOperator, SStateWindowOperatorI pRowSup->win.ekey = pRowSup->win.skey; int32_t ret = - setTimeWindowOutputBuf(&pInfo->binfo.resultRowInfo, &window, masterScan, &pResult, gid, pInfo->binfo.pCtx, - numOfOutput, pInfo->binfo.rowCellInfoOffset, &pInfo->aggSup, pTaskInfo); + setTimeWindowOutputBuf(&pInfo->binfo.resultRowInfo, &window, masterScan, &pResult, gid, pSup->pCtx, + numOfOutput, pSup->rowEntryInfoOffset, &pInfo->aggSup, pTaskInfo); if (ret != TSDB_CODE_SUCCESS) { // null data, too many state code longjmp(pTaskInfo->env, TSDB_CODE_QRY_APP_ERROR); } updateTimeWindowInfo(&pInfo->twAggSup.timeWindowData, &window, false); - doApplyFunctions(pTaskInfo, pInfo->binfo.pCtx, &window, &pInfo->twAggSup.timeWindowData, pRowSup->startRowIndex, + doApplyFunctions(pTaskInfo, pSup->pCtx, &window, &pInfo->twAggSup.timeWindowData, pRowSup->startRowIndex, pRowSup->numOfRows, NULL, pBlock->info.rows, numOfOutput, TSDB_ORDER_ASC); // here we start a new session window @@ -1067,14 +1068,14 @@ static void doStateWindowAggImpl(SOperatorInfo* pOperator, SStateWindowOperatorI SResultRow* pResult = NULL; pRowSup->win.ekey = tsList[pBlock->info.rows - 1]; int32_t ret = - setTimeWindowOutputBuf(&pInfo->binfo.resultRowInfo, &pRowSup->win, masterScan, &pResult, gid, pInfo->binfo.pCtx, - numOfOutput, pInfo->binfo.rowCellInfoOffset, &pInfo->aggSup, pTaskInfo); + setTimeWindowOutputBuf(&pInfo->binfo.resultRowInfo, &pRowSup->win, masterScan, &pResult, gid, pSup->pCtx, + numOfOutput, pSup->rowEntryInfoOffset, &pInfo->aggSup, pTaskInfo); if (ret != TSDB_CODE_SUCCESS) { // null data, too many state code longjmp(pTaskInfo->env, TSDB_CODE_QRY_APP_ERROR); } updateTimeWindowInfo(&pInfo->twAggSup.timeWindowData, &pRowSup->win, false); - doApplyFunctions(pTaskInfo, pInfo->binfo.pCtx, &pRowSup->win, &pInfo->twAggSup.timeWindowData, pRowSup->startRowIndex, + doApplyFunctions(pTaskInfo, pSup->pCtx, &pRowSup->win, &pInfo->twAggSup.timeWindowData, pRowSup->startRowIndex, pRowSup->numOfRows, NULL, pBlock->info.rows, numOfOutput, TSDB_ORDER_ASC); } @@ -1086,11 +1087,13 @@ static SSDataBlock* doStateWindowAgg(SOperatorInfo* pOperator) { SStateWindowOperatorInfo* pInfo = pOperator->info; SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo; + SExprSupp* pSup = &pOperator->exprSupp; + SOptrBasicInfo* pBInfo = &pInfo->binfo; if (pOperator->status == OP_RES_TO_RETURN) { doBuildResultDatablock(pOperator, pBInfo, &pInfo->groupResInfo, pInfo->aggSup.pResultBuf); - if (pBInfo->pRes->info.rows == 0 || !hashRemainDataInGroupInfo(&pInfo->groupResInfo)) { + if (pBInfo->pRes->info.rows == 0 || !hasDataInGroupInfo(&pInfo->groupResInfo)) { doSetOperatorCompleted(pOperator); return NULL; } @@ -1108,7 +1111,7 @@ static SSDataBlock* doStateWindowAgg(SOperatorInfo* pOperator) { break; } - setInputDataBlock(pOperator, pBInfo->pCtx, pBlock, order, MAIN_SCAN, true); + setInputDataBlock(pOperator, pSup->pCtx, pBlock, order, MAIN_SCAN, true); blockDataUpdateTsWindow(pBlock, pInfo->tsSlotId); doStateWindowAggImpl(pOperator, pInfo, pBlock); @@ -1122,7 +1125,7 @@ static SSDataBlock* doStateWindowAgg(SOperatorInfo* pOperator) { initGroupedResultInfo(&pInfo->groupResInfo, pInfo->aggSup.pResultRowHashTable, TSDB_ORDER_ASC); blockDataEnsureCapacity(pBInfo->pRes, pOperator->resultInfo.capacity); doBuildResultDatablock(pOperator, pBInfo, &pInfo->groupResInfo, pInfo->aggSup.pResultBuf); - if (pBInfo->pRes->info.rows == 0 || !hashRemainDataInGroupInfo(&pInfo->groupResInfo)) { + if (pBInfo->pRes->info.rows == 0 || !hasDataInGroupInfo(&pInfo->groupResInfo)) { doSetOperatorCompleted(pOperator); } @@ -1153,7 +1156,7 @@ static SSDataBlock* doBuildIntervalResult(SOperatorInfo* pOperator) { blockDataEnsureCapacity(pBlock, pOperator->resultInfo.capacity); doBuildResultDatablock(pOperator, &pInfo->binfo, &pInfo->groupResInfo, pInfo->aggSup.pResultBuf); - if (pBlock->info.rows == 0 || !hashRemainDataInGroupInfo(&pInfo->groupResInfo)) { + if (pBlock->info.rows == 0 || !hasDataInGroupInfo(&pInfo->groupResInfo)) { doSetOperatorCompleted(pOperator); } @@ -1166,7 +1169,7 @@ static SSDataBlock* doBuildIntervalResult(SOperatorInfo* pOperator) { // todo merged with the build group result. static void finalizeUpdatedResult(int32_t numOfOutput, SDiskbasedBuf* pBuf, SArray* pUpdateList, - int32_t* rowCellInfoOffset) { + int32_t* rowEntryInfoOffset) { size_t num = taosArrayGetSize(pUpdateList); for (int32_t i = 0; i < num; ++i) { @@ -1176,7 +1179,7 @@ static void finalizeUpdatedResult(int32_t numOfOutput, SDiskbasedBuf* pBuf, SArr SResultRow* pRow = (SResultRow*)((char*)bufPage + pPos->pos.offset); for (int32_t j = 0; j < numOfOutput; ++j) { - SResultRowEntryInfo* pEntry = getResultCell(pRow, j, rowCellInfoOffset); + SResultRowEntryInfo* pEntry = getResultEntryInfo(pRow, j, rowEntryInfoOffset); if (pRow->numOfRows < pEntry->numOfRes) { pRow->numOfRows = pEntry->numOfRes; } @@ -1195,11 +1198,11 @@ static void setInverFunction(SqlFunctionCtx* pCtx, int32_t num, EStreamType type } } -void doClearWindowImpl(SResultRowPosition* p1, SDiskbasedBuf* pResultBuf, SOptrBasicInfo* pBinfo, int32_t numOfOutput) { +void doClearWindowImpl(SResultRowPosition* p1, SDiskbasedBuf* pResultBuf, SExprSupp *pSup, int32_t numOfOutput) { SResultRow* pResult = getResultRowByPos(pResultBuf, p1); - SqlFunctionCtx* pCtx = pBinfo->pCtx; + SqlFunctionCtx* pCtx = pSup->pCtx; for (int32_t i = 0; i < numOfOutput; ++i) { - pCtx[i].resultInfo = getResultCell(pResult, i, pBinfo->rowCellInfoOffset); + pCtx[i].resultInfo = getResultEntryInfo(pResult, i, pSup->rowEntryInfoOffset); struct SResultRowEntryInfo* pResInfo = pCtx[i].resultInfo; if (fmIsWindowPseudoColumnFunc(pCtx[i].functionId)) { continue; @@ -1211,19 +1214,19 @@ void doClearWindowImpl(SResultRowPosition* p1, SDiskbasedBuf* pResultBuf, SOptrB } } -void doClearWindow(SAggSupporter* pSup, SOptrBasicInfo* pBinfo, char* pData, int16_t bytes, uint64_t groupId, +void doClearWindow(SAggSupporter* pAggSup, SExprSupp *pSup, char* pData, int16_t bytes, uint64_t groupId, int32_t numOfOutput) { - SET_RES_WINDOW_KEY(pSup->keyBuf, pData, bytes, groupId); + SET_RES_WINDOW_KEY(pAggSup->keyBuf, pData, bytes, groupId); SResultRowPosition* p1 = - (SResultRowPosition*)taosHashGet(pSup->pResultRowHashTable, pSup->keyBuf, GET_RES_WINDOW_KEY_LEN(bytes)); + (SResultRowPosition*)taosHashGet(pAggSup->pResultRowHashTable, pAggSup->keyBuf, GET_RES_WINDOW_KEY_LEN(bytes)); if (!p1) { // window has been closed return; } - doClearWindowImpl(p1, pSup->pResultBuf, pBinfo, numOfOutput); + doClearWindowImpl(p1, pAggSup->pResultBuf, pSup, numOfOutput); } -static void doClearWindows(SAggSupporter* pSup, SOptrBasicInfo* pBinfo, SInterval* pInterval, int32_t tsIndex, +static void doClearWindows(SAggSupporter* pAggSup, SExprSupp* pSup1, SInterval* pInterval, int32_t tsIndex, int32_t numOfOutput, SSDataBlock* pBlock, SArray* pUpWins) { SColumnInfoData* pColDataInfo = taosArrayGet(pBlock->pDataBlock, tsIndex); TSKEY* tsCols = (TSKEY*)pColDataInfo->pData; @@ -1233,7 +1236,7 @@ static void doClearWindows(SAggSupporter* pSup, SOptrBasicInfo* pBinfo, SInterva dumyInfo.cur.pageId = -1; STimeWindow win = getActiveTimeWindow(NULL, &dumyInfo, tsCols[i], pInterval, pInterval->precision, NULL); step = getNumOfRowsInTimeWindow(&pBlock->info, tsCols, i, win.ekey, binarySearchForKey, NULL, TSDB_ORDER_ASC); - doClearWindow(pSup, pBinfo, (char*)&win.skey, sizeof(TKEY), pBlock->info.groupId, numOfOutput); + doClearWindow(pAggSup, pSup1, (char*)&win.skey, sizeof(TKEY), pBlock->info.groupId, numOfOutput); if (pUpWins) { taosArrayPush(pUpWins, &win); } @@ -1294,6 +1297,7 @@ static SSDataBlock* doStreamIntervalAgg(SOperatorInfo* pOperator) { SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo; pInfo->order = TSDB_ORDER_ASC; + SExprSupp* pSup = &pOperator->exprSupp; if (pOperator->status == OP_EXEC_DONE) { return NULL; @@ -1301,7 +1305,7 @@ static SSDataBlock* doStreamIntervalAgg(SOperatorInfo* pOperator) { if (pOperator->status == OP_RES_TO_RETURN) { doBuildResultDatablock(pOperator, &pInfo->binfo, &pInfo->groupResInfo, pInfo->aggSup.pResultBuf); - if (pInfo->binfo.pRes->info.rows == 0 || !hashRemainDataInGroupInfo(&pInfo->groupResInfo)) { + if (pInfo->binfo.pRes->info.rows == 0 || !hasDataInGroupInfo(&pInfo->groupResInfo)) { pOperator->status = OP_EXEC_DONE; } return pInfo->binfo.pRes->info.rows == 0 ? NULL : pInfo->binfo.pRes; @@ -1317,7 +1321,7 @@ static SSDataBlock* doStreamIntervalAgg(SOperatorInfo* pOperator) { } if (pBlock->info.type == STREAM_REPROCESS) { - doClearWindows(&pInfo->aggSup, &pInfo->binfo, &pInfo->interval, 0, pOperator->numOfExprs, pBlock, NULL); + doClearWindows(&pInfo->aggSup, &pOperator->exprSupp, &pInfo->interval, 0, pOperator->exprSupp.numOfExprs, pBlock, NULL); qDebug("%s clear existed time window results for updates checked", GET_TASKID(pTaskInfo)); continue; } else if (pBlock->info.type == STREAM_GET_ALL) { @@ -1328,9 +1332,9 @@ static SSDataBlock* doStreamIntervalAgg(SOperatorInfo* pOperator) { // The timewindow that overlaps the timestamps of the input pBlock need to be recalculated and return to the // caller. Note that all the time window are not close till now. // the pDataBlock are always the same one, no need to call this again - setInputDataBlock(pOperator, pInfo->binfo.pCtx, pBlock, pInfo->order, MAIN_SCAN, true); + setInputDataBlock(pOperator, pSup->pCtx, pBlock, pInfo->order, MAIN_SCAN, true); if (pInfo->invertible) { - setInverFunction(pInfo->binfo.pCtx, pOperator->numOfExprs, pBlock->info.type); + setInverFunction(pSup->pCtx, pOperator->exprSupp.numOfExprs, pBlock->info.type); } pInfo->twAggSup.maxTs = TMAX(pInfo->twAggSup.maxTs, pBlock->info.window.ekey); @@ -1338,7 +1342,7 @@ static SSDataBlock* doStreamIntervalAgg(SOperatorInfo* pOperator) { } closeIntervalWindow(pInfo->aggSup.pResultRowHashTable, &pInfo->twAggSup, &pInfo->interval, pUpdated); - finalizeUpdatedResult(pOperator->numOfExprs, pInfo->aggSup.pResultBuf, pUpdated, pInfo->binfo.rowCellInfoOffset); + finalizeUpdatedResult(pOperator->exprSupp.numOfExprs, pInfo->aggSup.pResultBuf, pUpdated, pSup->rowEntryInfoOffset); initMultiResInfoFromArrayList(&pInfo->groupResInfo, pUpdated); blockDataEnsureCapacity(pInfo->binfo.pRes, pOperator->resultInfo.capacity); doBuildResultDatablock(pOperator, &pInfo->binfo, &pInfo->groupResInfo, pInfo->aggSup.pResultBuf); @@ -1350,19 +1354,19 @@ static SSDataBlock* doStreamIntervalAgg(SOperatorInfo* pOperator) { static void destroyStateWindowOperatorInfo(void* param, int32_t numOfOutput) { SStateWindowOperatorInfo* pInfo = (SStateWindowOperatorInfo*)param; - doDestroyBasicInfo(&pInfo->binfo, numOfOutput); + cleanupBasicInfo(&pInfo->binfo); taosMemoryFreeClear(pInfo->stateKey.pData); } void destroyIntervalOperatorInfo(void* param, int32_t numOfOutput) { SIntervalAggOperatorInfo* pInfo = (SIntervalAggOperatorInfo*)param; - doDestroyBasicInfo(&pInfo->binfo, numOfOutput); + cleanupBasicInfo(&pInfo->binfo); cleanupAggSup(&pInfo->aggSup); } void destroyStreamFinalIntervalOperatorInfo(void* param, int32_t numOfOutput) { SStreamFinalIntervalOperatorInfo* pInfo = (SStreamFinalIntervalOperatorInfo*)param; - doDestroyBasicInfo(&pInfo->binfo, numOfOutput); + cleanupBasicInfo(&pInfo->binfo); cleanupAggSup(&pInfo->aggSup); if (pInfo->pChildren) { int32_t size = taosArrayGetSize(pInfo->pChildren); @@ -1452,23 +1456,25 @@ SOperatorInfo* createIntervalOperatorInfo(SOperatorInfo* downstream, SExprInfo* pInfo->primaryTsIndex = primaryTsSlotId; + SExprSupp* pSup = &pOperator->exprSupp; + size_t keyBufSize = sizeof(int64_t) + sizeof(int64_t) + POINTER_BYTES; initResultSizeInfo(pOperator, 4096); - int32_t code = - initAggInfo(&pInfo->binfo, &pInfo->aggSup, pExprInfo, numOfCols, pResBlock, keyBufSize, pTaskInfo->id.str); - + int32_t code = initAggInfo(pSup, &pInfo->aggSup, pExprInfo, numOfCols, keyBufSize, pTaskInfo->id.str); + initBasicInfo(&pInfo->binfo, pResBlock); + if (isStream) { ASSERT(numOfCols > 0); - increaseTs(pInfo->binfo.pCtx); + increaseTs(pSup->pCtx); } initExecTimeWindowInfo(&pInfo->twAggSup.timeWindowData, &pInfo->win); - pInfo->invertible = allInvertible(pInfo->binfo.pCtx, numOfCols); + pInfo->invertible = allInvertible(pSup->pCtx, numOfCols); pInfo->invertible = false; // Todo(liuyao): Dependent TSDB API - pInfo->timeWindowInterpo = timeWindowinterpNeeded(pInfo->binfo.pCtx, numOfCols, pInfo); + pInfo->timeWindowInterpo = timeWindowinterpNeeded(pSup->pCtx, numOfCols, pInfo); if (pInfo->timeWindowInterpo) { pInfo->binfo.resultRowInfo.openWindow = tdListNew(sizeof(SResultRowPosition)); if (pInfo->binfo.resultRowInfo.openWindow == NULL) { @@ -1476,15 +1482,15 @@ SOperatorInfo* createIntervalOperatorInfo(SOperatorInfo* downstream, SExprInfo* } } - initResultRowInfo(&pInfo->binfo.resultRowInfo, (int32_t)1); + initResultRowInfo(&pInfo->binfo.resultRowInfo); pOperator->name = "TimeIntervalAggOperator"; pOperator->operatorType = QUERY_NODE_PHYSICAL_PLAN_HASH_INTERVAL; pOperator->blocking = true; pOperator->status = OP_NOT_OPENED; - pOperator->pExpr = pExprInfo; + pOperator->exprSupp.pExprInfo = pExprInfo; pOperator->pTaskInfo = pTaskInfo; - pOperator->numOfExprs = numOfCols; + pOperator->exprSupp.numOfExprs = numOfCols; pOperator->info = pInfo; pOperator->fpSet = createOperatorFpSet(doOpenIntervalAgg, doBuildIntervalResult, doStreamIntervalAgg, NULL, @@ -1525,23 +1531,23 @@ SOperatorInfo* createStreamIntervalOperatorInfo(SOperatorInfo* downstream, SExpr size_t keyBufSize = sizeof(int64_t) + sizeof(int64_t) + POINTER_BYTES; initResultSizeInfo(pOperator, numOfRows); - int32_t code = - initAggInfo(&pInfo->binfo, &pInfo->aggSup, pExprInfo, numOfCols, pResBlock, keyBufSize, pTaskInfo->id.str); + int32_t code = initAggInfo(&pOperator->exprSupp, &pInfo->aggSup, pExprInfo, numOfCols, keyBufSize, pTaskInfo->id.str); + initBasicInfo(&pInfo->binfo, pResBlock); initExecTimeWindowInfo(&pInfo->twAggSup.timeWindowData, &pInfo->win); if (code != TSDB_CODE_SUCCESS) { goto _error; } - initResultRowInfo(&pInfo->binfo.resultRowInfo, (int32_t)1); + initResultRowInfo(&pInfo->binfo.resultRowInfo); pOperator->name = "StreamTimeIntervalAggOperator"; pOperator->operatorType = QUERY_NODE_PHYSICAL_PLAN_HASH_INTERVAL; pOperator->blocking = true; pOperator->status = OP_NOT_OPENED; - pOperator->pExpr = pExprInfo; + pOperator->exprSupp.pExprInfo = pExprInfo; pOperator->pTaskInfo = pTaskInfo; - pOperator->numOfExprs = numOfCols; + pOperator->exprSupp.numOfExprs = numOfCols; pOperator->info = pInfo; pOperator->fpSet = createOperatorFpSet(doOpenIntervalAgg, doStreamIntervalAgg, doStreamIntervalAgg, NULL, @@ -1565,11 +1571,12 @@ _error: // todo handle multiple tables cases. static void doSessionWindowAggImpl(SOperatorInfo* pOperator, SSessionAggOperatorInfo* pInfo, SSDataBlock* pBlock) { SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo; + SExprSupp* pSup = &pOperator->exprSupp; SColumnInfoData* pColInfoData = taosArrayGet(pBlock->pDataBlock, pInfo->tsSlotId); bool masterScan = true; - int32_t numOfOutput = pOperator->numOfExprs; + int32_t numOfOutput = pOperator->exprSupp.numOfExprs; int64_t gid = pBlock->info.groupId; int64_t gap = pInfo->gap; @@ -1602,15 +1609,15 @@ static void doSessionWindowAggImpl(SOperatorInfo* pOperator, SSessionAggOperator pRowSup->win.ekey = pRowSup->win.skey; int32_t ret = - setTimeWindowOutputBuf(&pInfo->binfo.resultRowInfo, &window, masterScan, &pResult, gid, pInfo->binfo.pCtx, - numOfOutput, pInfo->binfo.rowCellInfoOffset, &pInfo->aggSup, pTaskInfo); + setTimeWindowOutputBuf(&pInfo->binfo.resultRowInfo, &window, masterScan, &pResult, gid, pSup->pCtx, + numOfOutput, pSup->rowEntryInfoOffset, &pInfo->aggSup, pTaskInfo); if (ret != TSDB_CODE_SUCCESS) { // null data, too many state code longjmp(pTaskInfo->env, TSDB_CODE_QRY_APP_ERROR); } // pInfo->numOfRows data belong to the current session window updateTimeWindowInfo(&pInfo->twAggSup.timeWindowData, &window, false); - doApplyFunctions(pTaskInfo, pInfo->binfo.pCtx, &window, &pInfo->twAggSup.timeWindowData, pRowSup->startRowIndex, + doApplyFunctions(pTaskInfo, pSup->pCtx, &window, &pInfo->twAggSup.timeWindowData, pRowSup->startRowIndex, pRowSup->numOfRows, NULL, pBlock->info.rows, numOfOutput, TSDB_ORDER_ASC); // here we start a new session window @@ -1622,14 +1629,14 @@ static void doSessionWindowAggImpl(SOperatorInfo* pOperator, SSessionAggOperator SResultRow* pResult = NULL; pRowSup->win.ekey = tsList[pBlock->info.rows - 1]; int32_t ret = - setTimeWindowOutputBuf(&pInfo->binfo.resultRowInfo, &pRowSup->win, masterScan, &pResult, gid, pInfo->binfo.pCtx, - numOfOutput, pInfo->binfo.rowCellInfoOffset, &pInfo->aggSup, pTaskInfo); + setTimeWindowOutputBuf(&pInfo->binfo.resultRowInfo, &pRowSup->win, masterScan, &pResult, gid, pSup->pCtx, + numOfOutput, pSup->rowEntryInfoOffset, &pInfo->aggSup, pTaskInfo); if (ret != TSDB_CODE_SUCCESS) { // null data, too many state code longjmp(pTaskInfo->env, TSDB_CODE_QRY_APP_ERROR); } updateTimeWindowInfo(&pInfo->twAggSup.timeWindowData, &pRowSup->win, false); - doApplyFunctions(pTaskInfo, pInfo->binfo.pCtx, &pRowSup->win, &pInfo->twAggSup.timeWindowData, pRowSup->startRowIndex, + doApplyFunctions(pTaskInfo, pSup->pCtx, &pRowSup->win, &pInfo->twAggSup.timeWindowData, pRowSup->startRowIndex, pRowSup->numOfRows, NULL, pBlock->info.rows, numOfOutput, TSDB_ORDER_ASC); } @@ -1640,10 +1647,11 @@ static SSDataBlock* doSessionWindowAgg(SOperatorInfo* pOperator) { SSessionAggOperatorInfo* pInfo = pOperator->info; SOptrBasicInfo* pBInfo = &pInfo->binfo; + SExprSupp* pSup = &pOperator->exprSupp; if (pOperator->status == OP_RES_TO_RETURN) { doBuildResultDatablock(pOperator, pBInfo, &pInfo->groupResInfo, pInfo->aggSup.pResultBuf); - if (pBInfo->pRes->info.rows == 0 || !hashRemainDataInGroupInfo(&pInfo->groupResInfo)) { + if (pBInfo->pRes->info.rows == 0 || !hasDataInGroupInfo(&pInfo->groupResInfo)) { doSetOperatorCompleted(pOperator); return NULL; } @@ -1663,7 +1671,7 @@ static SSDataBlock* doSessionWindowAgg(SOperatorInfo* pOperator) { } // the pDataBlock are always the same one, no need to call this again - setInputDataBlock(pOperator, pBInfo->pCtx, pBlock, order, MAIN_SCAN, true); + setInputDataBlock(pOperator, pSup->pCtx, pBlock, order, MAIN_SCAN, true); blockDataUpdateTsWindow(pBlock, pInfo->tsSlotId); doSessionWindowAggImpl(pOperator, pInfo, pBlock); @@ -1678,7 +1686,7 @@ static SSDataBlock* doSessionWindowAgg(SOperatorInfo* pOperator) { initGroupedResultInfo(&pInfo->groupResInfo, pInfo->aggSup.pResultRowHashTable, TSDB_ORDER_ASC); blockDataEnsureCapacity(pBInfo->pRes, pOperator->resultInfo.capacity); doBuildResultDatablock(pOperator, pBInfo, &pInfo->groupResInfo, pInfo->aggSup.pResultBuf); - if (pBInfo->pRes->info.rows == 0 || !hashRemainDataInGroupInfo(&pInfo->groupResInfo)) { + if (pBInfo->pRes->info.rows == 0 || !hasDataInGroupInfo(&pInfo->groupResInfo)) { doSetOperatorCompleted(pOperator); } @@ -1711,10 +1719,11 @@ static SSDataBlock* doTimeslice(SOperatorInfo* pOperator) { STimeSliceOperatorInfo* pSliceInfo = pOperator->info; SSDataBlock* pResBlock = pSliceInfo->binfo.pRes; + SExprSupp* pSup = &pOperator->exprSupp; // if (pOperator->status == OP_RES_TO_RETURN) { // // doBuildResultDatablock(&pRuntimeEnv->groupResInfo, pRuntimeEnv, pIntervalInfo->pRes); -// if (pResBlock->info.rows == 0 || !hashRemainDataInGroupInfo(&pSliceInfo->groupResInfo)) { +// if (pResBlock->info.rows == 0 || !hasDataInGroupInfo(&pSliceInfo->groupResInfo)) { // doSetOperatorCompleted(pOperator); // } // @@ -1733,15 +1742,15 @@ static SSDataBlock* doTimeslice(SOperatorInfo* pOperator) { } // the pDataBlock are always the same one, no need to call this again - setInputDataBlock(pOperator, pSliceInfo->binfo.pCtx, pBlock, order, MAIN_SCAN, true); + setInputDataBlock(pOperator, pSup->pCtx, pBlock, order, MAIN_SCAN, true); SColumnInfoData* pTsCol = taosArrayGet(pBlock->pDataBlock, 0); for(int32_t i = 0; i < pBlock->info.rows; ++i) { int64_t ts = *(int64_t*) colDataGetData(pTsCol, i); if (ts == pSliceInfo->current) { - for(int32_t j = 0; j < pOperator->numOfExprs; ++j) { - SExprInfo* pExprInfo = &pOperator->pExpr[j]; + for(int32_t j = 0; j < pOperator->exprSupp.numOfExprs; ++j) { + SExprInfo* pExprInfo = &pOperator->exprSupp.pExprInfo[j]; int32_t dstSlot = pExprInfo->base.resSchema.slotId; int32_t srcSlot = pExprInfo->base.pParam[0].pCol->slotId; @@ -1764,8 +1773,8 @@ static SSDataBlock* doTimeslice(SOperatorInfo* pOperator) { int64_t nextTs = *(int64_t*) colDataGetData(pTsCol, i + 1); if (nextTs > pSliceInfo->current) { // output the result - for (int32_t j = 0; j < pOperator->numOfExprs; ++j) { - SExprInfo* pExprInfo = &pOperator->pExpr[j]; + for (int32_t j = 0; j < pOperator->exprSupp.numOfExprs; ++j) { + SExprInfo* pExprInfo = &pOperator->exprSupp.pExprInfo[j]; int32_t dstSlot = pExprInfo->base.resSchema.slotId; int32_t srcSlot = pExprInfo->base.pParam[0].pCol->slotId; @@ -1903,12 +1912,14 @@ SOperatorInfo* createTimeSliceOperatorInfo(SOperatorInfo* downstream, SExprInfo* goto _error; } - int32_t code = initTimesliceInfo(pInfo, pInfo->binfo.pCtx, numOfCols); + SExprSupp* pSup = &pOperator->exprSupp; + + int32_t code = initTimesliceInfo(pInfo, pSup->pCtx, numOfCols); if (code != TSDB_CODE_SUCCESS) { goto _error; } - initResultRowInfo(&pInfo->binfo.resultRowInfo, 8); + initResultRowInfo(&pInfo->binfo.resultRowInfo); pInfo->pFillColInfo = createFillColInfo(pExprInfo, numOfCols, pValNode); pInfo->binfo.pRes = pResultBlock; @@ -1917,8 +1928,8 @@ SOperatorInfo* createTimeSliceOperatorInfo(SOperatorInfo* downstream, SExprInfo* // pOperator->operatorType = OP_AllTimeWindow; pOperator->blocking = true; pOperator->status = OP_NOT_OPENED; - pOperator->pExpr = pExprInfo; - pOperator->numOfExprs = numOfCols; + pOperator->exprSupp.pExprInfo = pExprInfo; + pOperator->exprSupp.numOfExprs = numOfCols; pOperator->info = pInfo; pOperator->pTaskInfo = pTaskInfo; @@ -1955,8 +1966,10 @@ SOperatorInfo* createStatewindowOperatorInfo(SOperatorInfo* downstream, SExprInf size_t keyBufSize = sizeof(int64_t) + sizeof(int64_t) + POINTER_BYTES; initResultSizeInfo(pOperator, 4096); - initAggInfo(&pInfo->binfo, &pInfo->aggSup, pExpr, numOfCols, pResBlock, keyBufSize, pTaskInfo->id.str); - initResultRowInfo(&pInfo->binfo.resultRowInfo, 8); + initAggInfo(&pOperator->exprSupp, &pInfo->aggSup, pExpr, numOfCols, keyBufSize, pTaskInfo->id.str); + initBasicInfo(&pInfo->binfo, pResBlock); + + initResultRowInfo(&pInfo->binfo.resultRowInfo); pInfo->twAggSup = *pTwAggSup; initExecTimeWindowInfo(&pInfo->twAggSup.timeWindowData, &pTaskInfo->window); @@ -1966,8 +1979,8 @@ SOperatorInfo* createStatewindowOperatorInfo(SOperatorInfo* downstream, SExprInf pOperator->operatorType = QUERY_NODE_PHYSICAL_PLAN_MERGE_STATE; pOperator->blocking = true; pOperator->status = OP_NOT_OPENED; - pOperator->pExpr = pExpr; - pOperator->numOfExprs = numOfCols; + pOperator->exprSupp.pExprInfo = pExpr; + pOperator->exprSupp.numOfExprs = numOfCols; pOperator->pTaskInfo = pTaskInfo; pOperator->info = pInfo; @@ -1984,7 +1997,7 @@ _error: void destroySWindowOperatorInfo(void* param, int32_t numOfOutput) { SSessionAggOperatorInfo* pInfo = (SSessionAggOperatorInfo*)param; - doDestroyBasicInfo(&pInfo->binfo, numOfOutput); + cleanupBasicInfo(&pInfo->binfo); } SOperatorInfo* createSessionAggOperatorInfo(SOperatorInfo* downstream, SExprInfo* pExprInfo, int32_t numOfCols, @@ -1999,14 +2012,15 @@ SOperatorInfo* createSessionAggOperatorInfo(SOperatorInfo* downstream, SExprInfo size_t keyBufSize = sizeof(int64_t) + sizeof(int64_t) + POINTER_BYTES; initResultSizeInfo(pOperator, 4096); - int32_t code = - initAggInfo(&pInfo->binfo, &pInfo->aggSup, pExprInfo, numOfCols, pResBlock, keyBufSize, pTaskInfo->id.str); + int32_t code = initAggInfo(&pOperator->exprSupp, &pInfo->aggSup, pExprInfo, numOfCols, keyBufSize, pTaskInfo->id.str); if (code != TSDB_CODE_SUCCESS) { goto _error; } + initBasicInfo(&pInfo->binfo, pResBlock); + pInfo->twAggSup = *pTwAggSupp; - initResultRowInfo(&pInfo->binfo.resultRowInfo, 8); + initResultRowInfo(&pInfo->binfo.resultRowInfo); initExecTimeWindowInfo(&pInfo->twAggSup.timeWindowData, &pTaskInfo->window); pInfo->tsSlotId = tsSlotId; @@ -2018,8 +2032,8 @@ SOperatorInfo* createSessionAggOperatorInfo(SOperatorInfo* downstream, SExprInfo pOperator->operatorType = QUERY_NODE_PHYSICAL_PLAN_MERGE_SESSION; pOperator->blocking = true; pOperator->status = OP_NOT_OPENED; - pOperator->pExpr = pExprInfo; - pOperator->numOfExprs = numOfCols; + pOperator->exprSupp.pExprInfo = pExprInfo; + pOperator->exprSupp.numOfExprs = numOfCols; pOperator->info = pInfo; pOperator->fpSet = createOperatorFpSet(operatorDummyOpenFn, doSessionWindowAgg, NULL, NULL, @@ -2060,23 +2074,25 @@ void compactFunctions(SqlFunctionCtx* pDestCtx, SqlFunctionCtx* pSourceCtx, int3 } } -static void rebuildIntervalWindow(SStreamFinalIntervalOperatorInfo* pInfo, SArray* pWinArray, int32_t groupId, +static void rebuildIntervalWindow(SStreamFinalIntervalOperatorInfo* pInfo, SExprSupp* pSup, SArray* pWinArray, int32_t groupId, int32_t numOfOutput, SExecTaskInfo* pTaskInfo) { int32_t size = taosArrayGetSize(pWinArray); ASSERT(pInfo->pChildren); for (int32_t i = 0; i < size; i++) { STimeWindow* pParentWin = taosArrayGet(pWinArray, i); SResultRow* pCurResult = NULL; - setTimeWindowOutputBuf(&pInfo->binfo.resultRowInfo, pParentWin, true, &pCurResult, 0, pInfo->binfo.pCtx, - numOfOutput, pInfo->binfo.rowCellInfoOffset, &pInfo->aggSup, pTaskInfo); + setTimeWindowOutputBuf(&pInfo->binfo.resultRowInfo, pParentWin, true, &pCurResult, 0, pSup->pCtx, + numOfOutput, pSup->rowEntryInfoOffset, &pInfo->aggSup, pTaskInfo); int32_t numOfChildren = taosArrayGetSize(pInfo->pChildren); for (int32_t j = 0; j < numOfChildren; j++) { SOperatorInfo* pChildOp = taosArrayGetP(pInfo->pChildren, j); SIntervalAggOperatorInfo* pChInfo = pChildOp->info; + SExprSupp* pChildSup = &pChildOp->exprSupp; + SResultRow* pChResult = NULL; - setTimeWindowOutputBuf(&pChInfo->binfo.resultRowInfo, pParentWin, true, &pChResult, 0, pChInfo->binfo.pCtx, - pChildOp->numOfExprs, pChInfo->binfo.rowCellInfoOffset, &pChInfo->aggSup, pTaskInfo); - compactFunctions(pInfo->binfo.pCtx, pChInfo->binfo.pCtx, numOfOutput, pTaskInfo); + setTimeWindowOutputBuf(&pChInfo->binfo.resultRowInfo, pParentWin, true, &pChResult, 0, pChildSup->pCtx, + pChildSup->numOfExprs, pChildSup->rowEntryInfoOffset, &pChInfo->aggSup, pTaskInfo); + compactFunctions(pSup->pCtx, pChildSup->pCtx, numOfOutput, pTaskInfo); } } } @@ -2093,7 +2109,8 @@ static void doHashInterval(SOperatorInfo* pOperatorInfo, SSDataBlock* pSDataBloc SStreamFinalIntervalOperatorInfo* pInfo = (SStreamFinalIntervalOperatorInfo*)pOperatorInfo->info; SResultRowInfo* pResultRowInfo = &(pInfo->binfo.resultRowInfo); SExecTaskInfo* pTaskInfo = pOperatorInfo->pTaskInfo; - int32_t numOfOutput = pOperatorInfo->numOfExprs; + SExprSupp* pSup = &pOperatorInfo->exprSupp; + int32_t numOfOutput = pSup->numOfExprs; int32_t step = 1; bool ascScan = true; TSKEY* tsCols = NULL; @@ -2116,12 +2133,11 @@ static void doHashInterval(SOperatorInfo* pOperatorInfo, SSDataBlock* pSDataBloc isDeletedWindow(&nextWin, tableGroupId, &pInfo->aggSup)) { SArray* pUpWins = taosArrayInit(8, sizeof(STimeWindow)); taosArrayPush(pUpWins, &nextWin); - rebuildIntervalWindow(pInfo, pUpWins, pInfo->binfo.pRes->info.groupId, - pOperatorInfo->numOfExprs, pOperatorInfo->pTaskInfo); + rebuildIntervalWindow(pInfo, pSup, pUpWins, pInfo->binfo.pRes->info.groupId, pSup->numOfExprs, pOperatorInfo->pTaskInfo); taosArrayDestroy(pUpWins); } - int32_t code = setTimeWindowOutputBuf(pResultRowInfo, &nextWin, true, &pResult, tableGroupId, pInfo->binfo.pCtx, - numOfOutput, pInfo->binfo.rowCellInfoOffset, &pInfo->aggSup, pTaskInfo); + int32_t code = setTimeWindowOutputBuf(pResultRowInfo, &nextWin, true, &pResult, tableGroupId, pSup->pCtx, + numOfOutput, pSup->rowEntryInfoOffset, &pInfo->aggSup, pTaskInfo); if (code != TSDB_CODE_SUCCESS || pResult == NULL) { longjmp(pTaskInfo->env, TSDB_CODE_QRY_OUT_OF_MEMORY); } @@ -2135,10 +2151,10 @@ static void doHashInterval(SOperatorInfo* pOperatorInfo, SSDataBlock* pSDataBloc saveResultRow(pResult, tableGroupId, pUpdated); } // window start(end) key interpolation - // doWindowBorderInterpolation(pInfo, pSDataBlock, numOfOutput, pInfo->binfo.pCtx, pResult, &nextWin, startPos, + // doWindowBorderInterpolation(pInfo, pSDataBlock, numOfOutput, pSup->pCtx, pResult, &nextWin, startPos, // forwardRows); updateTimeWindowInfo(&pInfo->twAggSup.timeWindowData, &nextWin, true); - doApplyFunctions(pTaskInfo, pInfo->binfo.pCtx, &nextWin, &pInfo->twAggSup.timeWindowData, startPos, forwardRows, + doApplyFunctions(pTaskInfo, pSup->pCtx, &nextWin, &pInfo->twAggSup.timeWindowData, startPos, forwardRows, tsCols, pSDataBlock->info.rows, numOfOutput, TSDB_ORDER_ASC); int32_t prevEndPos = (forwardRows - 1) * step + startPos; ASSERT(pSDataBlock->info.window.skey > 0 && pSDataBlock->info.window.ekey > 0); @@ -2153,7 +2169,7 @@ static void clearStreamIntervalOperator(SStreamFinalIntervalOperatorInfo* pInfo) taosHashClear(pInfo->aggSup.pResultRowHashTable); clearDiskbasedBuf(pInfo->aggSup.pResultBuf); cleanupResultRowInfo(&pInfo->binfo.resultRowInfo); - initResultRowInfo(&pInfo->binfo.resultRowInfo, 1); + initResultRowInfo(&pInfo->binfo.resultRowInfo); } static void clearUpdateDataBlock(SSDataBlock* pBlock) { @@ -2190,6 +2206,8 @@ static SSDataBlock* doStreamFinalIntervalAgg(SOperatorInfo* pOperator) { SArray* pUpdated = taosArrayInit(4, POINTER_BYTES); TSKEY maxTs = INT64_MIN; + SExprSupp* pSup = &pOperator->exprSupp; + if (pOperator->status == OP_EXEC_DONE) { return NULL; } else if (pOperator->status == OP_RES_TO_RETURN) { @@ -2219,15 +2237,17 @@ static SSDataBlock* doStreamFinalIntervalAgg(SOperatorInfo* pOperator) { if (pBlock->info.type == STREAM_REPROCESS) { SArray* pUpWins = taosArrayInit(8, sizeof(STimeWindow)); - doClearWindows(&pInfo->aggSup, &pInfo->binfo, &pInfo->interval, pInfo->primaryTsIndex, pOperator->numOfExprs, + doClearWindows(&pInfo->aggSup, pSup, &pInfo->interval, pInfo->primaryTsIndex, pOperator->exprSupp.numOfExprs, pBlock, pUpWins); if (isFinalInterval(pInfo)) { int32_t childIndex = getChildIndex(pBlock); SOperatorInfo* pChildOp = taosArrayGetP(pInfo->pChildren, childIndex); SIntervalAggOperatorInfo* pChildInfo = pChildOp->info; - doClearWindows(&pChildInfo->aggSup, &pChildInfo->binfo, &pChildInfo->interval, pChildInfo->primaryTsIndex, - pChildOp->numOfExprs, pBlock, NULL); - rebuildIntervalWindow(pInfo, pUpWins, pInfo->binfo.pRes->info.groupId, pOperator->numOfExprs, + SExprSupp* pChildSup = &pChildOp->exprSupp; + + doClearWindows(&pChildInfo->aggSup, pChildSup, &pChildInfo->interval, pChildInfo->primaryTsIndex, + pChildSup->numOfExprs, pBlock, NULL); + rebuildIntervalWindow(pInfo, pSup, pUpWins, pInfo->binfo.pRes->info.groupId, pOperator->exprSupp.numOfExprs, pOperator->pTaskInfo); taosArrayDestroy(pUpWins); continue; @@ -2241,7 +2261,7 @@ static SSDataBlock* doStreamFinalIntervalAgg(SOperatorInfo* pOperator) { continue; } - setInputDataBlock(pOperator, pInfo->binfo.pCtx, pBlock, pInfo->order, MAIN_SCAN, true); + setInputDataBlock(pOperator, pSup->pCtx, pBlock, pInfo->order, MAIN_SCAN, true); doHashInterval(pOperator, pBlock, pBlock->info.groupId, pUpdated); if (isFinalInterval(pInfo)) { int32_t chIndex = getChildIndex(pBlock); @@ -2256,7 +2276,7 @@ static SSDataBlock* doStreamFinalIntervalAgg(SOperatorInfo* pOperator) { } SOperatorInfo* pChildOp = taosArrayGetP(pInfo->pChildren, chIndex); SStreamFinalIntervalOperatorInfo* pChInfo = pChildOp->info; - setInputDataBlock(pChildOp, pChInfo->binfo.pCtx, pBlock, pChInfo->order, MAIN_SCAN, true); + setInputDataBlock(pChildOp, pChildOp->exprSupp.pCtx, pBlock, pChInfo->order, MAIN_SCAN, true); doHashInterval(pChildOp, pBlock, pBlock->info.groupId, NULL); } maxTs = TMAX(maxTs, pBlock->info.window.ekey); @@ -2267,7 +2287,7 @@ static SSDataBlock* doStreamFinalIntervalAgg(SOperatorInfo* pOperator) { closeIntervalWindow(pInfo->aggSup.pResultRowHashTable, &pInfo->twAggSup, &pInfo->interval, pUpdated); } - finalizeUpdatedResult(pOperator->numOfExprs, pInfo->aggSup.pResultBuf, pUpdated, pInfo->binfo.rowCellInfoOffset); + finalizeUpdatedResult(pOperator->exprSupp.numOfExprs, pInfo->aggSup.pResultBuf, pUpdated, pSup->rowEntryInfoOffset); initMultiResInfoFromArrayList(&pInfo->groupResInfo, pUpdated); blockDataEnsureCapacity(pInfo->binfo.pRes, pOperator->resultInfo.capacity); doBuildResultDatablock(pOperator, &pInfo->binfo, &pInfo->groupResInfo, pInfo->aggSup.pResultBuf); @@ -2292,6 +2312,7 @@ SOperatorInfo* createStreamFinalIntervalOperatorInfo(SOperatorInfo* downstream, if (pInfo == NULL || pOperator == NULL) { goto _error; } + pInfo->order = TSDB_ORDER_ASC; pInfo->interval = (SInterval){.interval = pIntervalPhyNode->interval, .sliding = pIntervalPhyNode->sliding, @@ -2311,15 +2332,17 @@ SOperatorInfo* createStreamFinalIntervalOperatorInfo(SOperatorInfo* downstream, int32_t numOfCols = 0; SExprInfo* pExprInfo = createExprInfo(pIntervalPhyNode->window.pFuncs, NULL, &numOfCols); SSDataBlock* pResBlock = createResDataBlock(pPhyNode->pOutputDataBlockDesc); - int32_t code = initAggInfo(&pInfo->binfo, &pInfo->aggSup, pExprInfo, numOfCols, - pResBlock, keyBufSize, pTaskInfo->id.str); + + int32_t code = initAggInfo(&pOperator->exprSupp, &pInfo->aggSup, pExprInfo, numOfCols, keyBufSize, pTaskInfo->id.str); + initBasicInfo(&pInfo->binfo, pResBlock); + ASSERT(numOfCols > 0); - increaseTs(pInfo->binfo.pCtx); + increaseTs(pOperator->exprSupp.pCtx); initExecTimeWindowInfo(&pInfo->twAggSup.timeWindowData, &pTaskInfo->window); if (code != TSDB_CODE_SUCCESS) { goto _error; } - initResultRowInfo(&pInfo->binfo.resultRowInfo, (int32_t)1); + initResultRowInfo(&pInfo->binfo.resultRowInfo); pInfo->pChildren = NULL; if (numOfChild > 0) { pInfo->pChildren = taosArrayInit(numOfChild, sizeof(SOperatorInfo)); @@ -2345,9 +2368,9 @@ SOperatorInfo* createStreamFinalIntervalOperatorInfo(SOperatorInfo* downstream, pOperator->operatorType = QUERY_NODE_PHYSICAL_PLAN_STREAM_FINAL_INTERVAL; pOperator->blocking = true; pOperator->status = OP_NOT_OPENED; - pOperator->pExpr = pExprInfo; + pOperator->exprSupp.pExprInfo = pExprInfo; pOperator->pTaskInfo = pTaskInfo; - pOperator->numOfExprs = numOfCols; + pOperator->exprSupp.numOfExprs = numOfCols; pOperator->info = pInfo; pOperator->fpSet = @@ -2382,7 +2405,7 @@ void destroyStreamAggSupporter(SStreamAggSupporter* pSup) { void destroyStreamSessionAggOperatorInfo(void* param, int32_t numOfOutput) { SStreamSessionAggOperatorInfo* pInfo = (SStreamSessionAggOperatorInfo*)param; - doDestroyBasicInfo(&pInfo->binfo, numOfOutput); + cleanupBasicInfo(&pInfo->binfo); destroyStreamAggSupporter(&pInfo->streamAggSup); cleanupGroupResInfo(&pInfo->groupResInfo); if (pInfo->pChildren != NULL) { @@ -2397,14 +2420,16 @@ void destroyStreamSessionAggOperatorInfo(void* param, int32_t numOfOutput) { } } -int32_t initBiasicInfo(SOptrBasicInfo* pBasicInfo, SExprInfo* pExprInfo, int32_t numOfCols, SSDataBlock* pResultBlock) { - pBasicInfo->pCtx = createSqlFunctionCtx(pExprInfo, numOfCols, &pBasicInfo->rowCellInfoOffset); - pBasicInfo->pRes = pResultBlock; +int32_t initBasicInfoEx(SOptrBasicInfo* pBasicInfo, SExprSupp* pSup, SExprInfo* pExprInfo, int32_t numOfCols, SSDataBlock* pResultBlock) { + initExprSupp(pSup, pExprInfo, numOfCols); + initBasicInfo(pBasicInfo, pResultBlock); + for (int32_t i = 0; i < numOfCols; ++i) { - pBasicInfo->pCtx[i].pBuf = NULL; + pSup->pCtx[i].pBuf = NULL; } + ASSERT(numOfCols > 0); - increaseTs(pBasicInfo->pCtx); + increaseTs(pSup->pCtx); return TSDB_CODE_SUCCESS; } @@ -2425,24 +2450,29 @@ int32_t initSessionAggSupporter(SStreamAggSupporter* pSup, const char* pKey, Sql return initStreamAggSupporter(pSup, pKey, pCtx, numOfOutput, sizeof(SResultWindowInfo)); } -SOperatorInfo* createStreamSessionAggOperatorInfo(SOperatorInfo* downstream, SExprInfo* pExprInfo, int32_t numOfCols, - SSDataBlock* pResBlock, int64_t gap, int32_t tsSlotId, - STimeWindowAggSupp* pTwAggSupp, SExecTaskInfo* pTaskInfo) { - int32_t code = TSDB_CODE_OUT_OF_MEMORY; +SOperatorInfo* createStreamSessionAggOperatorInfo(SOperatorInfo* downstream, SPhysiNode* pPhyNode, SExecTaskInfo* pTaskInfo) { + SSessionWinodwPhysiNode* pSessionNode = (SSessionWinodwPhysiNode*)pPhyNode; + int32_t numOfCols = 0; + SExprInfo* pExprInfo = createExprInfo(pSessionNode->window.pFuncs, NULL, &numOfCols); + SSDataBlock* pResBlock = createResDataBlock(pPhyNode->pOutputDataBlockDesc); + int32_t tsSlotId = ((SColumnNode*)pSessionNode->window.pTspk)->slotId; + int32_t code = TSDB_CODE_OUT_OF_MEMORY; SStreamSessionAggOperatorInfo* pInfo = taosMemoryCalloc(1, sizeof(SStreamSessionAggOperatorInfo)); - SOperatorInfo* pOperator = taosMemoryCalloc(1, sizeof(SOperatorInfo)); + SOperatorInfo* pOperator = taosMemoryCalloc(1, sizeof(SOperatorInfo)); if (pInfo == NULL || pOperator == NULL) { goto _error; } initResultSizeInfo(pOperator, 4096); + SExprSupp* pSup = &pOperator->exprSupp; - code = initBiasicInfo(&pInfo->binfo, pExprInfo, numOfCols, pResBlock); + code = initBasicInfoEx(&pInfo->binfo, pSup, pExprInfo, numOfCols, pResBlock); if (code != TSDB_CODE_SUCCESS) { goto _error; } - code = initSessionAggSupporter(&pInfo->streamAggSup, "StreamSessionAggOperatorInfo", pInfo->binfo.pCtx, numOfCols); + + code = initSessionAggSupporter(&pInfo->streamAggSup, "StreamSessionAggOperatorInfo", pSup->pCtx, numOfCols); if (code != TSDB_CODE_SUCCESS) { goto _error; } @@ -2451,14 +2481,18 @@ SOperatorInfo* createStreamSessionAggOperatorInfo(SOperatorInfo* downstream, SEx if (pInfo->pDummyCtx == NULL) { goto _error; } - initDummyFunction(pInfo->pDummyCtx, pInfo->binfo.pCtx, numOfCols); + initDummyFunction(pInfo->pDummyCtx, pSup->pCtx, numOfCols); - pInfo->twAggSup = *pTwAggSupp; - initResultRowInfo(&pInfo->binfo.resultRowInfo, 8); + pInfo->twAggSup = (STimeWindowAggSupp) { + .waterMark = pSessionNode->window.watermark, + .calTrigger = pSessionNode->window.triggerType, + .maxTs = INT64_MIN}; + + initResultRowInfo(&pInfo->binfo.resultRowInfo); initExecTimeWindowInfo(&pInfo->twAggSup.timeWindowData, &pTaskInfo->window); pInfo->primaryTsIndex = tsSlotId; - pInfo->gap = gap; + pInfo->gap = pSessionNode->gap; pInfo->binfo.pRes = pResBlock; pInfo->order = TSDB_ORDER_ASC; _hash_fn_t hashFn = taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY); @@ -2472,8 +2506,8 @@ SOperatorInfo* createStreamSessionAggOperatorInfo(SOperatorInfo* downstream, SEx pOperator->operatorType = QUERY_NODE_PHYSICAL_PLAN_STREAM_SESSION; pOperator->blocking = true; pOperator->status = OP_NOT_OPENED; - pOperator->pExpr = pExprInfo; - pOperator->numOfExprs = numOfCols; + pOperator->exprSupp.pExprInfo = pExprInfo; + pOperator->exprSupp.numOfExprs = numOfCols; pOperator->info = pInfo; pOperator->fpSet = createOperatorFpSet(operatorDummyOpenFn, doStreamSessionAgg, NULL, NULL, destroyStreamSessionAggOperatorInfo, @@ -2585,7 +2619,7 @@ int32_t updateSessionWindowInfo(SResultWindowInfo* pWinInfo, TSKEY* pTs, int32_t } static int32_t setWindowOutputBuf(SResultWindowInfo* pWinInfo, SResultRow** pResult, SqlFunctionCtx* pCtx, - uint64_t groupId, int32_t numOfOutput, int32_t* rowCellInfoOffset, + uint64_t groupId, int32_t numOfOutput, int32_t* rowEntryInfoOffset, SStreamAggSupporter* pAggSup, SExecTaskInfo* pTaskInfo) { assert(pWinInfo->win.skey <= pWinInfo->win.ekey); // too many time window in query @@ -2614,39 +2648,42 @@ static int32_t setWindowOutputBuf(SResultWindowInfo* pWinInfo, SResultRow** pRes // set time window for current result (*pResult)->win = pWinInfo->win; - setResultRowInitCtx(*pResult, pCtx, numOfOutput, rowCellInfoOffset); + setResultRowInitCtx(*pResult, pCtx, numOfOutput, rowEntryInfoOffset); return TSDB_CODE_SUCCESS; } static int32_t doOneWindowAggImpl(int32_t tsColId, SOptrBasicInfo* pBinfo, SStreamAggSupporter* pAggSup, SColumnInfoData* pTimeWindowData, SSDataBlock* pSDataBlock, SResultWindowInfo* pCurWin, SResultRow** pResult, int32_t startIndex, int32_t winRows, - int32_t numOutput, SExecTaskInfo* pTaskInfo) { + int32_t numOutput, SOperatorInfo* pOperator) { + SExprSupp* pSup = &pOperator->exprSupp; + SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo; + SColumnInfoData* pColDataInfo = taosArrayGet(pSDataBlock->pDataBlock, tsColId); TSKEY* tsCols = (int64_t*)pColDataInfo->pData; - int32_t code = setWindowOutputBuf(pCurWin, pResult, pBinfo->pCtx, pSDataBlock->info.groupId, numOutput, - pBinfo->rowCellInfoOffset, pAggSup, pTaskInfo); + int32_t code = setWindowOutputBuf(pCurWin, pResult, pSup->pCtx, pSDataBlock->info.groupId, numOutput, + pSup->rowEntryInfoOffset, pAggSup, pTaskInfo); if (code != TSDB_CODE_SUCCESS || (*pResult) == NULL) { return TSDB_CODE_QRY_OUT_OF_MEMORY; } updateTimeWindowInfo(pTimeWindowData, &pCurWin->win, true); - doApplyFunctions(pTaskInfo, pBinfo->pCtx, &pCurWin->win, pTimeWindowData, startIndex, winRows, tsCols, + doApplyFunctions(pTaskInfo, pSup->pCtx, &pCurWin->win, pTimeWindowData, startIndex, winRows, tsCols, pSDataBlock->info.rows, numOutput, TSDB_ORDER_ASC); return TSDB_CODE_SUCCESS; } static int32_t doOneWindowAgg(SStreamSessionAggOperatorInfo* pInfo, SSDataBlock* pSDataBlock, SResultWindowInfo* pCurWin, SResultRow** pResult, int32_t startIndex, int32_t winRows, - int32_t numOutput, SExecTaskInfo* pTaskInfo) { + int32_t numOutput, SOperatorInfo * pOperator) { return doOneWindowAggImpl(pInfo->primaryTsIndex, &pInfo->binfo, &pInfo->streamAggSup, &pInfo->twAggSup.timeWindowData, - pSDataBlock, pCurWin, pResult, startIndex, winRows, numOutput, pTaskInfo); + pSDataBlock, pCurWin, pResult, startIndex, winRows, numOutput, pOperator); } static int32_t doOneStateWindowAgg(SStreamStateAggOperatorInfo* pInfo, SSDataBlock* pSDataBlock, SResultWindowInfo* pCurWin, SResultRow** pResult, int32_t startIndex, - int32_t winRows, int32_t numOutput, SExecTaskInfo* pTaskInfo) { + int32_t winRows, int32_t numOutput, SOperatorInfo * pOperator) { return doOneWindowAggImpl(pInfo->primaryTsIndex, &pInfo->binfo, &pInfo->streamAggSup, &pInfo->twAggSup.timeWindowData, - pSDataBlock, pCurWin, pResult, startIndex, winRows, numOutput, pTaskInfo); + pSDataBlock, pCurWin, pResult, startIndex, winRows, numOutput, pOperator); } int32_t getNumCompactWindow(SArray* pWinInfos, int32_t startIndex, int64_t gap) { @@ -2664,10 +2701,13 @@ int32_t getNumCompactWindow(SArray* pWinInfos, int32_t startIndex, int64_t gap) } void compactTimeWindow(SStreamSessionAggOperatorInfo* pInfo, int32_t startIndex, int32_t num, uint64_t groupId, - int32_t numOfOutput, SExecTaskInfo* pTaskInfo, SHashObj* pStUpdated, SHashObj* pStDeleted) { + int32_t numOfOutput, SHashObj* pStUpdated, SHashObj* pStDeleted, SOperatorInfo* pOperator) { + SExprSupp* pSup = &pOperator->exprSupp; + SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo; + SResultWindowInfo* pCurWin = taosArrayGet(pInfo->streamAggSup.pCurWins, startIndex); SResultRow* pCurResult = NULL; - setWindowOutputBuf(pCurWin, &pCurResult, pInfo->binfo.pCtx, groupId, numOfOutput, pInfo->binfo.rowCellInfoOffset, + setWindowOutputBuf(pCurWin, &pCurResult, pSup->pCtx, groupId, numOfOutput, pSup->rowEntryInfoOffset, &pInfo->streamAggSup, pTaskInfo); num += startIndex + 1; ASSERT(num <= taosArrayGetSize(pInfo->streamAggSup.pCurWins)); @@ -2675,10 +2715,10 @@ void compactTimeWindow(SStreamSessionAggOperatorInfo* pInfo, int32_t startIndex, for (int32_t i = startIndex + 1; i < num; i++) { SResultWindowInfo* pWinInfo = taosArrayGet(pInfo->streamAggSup.pCurWins, i); SResultRow* pWinResult = NULL; - setWindowOutputBuf(pWinInfo, &pWinResult, pInfo->pDummyCtx, groupId, numOfOutput, pInfo->binfo.rowCellInfoOffset, + setWindowOutputBuf(pWinInfo, &pWinResult, pInfo->pDummyCtx, groupId, numOfOutput, pSup->rowEntryInfoOffset, &pInfo->streamAggSup, pTaskInfo); pCurWin->win.ekey = TMAX(pCurWin->win.ekey, pWinInfo->win.ekey); - compactFunctions(pInfo->binfo.pCtx, pInfo->pDummyCtx, numOfOutput, pTaskInfo); + compactFunctions(pSup->pCtx, pInfo->pDummyCtx, numOfOutput, pTaskInfo); taosHashRemove(pStUpdated, &pWinInfo->pos, sizeof(SResultRowPosition)); if (pWinInfo->isOutput) { taosHashPut(pStDeleted, &pWinInfo->pos, sizeof(SResultRowPosition), &pWinInfo->win.skey, sizeof(TSKEY)); @@ -2698,7 +2738,7 @@ static void doStreamSessionAggImpl(SOperatorInfo* pOperator, SSDataBlock* pSData SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo; SStreamSessionAggOperatorInfo* pInfo = pOperator->info; bool masterScan = true; - int32_t numOfOutput = pOperator->numOfExprs; + int32_t numOfOutput = pOperator->exprSupp.numOfExprs; uint64_t groupId = pSDataBlock->info.groupId; int64_t gap = pInfo->gap; int64_t code = TSDB_CODE_SUCCESS; @@ -2721,17 +2761,17 @@ static void doStreamSessionAggImpl(SOperatorInfo* pOperator, SSDataBlock* pSData int32_t winIndex = 0; SResultWindowInfo* pCurWin = getSessionTimeWindow(pAggSup, tsCols[i], groupId, gap, &winIndex); winRows = updateSessionWindowInfo(pCurWin, tsCols, pSDataBlock->info.rows, i, pInfo->gap, pStDeleted); - code = doOneWindowAgg(pInfo, pSDataBlock, pCurWin, &pResult, i, winRows, numOfOutput, pTaskInfo); + code = doOneWindowAgg(pInfo, pSDataBlock, pCurWin, &pResult, i, winRows, numOfOutput, pOperator); if (code != TSDB_CODE_SUCCESS || pResult == NULL) { longjmp(pTaskInfo->env, TSDB_CODE_QRY_OUT_OF_MEMORY); } // window start(end) key interpolation - // doWindowBorderInterpolation(pOperatorInfo, pSDataBlock, pInfo->binfo.pCtx, pResult, &nextWin, startPos, + // doWindowBorderInterpolation(pOperatorInfo, pSDataBlock, pSup->pCtx, pResult, &nextWin, startPos, // forwardRows, // pInfo->order, false); int32_t winNum = getNumCompactWindow(pAggSup->pCurWins, winIndex, gap); if (winNum > 0) { - compactTimeWindow(pInfo, winIndex, winNum, groupId, numOfOutput, pTaskInfo, pStUpdated, pStDeleted); + compactTimeWindow(pInfo, winIndex, winNum, groupId, numOfOutput, pStUpdated, pStDeleted, pOperator); } pCurWin->isClosed = false; if (pInfo->twAggSup.calTrigger == STREAM_TRIGGER_AT_ONCE) { @@ -2746,7 +2786,7 @@ static void doStreamSessionAggImpl(SOperatorInfo* pOperator, SSDataBlock* pSData } } -static void doClearSessionWindows(SStreamAggSupporter* pAggSup, SOptrBasicInfo* pBinfo, SSDataBlock* pBlock, +static void doClearSessionWindows(SStreamAggSupporter* pAggSup, SExprSupp* pSup, SSDataBlock* pBlock, int32_t tsIndex, int32_t numOfOutput, int64_t gap, SArray* result) { SColumnInfoData* pColDataInfo = taosArrayGet(pBlock->pDataBlock, tsIndex); TSKEY* tsCols = (TSKEY*)pColDataInfo->pData; @@ -2756,7 +2796,7 @@ static void doClearSessionWindows(SStreamAggSupporter* pAggSup, SOptrBasicInfo* SResultWindowInfo* pCurWin = getSessionTimeWindow(pAggSup, tsCols[i], pBlock->info.groupId, gap, &winIndex); step = updateSessionWindowInfo(pCurWin, tsCols, pBlock->info.rows, i, gap, NULL); ASSERT(isInWindow(pCurWin, tsCols[i], gap)); - doClearWindowImpl(&pCurWin->pos, pAggSup->pResultBuf, pBinfo, numOfOutput); + doClearWindowImpl(&pCurWin->pos, pAggSup->pResultBuf, pSup, numOfOutput); if (result) { taosArrayPush(result, pCurWin); } @@ -2802,13 +2842,17 @@ void doBuildDeleteDataBlock(SHashObj* pStDeleted, SSDataBlock* pBlock, void** It } static void rebuildTimeWindow(SStreamSessionAggOperatorInfo* pInfo, SArray* pWinArray, int32_t groupId, - int32_t numOfOutput, SExecTaskInfo* pTaskInfo) { + int32_t numOfOutput, SOperatorInfo* pOperator) { + SExprSupp* pSup = &pOperator->exprSupp; + SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo; + int32_t size = taosArrayGetSize(pWinArray); ASSERT(pInfo->pChildren); + for (int32_t i = 0; i < size; i++) { SResultWindowInfo* pParentWin = taosArrayGet(pWinArray, i); SResultRow* pCurResult = NULL; - setWindowOutputBuf(pParentWin, &pCurResult, pInfo->binfo.pCtx, groupId, numOfOutput, pInfo->binfo.rowCellInfoOffset, + setWindowOutputBuf(pParentWin, &pCurResult, pSup->pCtx, groupId, numOfOutput, pSup->rowEntryInfoOffset, &pInfo->streamAggSup, pTaskInfo); int32_t numOfChildren = taosArrayGetSize(pInfo->pChildren); for (int32_t j = 0; j < numOfChildren; j++) { @@ -2821,9 +2865,9 @@ static void rebuildTimeWindow(SStreamSessionAggOperatorInfo* pInfo, SArray* pWin SResultWindowInfo* pcw = taosArrayGet(pChWins, k); if (pParentWin->win.skey <= pcw->win.skey && pcw->win.ekey <= pParentWin->win.ekey) { SResultRow* pChResult = NULL; - setWindowOutputBuf(pcw, &pChResult, pChInfo->binfo.pCtx, groupId, numOfOutput, - pChInfo->binfo.rowCellInfoOffset, &pChInfo->streamAggSup, pTaskInfo); - compactFunctions(pInfo->binfo.pCtx, pChInfo->binfo.pCtx, numOfOutput, pTaskInfo); + setWindowOutputBuf(pcw, &pChResult, pChild->exprSupp.pCtx, groupId, numOfOutput, + pChild->exprSupp.rowEntryInfoOffset, &pChInfo->streamAggSup, pTaskInfo); + compactFunctions(pSup->pCtx, pChild->exprSupp.pCtx, numOfOutput, pTaskInfo); continue; } break; @@ -2888,6 +2932,7 @@ static SSDataBlock* doStreamSessionAgg(SOperatorInfo* pOperator) { return NULL; } + SExprSupp* pSup = &pOperator->exprSupp; SStreamSessionAggOperatorInfo* pInfo = pOperator->info; SOptrBasicInfo* pBInfo = &pInfo->binfo; if (pOperator->status == OP_RES_TO_RETURN) { @@ -2896,7 +2941,7 @@ static SSDataBlock* doStreamSessionAgg(SOperatorInfo* pOperator) { return pInfo->pDelRes; } doBuildResultDatablock(pOperator, pBInfo, &pInfo->groupResInfo, pInfo->streamAggSup.pResultBuf); - if (pBInfo->pRes->info.rows == 0 || !hashRemainDataInGroupInfo(&pInfo->groupResInfo)) { + if (pBInfo->pRes->info.rows == 0 || !hasDataInGroupInfo(&pInfo->groupResInfo)) { doSetOperatorCompleted(pOperator); } return pBInfo->pRes->info.rows == 0 ? NULL : pBInfo->pRes; @@ -2914,14 +2959,14 @@ static SSDataBlock* doStreamSessionAgg(SOperatorInfo* pOperator) { if (pBlock->info.type == STREAM_REPROCESS) { SArray* pWins = taosArrayInit(16, sizeof(SResultWindowInfo)); - doClearSessionWindows(&pInfo->streamAggSup, &pInfo->binfo, pBlock, 0, pOperator->numOfExprs, pInfo->gap, pWins); + doClearSessionWindows(&pInfo->streamAggSup, &pOperator->exprSupp, pBlock, 0, pOperator->exprSupp.numOfExprs, pInfo->gap, pWins); if (isFinalSession(pInfo)) { int32_t childIndex = 0; // Todo(liuyao) get child id from SSDataBlock SOperatorInfo* pChildOp = taosArrayGetP(pInfo->pChildren, childIndex); SStreamSessionAggOperatorInfo* pChildInfo = pChildOp->info; - doClearSessionWindows(&pChildInfo->streamAggSup, &pChildInfo->binfo, pBlock, 0, pChildOp->numOfExprs, + doClearSessionWindows(&pChildInfo->streamAggSup, &pChildOp->exprSupp, pBlock, 0, pChildOp->exprSupp.numOfExprs, pChildInfo->gap, NULL); - rebuildTimeWindow(pInfo, pWins, pBlock->info.groupId, pOperator->numOfExprs, pOperator->pTaskInfo); + rebuildTimeWindow(pInfo, pWins, pBlock->info.groupId, pOperator->exprSupp.numOfExprs, pOperator); } taosArrayDestroy(pWins); continue; @@ -2931,7 +2976,7 @@ static SSDataBlock* doStreamSessionAgg(SOperatorInfo* pOperator) { } // the pDataBlock are always the same one, no need to call this again - setInputDataBlock(pOperator, pBInfo->pCtx, pBlock, TSDB_ORDER_ASC, MAIN_SCAN, true); + setInputDataBlock(pOperator, pSup->pCtx, pBlock, TSDB_ORDER_ASC, MAIN_SCAN, true); if (isFinalSession(pInfo)) { int32_t childIndex = 0; // Todo(liuyao) get child id from SSDataBlock SOptrBasicInfo* pChildOp = taosArrayGetP(pInfo->pChildren, childIndex); @@ -2948,8 +2993,8 @@ static SSDataBlock* doStreamSessionAgg(SOperatorInfo* pOperator) { copyUpdateResult(pStUpdated, pUpdated); taosHashCleanup(pStUpdated); - finalizeUpdatedResult(pOperator->numOfExprs, pInfo->streamAggSup.pResultBuf, pUpdated, - pInfo->binfo.rowCellInfoOffset); + finalizeUpdatedResult(pOperator->exprSupp.numOfExprs, pInfo->streamAggSup.pResultBuf, pUpdated, + pSup->rowEntryInfoOffset); initMultiResInfoFromArrayList(&pInfo->groupResInfo, pUpdated); blockDataEnsureCapacity(pInfo->binfo.pRes, pOperator->resultInfo.capacity); doBuildDeleteDataBlock(pInfo->pStDeleted, pInfo->pDelRes, &pInfo->pDelIterator); @@ -2960,25 +3005,20 @@ static SSDataBlock* doStreamSessionAgg(SOperatorInfo* pOperator) { return pBInfo->pRes->info.rows == 0 ? NULL : pBInfo->pRes; } -SOperatorInfo* createStreamFinalSessionAggOperatorInfo(SOperatorInfo* downstream, SExprInfo* pExprInfo, - int32_t numOfCols, SSDataBlock* pResBlock, int64_t gap, - int32_t tsSlotId, STimeWindowAggSupp* pTwAggSupp, - SExecTaskInfo* pTaskInfo) { - int32_t code = TSDB_CODE_OUT_OF_MEMORY; - SStreamSessionAggOperatorInfo* pInfo = NULL; - SOperatorInfo* pOperator = createStreamSessionAggOperatorInfo(downstream, pExprInfo, numOfCols, pResBlock, gap, - tsSlotId, pTwAggSupp, pTaskInfo); +SOperatorInfo* createStreamFinalSessionAggOperatorInfo(SOperatorInfo* downstream, + SPhysiNode* pPhyNode, SExecTaskInfo* pTaskInfo, int32_t numOfChild) { + int32_t code = TSDB_CODE_OUT_OF_MEMORY; + SOperatorInfo* pOperator = createStreamSessionAggOperatorInfo(downstream, pPhyNode, pTaskInfo); if (pOperator == NULL) { goto _error; } pOperator->name = "StreamFinalSessionWindowAggOperator"; pOperator->operatorType = QUERY_NODE_PHYSICAL_PLAN_STREAM_FINAL_SESSION; - int32_t numOfChild = 1; // Todo(liuyao) get it from phy plan - pInfo = pOperator->info; + SStreamSessionAggOperatorInfo* pInfo = pOperator->info; pInfo->pChildren = taosArrayInit(8, sizeof(void*)); for (int32_t i = 0; i < numOfChild; i++) { SOperatorInfo* pChild = - createStreamSessionAggOperatorInfo(NULL, pExprInfo, numOfCols, NULL, gap, tsSlotId, pTwAggSupp, pTaskInfo); + createStreamSessionAggOperatorInfo(NULL, pPhyNode, pTaskInfo); if (pChild == NULL) { goto _error; } @@ -2988,7 +3028,7 @@ SOperatorInfo* createStreamFinalSessionAggOperatorInfo(SOperatorInfo* downstream _error: if (pInfo != NULL) { - destroyStreamSessionAggOperatorInfo(pInfo, numOfCols); + destroyStreamSessionAggOperatorInfo(pInfo, pOperator->exprSupp.numOfExprs); } taosMemoryFreeClear(pInfo); @@ -2999,7 +3039,7 @@ _error: void destroyStreamStateOperatorInfo(void* param, int32_t numOfOutput) { SStreamStateAggOperatorInfo* pInfo = (SStreamStateAggOperatorInfo*)param; - doDestroyBasicInfo(&pInfo->binfo, numOfOutput); + cleanupBasicInfo(&pInfo->binfo); destroyStreamAggSupporter(&pInfo->streamAggSup); cleanupGroupResInfo(&pInfo->groupResInfo); if (pInfo->pChildren != NULL) { @@ -3207,7 +3247,7 @@ static void doStreamStateAggImpl(SOperatorInfo* pOperator, SSDataBlock* pSDataBl SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo; SStreamStateAggOperatorInfo* pInfo = pOperator->info; bool masterScan = true; - int32_t numOfOutput = pOperator->numOfExprs; + int32_t numOfOutput = pOperator->exprSupp.numOfExprs; int64_t groupId = pSDataBlock->info.groupId; int64_t code = TSDB_CODE_SUCCESS; int32_t step = 1; @@ -3239,7 +3279,7 @@ static void doStreamStateAggImpl(SOperatorInfo* pOperator, SSDataBlock* pSDataBl deleteWindow(pAggSup->pCurWins, winIndex); continue; } - code = doOneStateWindowAgg(pInfo, pSDataBlock, &pCurWin->winInfo, &pResult, i, winRows, numOfOutput, pTaskInfo); + code = doOneStateWindowAgg(pInfo, pSDataBlock, &pCurWin->winInfo, &pResult, i, winRows, numOfOutput, pOperator); if (code != TSDB_CODE_SUCCESS || pResult == NULL) { longjmp(pTaskInfo->env, TSDB_CODE_QRY_OUT_OF_MEMORY); } @@ -3261,6 +3301,7 @@ static SSDataBlock* doStreamStateAgg(SOperatorInfo* pOperator) { return NULL; } + SExprSupp* pSup = &pOperator->exprSupp; SStreamStateAggOperatorInfo* pInfo = pOperator->info; SOptrBasicInfo* pBInfo = &pInfo->binfo; if (pOperator->status == OP_RES_TO_RETURN) { @@ -3269,7 +3310,7 @@ static SSDataBlock* doStreamStateAgg(SOperatorInfo* pOperator) { return pInfo->pDelRes; } doBuildResultDatablock(pOperator, pBInfo, &pInfo->groupResInfo, pInfo->streamAggSup.pResultBuf); - if (pBInfo->pRes->info.rows == 0 || !hashRemainDataInGroupInfo(&pInfo->groupResInfo)) { + if (pBInfo->pRes->info.rows == 0 || !hasDataInGroupInfo(&pInfo->groupResInfo)) { doSetOperatorCompleted(pOperator); } return pBInfo->pRes->info.rows == 0 ? NULL : pBInfo->pRes; @@ -3295,7 +3336,7 @@ static SSDataBlock* doStreamStateAgg(SOperatorInfo* pOperator) { } // the pDataBlock are always the same one, no need to call this again - setInputDataBlock(pOperator, pBInfo->pCtx, pBlock, TSDB_ORDER_ASC, MAIN_SCAN, true); + setInputDataBlock(pOperator, pSup->pCtx, pBlock, TSDB_ORDER_ASC, MAIN_SCAN, true); doStreamStateAggImpl(pOperator, pBlock, pSeUpdated, pInfo->pSeDeleted); pInfo->twAggSup.maxTs = TMAX(pInfo->twAggSup.maxTs, pBlock->info.window.ekey); } @@ -3307,8 +3348,8 @@ static SSDataBlock* doStreamStateAgg(SOperatorInfo* pOperator) { copyUpdateResult(pSeUpdated, pUpdated); taosHashCleanup(pSeUpdated); - finalizeUpdatedResult(pOperator->numOfExprs, pInfo->streamAggSup.pResultBuf, pUpdated, - pInfo->binfo.rowCellInfoOffset); + finalizeUpdatedResult(pOperator->exprSupp.numOfExprs, pInfo->streamAggSup.pResultBuf, pUpdated, + pSup->rowEntryInfoOffset); initMultiResInfoFromArrayList(&pInfo->groupResInfo, pUpdated); blockDataEnsureCapacity(pInfo->binfo.pRes, pOperator->resultInfo.capacity); doBuildDeleteDataBlock(pInfo->pSeDeleted, pInfo->pDelRes, &pInfo->pDelIterator); @@ -3337,12 +3378,14 @@ SOperatorInfo* createStreamStateAggOperatorInfo(SOperatorInfo* downstream, SPhys goto _error; } + SExprSupp* pSup = &pOperator->exprSupp; + int32_t numOfCols = 0; SExprInfo* pExprInfo = createExprInfo(pStateNode->window.pFuncs, NULL, &numOfCols); pInfo->stateCol = extractColumnFromColumnNode(pColNode); initResultSizeInfo(pOperator, 4096); - initResultRowInfo(&pInfo->binfo.resultRowInfo, 8); + initResultRowInfo(&pInfo->binfo.resultRowInfo); pInfo->twAggSup = (STimeWindowAggSupp){ .waterMark = pStateNode->window.watermark, .calTrigger = pStateNode->window.triggerType, @@ -3350,12 +3393,12 @@ SOperatorInfo* createStreamStateAggOperatorInfo(SOperatorInfo* downstream, SPhys }; initExecTimeWindowInfo(&pInfo->twAggSup.timeWindowData, &pTaskInfo->window); - code = initBiasicInfo(&pInfo->binfo, pExprInfo, numOfCols, pResBlock); + code = initBasicInfoEx(&pInfo->binfo, pSup, pExprInfo, numOfCols, pResBlock); if (code != TSDB_CODE_SUCCESS) { goto _error; } - code = initStateAggSupporter(&pInfo->streamAggSup, "StreamStateAggOperatorInfo", pInfo->binfo.pCtx, numOfCols); + code = initStateAggSupporter(&pInfo->streamAggSup, "StreamStateAggOperatorInfo", pSup->pCtx, numOfCols); if (code != TSDB_CODE_SUCCESS) { goto _error; } @@ -3365,7 +3408,7 @@ SOperatorInfo* createStreamStateAggOperatorInfo(SOperatorInfo* downstream, SPhys goto _error; } - initDummyFunction(pInfo->pDummyCtx, pInfo->binfo.pCtx, numOfCols); + initDummyFunction(pInfo->pDummyCtx, pSup->pCtx, numOfCols); pInfo->primaryTsIndex = tsSlotId; pInfo->order = TSDB_ORDER_ASC; _hash_fn_t hashFn = taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY); @@ -3379,8 +3422,8 @@ SOperatorInfo* createStreamStateAggOperatorInfo(SOperatorInfo* downstream, SPhys pOperator->operatorType = QUERY_NODE_PHYSICAL_PLAN_STREAM_STATE; pOperator->blocking = true; pOperator->status = OP_NOT_OPENED; - pOperator->numOfExprs = numOfCols; - pOperator->pExpr = pExprInfo; + pOperator->exprSupp.numOfExprs = numOfCols; + pOperator->exprSupp.pExprInfo = pExprInfo; pOperator->pTaskInfo = pTaskInfo; pOperator->info = pInfo; pOperator->fpSet = createOperatorFpSet(operatorDummyOpenFn, doStreamStateAgg, NULL, NULL, @@ -3418,6 +3461,8 @@ static int32_t outputMergeIntervalResult(SOperatorInfo* pOperatorInfo, uint64_t SMergeIntervalAggOperatorInfo* miaInfo = pOperatorInfo->info; SIntervalAggOperatorInfo* iaInfo = &miaInfo->intervalAggOperatorInfo; SExecTaskInfo* pTaskInfo = pOperatorInfo->pTaskInfo; + + SExprSupp* pSup = &pOperatorInfo->exprSupp; bool ascScan = (iaInfo->order == TSDB_ORDER_ASC); SET_RES_WINDOW_KEY(iaInfo->aggSup.keyBuf, &wstartTs, TSDB_KEYSIZE, tableGroupId); @@ -3425,8 +3470,8 @@ static int32_t outputMergeIntervalResult(SOperatorInfo* pOperatorInfo, uint64_t GET_RES_WINDOW_KEY_LEN(TSDB_KEYSIZE)); ASSERT(p1 != NULL); - finalizeResultRowIntoResultDataBlock(iaInfo->aggSup.pResultBuf, p1, iaInfo->binfo.pCtx, pOperatorInfo->pExpr, - pOperatorInfo->numOfExprs, iaInfo->binfo.rowCellInfoOffset, pResultBlock, + finalizeResultRowIntoResultDataBlock(iaInfo->aggSup.pResultBuf, p1, pSup->pCtx, pSup->pExprInfo, + pSup->numOfExprs, pSup->rowEntryInfoOffset, pResultBlock, pTaskInfo); taosHashRemove(iaInfo->aggSup.pResultRowHashTable, iaInfo->aggSup.keyBuf, GET_RES_WINDOW_KEY_LEN(TSDB_KEYSIZE)); @@ -3439,9 +3484,10 @@ static void doMergeIntervalAggImpl(SOperatorInfo* pOperatorInfo, SResultRowInfo* SIntervalAggOperatorInfo* iaInfo = &miaInfo->intervalAggOperatorInfo; SExecTaskInfo* pTaskInfo = pOperatorInfo->pTaskInfo; + SExprSupp* pSup = &pOperatorInfo->exprSupp; int32_t startPos = 0; - int32_t numOfOutput = pOperatorInfo->numOfExprs; + int32_t numOfOutput = pSup->numOfExprs; int64_t* tsCols = extractTsCol(pBlock, iaInfo); uint64_t tableGroupId = pBlock->info.groupId; TSKEY blockStartTs = getStartTsKey(&pBlock->info.window, tsCols); @@ -3453,8 +3499,8 @@ static void doMergeIntervalAggImpl(SOperatorInfo* pOperatorInfo, SResultRowInfo* //TODO: remove the hash table usage (groupid + winkey => result row position) int32_t ret = - setTimeWindowOutputBuf(pResultRowInfo, &win, (scanFlag == MAIN_SCAN), &pResult, tableGroupId, iaInfo->binfo.pCtx, - numOfOutput, iaInfo->binfo.rowCellInfoOffset, &iaInfo->aggSup, pTaskInfo); + setTimeWindowOutputBuf(pResultRowInfo, &win, (scanFlag == MAIN_SCAN), &pResult, tableGroupId, pSup->pCtx, + numOfOutput, pSup->rowEntryInfoOffset, &iaInfo->aggSup, pTaskInfo); if (ret != TSDB_CODE_SUCCESS || pResult == NULL) { longjmp(pTaskInfo->env, TSDB_CODE_QRY_OUT_OF_MEMORY); } @@ -3471,7 +3517,7 @@ static void doMergeIntervalAggImpl(SOperatorInfo* pOperatorInfo, SResultRowInfo* continue; } else { updateTimeWindowInfo(&iaInfo->twAggSup.timeWindowData, &currWin, true); - doApplyFunctions(pTaskInfo, iaInfo->binfo.pCtx, &currWin, &iaInfo->twAggSup.timeWindowData, startPos, + doApplyFunctions(pTaskInfo, pSup->pCtx, &currWin, &iaInfo->twAggSup.timeWindowData, startPos, currPos - startPos, tsCols, pBlock->info.rows, numOfOutput, iaInfo->order); outputMergeIntervalResult(pOperatorInfo, tableGroupId, pResultBlock, currTs); @@ -3480,15 +3526,15 @@ static void doMergeIntervalAggImpl(SOperatorInfo* pOperatorInfo, SResultRowInfo* currWin.skey = currTs; currWin.ekey = taosTimeAdd(currWin.skey, iaInfo->interval.interval, iaInfo->interval.intervalUnit, iaInfo->interval.precision) - 1; startPos = currPos; - ret = setTimeWindowOutputBuf(pResultRowInfo, &currWin, (scanFlag == MAIN_SCAN), &pResult, tableGroupId, iaInfo->binfo.pCtx, - numOfOutput, iaInfo->binfo.rowCellInfoOffset, &iaInfo->aggSup, pTaskInfo); + ret = setTimeWindowOutputBuf(pResultRowInfo, &currWin, (scanFlag == MAIN_SCAN), &pResult, tableGroupId, pSup->pCtx, + numOfOutput, pSup->rowEntryInfoOffset, &iaInfo->aggSup, pTaskInfo); if (ret != TSDB_CODE_SUCCESS || pResult == NULL) { longjmp(pTaskInfo->env, TSDB_CODE_QRY_OUT_OF_MEMORY); } } } updateTimeWindowInfo(&iaInfo->twAggSup.timeWindowData, &currWin, true); - doApplyFunctions(pTaskInfo, iaInfo->binfo.pCtx, &currWin, &iaInfo->twAggSup.timeWindowData, startPos, + doApplyFunctions(pTaskInfo, pSup->pCtx, &currWin, &iaInfo->twAggSup.timeWindowData, startPos, currPos - startPos, tsCols, pBlock->info.rows, numOfOutput, iaInfo->order); outputMergeIntervalResult(pOperatorInfo, tableGroupId, pResultBlock, currTs); @@ -3503,6 +3549,7 @@ static SSDataBlock* doMergeIntervalAgg(SOperatorInfo* pOperator) { return NULL; } + SExprSupp* pSup = &pOperator->exprSupp; SSDataBlock* pRes = iaInfo->binfo.pRes; blockDataCleanup(pRes); blockDataEnsureCapacity(pRes, pOperator->resultInfo.capacity); @@ -3533,7 +3580,7 @@ static SSDataBlock* doMergeIntervalAgg(SOperatorInfo* pOperator) { } getTableScanInfo(pOperator, &iaInfo->order, &scanFlag); - setInputDataBlock(pOperator, iaInfo->binfo.pCtx, pBlock, iaInfo->order, scanFlag, true); + setInputDataBlock(pOperator, pSup->pCtx, pBlock, iaInfo->order, scanFlag, true); doMergeIntervalAggImpl(pOperator, &iaInfo->binfo.resultRowInfo, pBlock, scanFlag, pRes); if (pRes->info.rows >= pOperator->resultInfo.threshold) { @@ -3563,42 +3610,40 @@ SOperatorInfo* createMergeIntervalOperatorInfo(SOperatorInfo* downstream, SExprI } SIntervalAggOperatorInfo* iaInfo = &miaInfo->intervalAggOperatorInfo; + SExprSupp* pSup = &pOperator->exprSupp; - iaInfo->win = pTaskInfo->window; - iaInfo->order = TSDB_ORDER_ASC; - iaInfo->interval = *pInterval; - + iaInfo->win = pTaskInfo->window; + iaInfo->order = TSDB_ORDER_ASC; + iaInfo->interval = *pInterval; iaInfo->execModel = pTaskInfo->execModel; - iaInfo->primaryTsIndex = primaryTsSlotId; size_t keyBufSize = sizeof(int64_t) + sizeof(int64_t) + POINTER_BYTES; initResultSizeInfo(pOperator, 4096); - int32_t code = - initAggInfo(&iaInfo->binfo, &iaInfo->aggSup, pExprInfo, numOfCols, pResBlock, keyBufSize, pTaskInfo->id.str); + int32_t code = initAggInfo(&pOperator->exprSupp, &iaInfo->aggSup, pExprInfo, numOfCols, keyBufSize, pTaskInfo->id.str); + initBasicInfo(&iaInfo->binfo, pResBlock); initExecTimeWindowInfo(&iaInfo->twAggSup.timeWindowData, &iaInfo->win); - iaInfo->timeWindowInterpo = timeWindowinterpNeeded(iaInfo->binfo.pCtx, numOfCols, iaInfo); + iaInfo->timeWindowInterpo = timeWindowinterpNeeded(pSup->pCtx, numOfCols, iaInfo); if (iaInfo->timeWindowInterpo) { iaInfo->binfo.resultRowInfo.openWindow = tdListNew(sizeof(SResultRowPosition)); } - // iaInfo->pTableQueryInfo = initTableQueryInfo(pTableGroupInfo); - if (code != TSDB_CODE_SUCCESS /* || iaInfo->pTableQueryInfo == NULL*/) { + if (code != TSDB_CODE_SUCCESS) { goto _error; } - initResultRowInfo(&iaInfo->binfo.resultRowInfo, (int32_t)1); + initResultRowInfo(&iaInfo->binfo.resultRowInfo); pOperator->name = "TimeMergeIntervalAggOperator"; pOperator->operatorType = QUERY_NODE_PHYSICAL_PLAN_MERGE_INTERVAL; pOperator->blocking = false; pOperator->status = OP_NOT_OPENED; - pOperator->pExpr = pExprInfo; + pOperator->exprSupp.pExprInfo = pExprInfo; pOperator->pTaskInfo = pTaskInfo; - pOperator->numOfExprs = numOfCols; + pOperator->exprSupp.numOfExprs = numOfCols; pOperator->info = miaInfo; pOperator->fpSet = createOperatorFpSet(operatorDummyOpenFn, doMergeIntervalAgg, NULL, NULL, diff --git a/source/libs/executor/src/tsort.c b/source/libs/executor/src/tsort.c index 1502387360cb5a1722949e243e3bb054ef15d61d..f21cad2dd62156d3acb87fdc811c747bd6b8ddad 100644 --- a/source/libs/executor/src/tsort.c +++ b/source/libs/executor/src/tsort.c @@ -71,7 +71,7 @@ SSDataBlock* tsortGetSortedDataBlock(const SSortHandle* pSortHandle) { * @param type * @return */ -SSortHandle* tsortCreateSortHandle(SArray* pSortInfo, SArray* pIndexMap, int32_t type, int32_t pageSize, int32_t numOfPages, SSDataBlock* pBlock, const char* idstr) { +SSortHandle* tsortCreateSortHandle(SArray* pSortInfo, int32_t type, int32_t pageSize, int32_t numOfPages, SSDataBlock* pBlock, const char* idstr) { SSortHandle* pSortHandle = taosMemoryCalloc(1, sizeof(SSortHandle)); pSortHandle->type = type; @@ -195,6 +195,11 @@ static int32_t doAddToBuf(SSDataBlock* pDataBlock, SSortHandle* pHandle) { return doAddNewExternalMemSource(pHandle->pBuf, pHandle->pOrderedSource, pBlock, &pHandle->sourceId); } +static void setCurrentSourceIsDone(SSortSource* pSource, SSortHandle* pHandle) { + pSource->src.rowIndex = -1; + ++pHandle->numOfCompletedSources; +} + static int32_t sortComparInit(SMsortComparParam* cmpParam, SArray* pSources, int32_t startIndex, int32_t endIndex, SSortHandle* pHandle) { cmpParam->pSources = taosArrayGet(pSources, startIndex); cmpParam->numOfSources = (endIndex - startIndex + 1); @@ -205,8 +210,10 @@ static int32_t sortComparInit(SMsortComparParam* cmpParam, SArray* pSources, int for (int32_t i = 0; i < cmpParam->numOfSources; ++i) { SSortSource* pSource = cmpParam->pSources[i]; + // set current source is done if (taosArrayGetSize(pSource->pageIdList) == 0) { - return TSDB_CODE_SUCCESS; + setCurrentSourceIsDone(pSource, pHandle); + continue; } SPageInfo* pPgInfo = *(SPageInfo**)taosArrayGet(pSource->pageIdList, pSource->pageIndex); @@ -233,10 +240,9 @@ static int32_t sortComparInit(SMsortComparParam* cmpParam, SArray* pSources, int SSortSource* pSource = cmpParam->pSources[i]; pSource->src.pBlock = pHandle->fetchfp(pSource->param); - // set current source id done + // set current source is done if (pSource->src.pBlock == NULL) { - pSource->src.rowIndex = -1; - ++pHandle->numOfCompletedSources; + setCurrentSourceIsDone(pSource, pHandle); } } } diff --git a/source/libs/executor/test/sortTests.cpp b/source/libs/executor/test/sortTests.cpp index c037fae75f258411c3cecc32b4d3032bc48d332f..66ed078bbe305d65906b8ab00ffbb47004cd311f 100644 --- a/source/libs/executor/test/sortTests.cpp +++ b/source/libs/executor/test/sortTests.cpp @@ -209,7 +209,7 @@ TEST(testCase, inMem_sort_Test) { SArray* orderInfo = taosArrayInit(1, sizeof(SBlockOrderInfo)); taosArrayPush(orderInfo, &oi); - SSortHandle* phandle = tsortCreateSortHandle(orderInfo, NULL, SORT_SINGLESOURCE_SORT, 1024, 5, NULL, "test_abc"); + SSortHandle* phandle = tsortCreateSortHandle(orderInfo, SORT_SINGLESOURCE_SORT, 1024, 5, NULL, "test_abc"); tsortSetFetchRawDataFp(phandle, getSingleColDummyBlock, NULL, NULL); _info* pInfo = (_info*) taosMemoryCalloc(1, sizeof(_info)); @@ -298,7 +298,7 @@ TEST(testCase, external_mem_sort_Test) { SArray* orderInfo = taosArrayInit(1, sizeof(SBlockOrderInfo)); taosArrayPush(orderInfo, &oi); - SSortHandle* phandle = tsortCreateSortHandle(orderInfo, NULL, SORT_SINGLESOURCE_SORT, 128, 3, NULL, "test_abc"); + SSortHandle* phandle = tsortCreateSortHandle(orderInfo, SORT_SINGLESOURCE_SORT, 128, 3, NULL, "test_abc"); tsortSetFetchRawDataFp(phandle, getSingleColDummyBlock, NULL, NULL); SSortSource* ps = static_cast(taosMemoryCalloc(1, sizeof(SSortSource))); @@ -365,7 +365,7 @@ TEST(testCase, ordered_merge_sort_Test) { taosArrayPush(pBlock->pDataBlock, &colInfo); } - SSortHandle* phandle = tsortCreateSortHandle(orderInfo, NULL, SORT_MULTISOURCE_MERGE, 1024, 5, pBlock,"test_abc"); + SSortHandle* phandle = tsortCreateSortHandle(orderInfo, SORT_MULTISOURCE_MERGE, 1024, 5, pBlock,"test_abc"); tsortSetFetchRawDataFp(phandle, getSingleColDummyBlock, NULL, NULL); tsortSetComparFp(phandle, docomp); diff --git a/source/libs/function/inc/functionMgtInt.h b/source/libs/function/inc/functionMgtInt.h index 1443c26820fe539f12e8692a843150061f587376..c02c09e18a9d5cac99fae06347064d59039a2a4f 100644 --- a/source/libs/function/inc/functionMgtInt.h +++ b/source/libs/function/inc/functionMgtInt.h @@ -44,6 +44,8 @@ extern "C" { #define FUNC_MGT_FORBID_FILL_FUNC FUNC_MGT_FUNC_CLASSIFICATION_MASK(15) #define FUNC_MGT_INTERVAL_INTERPO_FUNC FUNC_MGT_FUNC_CLASSIFICATION_MASK(16) #define FUNC_MGT_FORBID_STREAM_FUNC FUNC_MGT_FUNC_CLASSIFICATION_MASK(17) +#define FUNC_MGT_FORBID_WINDOW_FUNC FUNC_MGT_FUNC_CLASSIFICATION_MASK(18) +#define FUNC_MGT_FORBID_GROUP_BY_FUNC FUNC_MGT_FUNC_CLASSIFICATION_MASK(19) #define FUNC_MGT_TEST_MASK(val, mask) (((val) & (mask)) != 0) diff --git a/source/libs/function/src/builtins.c b/source/libs/function/src/builtins.c index b0741908a6ae7e11271a84a784cfe711f624bb7c..e1375e8a6f1773675e61a1cc5e620c7e85e867b3 100644 --- a/source/libs/function/src/builtins.c +++ b/source/libs/function/src/builtins.c @@ -1017,7 +1017,7 @@ static int32_t translateDerivative(SFunctionNode* pFunc, char* pErrBuf, int32_t return invaildFuncParaTypeErrMsg(pErrBuf, len, pFunc->functionName); } - SNode* pParamNode2 = nodesListGetNode(pFunc->pParameterList, 2); + SNode* pParamNode2 = nodesListGetNode(pFunc->pParameterList, 2); SValueNode* pValue2 = (SValueNode*)pParamNode2; pValue2->notReserved = true; @@ -1086,8 +1086,8 @@ static int32_t translateUnique(SFunctionNode* pFunc, char* pErrBuf, int32_t len) } SNode* pPara = nodesListGetNode(pFunc->pParameterList, 0); - if (QUERY_NODE_COLUMN != nodeType(pPara)) { - return buildFuncErrMsg(pErrBuf, len, TSDB_CODE_FUNC_FUNTION_ERROR, "The parameters of UNIQUE can only be columns"); + if (!nodesExprHasColumn(pPara)) { + return buildFuncErrMsg(pErrBuf, len, TSDB_CODE_FUNC_FUNTION_ERROR, "The parameters of UNIQUE must contain columns"); } pFunc->node.resType = ((SExprNode*)pPara)->resType; @@ -2114,7 +2114,7 @@ const SBuiltinFuncDefinition funcMgtBuiltins[] = { { .name = "tail", .type = FUNCTION_TYPE_TAIL, - .classification = FUNC_MGT_INDEFINITE_ROWS_FUNC | FUNC_MGT_TIMELINE_FUNC | FUNC_MGT_FORBID_STREAM_FUNC, + .classification = FUNC_MGT_INDEFINITE_ROWS_FUNC | FUNC_MGT_TIMELINE_FUNC | FUNC_MGT_FORBID_STREAM_FUNC | FUNC_MGT_FORBID_WINDOW_FUNC | FUNC_MGT_FORBID_GROUP_BY_FUNC, .translateFunc = translateTail, .getEnvFunc = getTailFuncEnv, .initFunc = tailFunctionSetup, @@ -2124,7 +2124,8 @@ const SBuiltinFuncDefinition funcMgtBuiltins[] = { { .name = "unique", .type = FUNCTION_TYPE_UNIQUE, - .classification = FUNC_MGT_INDEFINITE_ROWS_FUNC | FUNC_MGT_TIMELINE_FUNC | FUNC_MGT_FORBID_STREAM_FUNC, + .classification = FUNC_MGT_AGG_FUNC | FUNC_MGT_SELECT_FUNC | FUNC_MGT_INDEFINITE_ROWS_FUNC | FUNC_MGT_TIMELINE_FUNC | + FUNC_MGT_FORBID_STREAM_FUNC | FUNC_MGT_FORBID_WINDOW_FUNC | FUNC_MGT_FORBID_GROUP_BY_FUNC, .translateFunc = translateUnique, .getEnvFunc = getUniqueFuncEnv, .initFunc = uniqueFunctionSetup, diff --git a/source/libs/function/src/functionMgt.c b/source/libs/function/src/functionMgt.c index 0c762513cb6bd6bb6787b4d6cd1d45a5b41a097b..7dbe456d4a604ae6e4d424b37ed13ca937e83701 100644 --- a/source/libs/function/src/functionMgt.c +++ b/source/libs/function/src/functionMgt.c @@ -165,6 +165,10 @@ bool fmIsIntervalInterpoFunc(int32_t funcId) { return isSpecificClassifyFunc(fun bool fmIsForbidStreamFunc(int32_t funcId) { return isSpecificClassifyFunc(funcId, FUNC_MGT_FORBID_STREAM_FUNC); } +bool fmIsForbidWindowFunc(int32_t funcId) { return isSpecificClassifyFunc(funcId, FUNC_MGT_FORBID_WINDOW_FUNC); } + +bool fmIsForbidGroupByFunc(int32_t funcId) { return isSpecificClassifyFunc(funcId, FUNC_MGT_FORBID_GROUP_BY_FUNC); } + void fmFuncMgtDestroy() { void* m = gFunMgtService.pFuncNameHashTable; if (m != NULL && atomic_val_compare_exchange_ptr((void**)&gFunMgtService.pFuncNameHashTable, m, 0) == m) { @@ -206,7 +210,7 @@ bool fmIsInvertible(int32_t funcId) { return res; } -//function has same input/output type +// function has same input/output type bool fmIsSameInOutType(int32_t funcId) { bool res = false; switch (funcMgtBuiltins[funcId].type) { @@ -301,7 +305,7 @@ static int32_t createMergeFunction(const SFunctionNode* pSrcFunc, const SFunctio nodesDestroyList(pParameterList); return TSDB_CODE_OUT_OF_MEMORY; } - //overwrite function restype set by translate function + // overwrite function restype set by translate function if (fmIsSameInOutType(pSrcFunc->funcId)) { (*pMergeFunc)->node.resType = pSrcFunc->node.resType; } diff --git a/source/libs/nodes/src/nodesCloneFuncs.c b/source/libs/nodes/src/nodesCloneFuncs.c index 2e30b01357464acbd0c50eb842526879f3c5c4aa..d0eb1548c1f209ad260ec2359c18aeda2216de09 100644 --- a/source/libs/nodes/src/nodesCloneFuncs.c +++ b/source/libs/nodes/src/nodesCloneFuncs.c @@ -427,7 +427,7 @@ static SNode* logicWindowCopy(const SWindowLogicNode* pSrc, SWindowLogicNode* pD COPY_SCALAR_FIELD(triggerType); COPY_SCALAR_FIELD(watermark); COPY_SCALAR_FIELD(filesFactor); - COPY_SCALAR_FIELD(intervalAlgo); + COPY_SCALAR_FIELD(windowAlgo); return (SNode*)pDst; } @@ -538,6 +538,12 @@ static SNode* physiIntervalCopy(const SIntervalPhysiNode* pSrc, SIntervalPhysiNo return (SNode*)pDst; } +static SNode* physiSessionCopy(const SSessionWinodwPhysiNode* pSrc, SSessionWinodwPhysiNode* pDst) { + COPY_BASE_OBJECT_FIELD(window, physiWindowCopy); + COPY_SCALAR_FIELD(gap); + return (SNode*)pDst; +} + static SNode* dataBlockDescCopy(const SDataBlockDescNode* pSrc, SDataBlockDescNode* pDst) { COPY_SCALAR_FIELD(dataBlockId); CLONE_NODE_LIST_FIELD(pSlots); @@ -678,6 +684,9 @@ SNode* nodesCloneNode(const SNode* pNode) { case QUERY_NODE_PHYSICAL_PLAN_STREAM_FINAL_INTERVAL: case QUERY_NODE_PHYSICAL_PLAN_STREAM_SEMI_INTERVAL: return physiIntervalCopy((const SIntervalPhysiNode*)pNode, (SIntervalPhysiNode*)pDst); + case QUERY_NODE_PHYSICAL_PLAN_STREAM_SEMI_SESSION: + case QUERY_NODE_PHYSICAL_PLAN_STREAM_FINAL_SESSION: + return physiSessionCopy((const SSessionWinodwPhysiNode*)pNode, (SSessionWinodwPhysiNode*)pDst); default: break; } diff --git a/source/libs/nodes/src/nodesCodeFuncs.c b/source/libs/nodes/src/nodesCodeFuncs.c index fb6a428d3c82c190b76be62b24ddc149ce9b46ee..1b2eadd8e8febd1a256d2ef671e9084c90ae546f 100644 --- a/source/libs/nodes/src/nodesCodeFuncs.c +++ b/source/libs/nodes/src/nodesCodeFuncs.c @@ -172,8 +172,6 @@ const char* nodesNodeName(ENodeType type) { return "ShowSubscribesStmt"; case QUERY_NODE_SHOW_SMAS_STMT: return "ShowSmasStmt"; - case QUERY_NODE_SHOW_CONFIGS_STMT: - return "ShowConfigsStmt"; case QUERY_NODE_SHOW_QUERIES_STMT: return "ShowQueriesStmt"; case QUERY_NODE_SHOW_VNODES_STMT: @@ -246,6 +244,10 @@ const char* nodesNodeName(ENodeType type) { return "PhysiSessionWindow"; case QUERY_NODE_PHYSICAL_PLAN_STREAM_SESSION: return "PhysiStreamSessionWindow"; + case QUERY_NODE_PHYSICAL_PLAN_STREAM_SEMI_SESSION: + return "PhysiStreamSemiSessionWindow"; + case QUERY_NODE_PHYSICAL_PLAN_STREAM_FINAL_SESSION: + return "PhysiStreamFinalSessionWindow"; case QUERY_NODE_PHYSICAL_PLAN_MERGE_STATE: return "PhysiStateWindow"; case QUERY_NODE_PHYSICAL_PLAN_STREAM_STATE: @@ -3998,6 +4000,8 @@ static int32_t specificNodeToJson(const void* pObj, SJson* pJson) { return physiFillNodeToJson(pObj, pJson); case QUERY_NODE_PHYSICAL_PLAN_MERGE_SESSION: case QUERY_NODE_PHYSICAL_PLAN_STREAM_SESSION: + case QUERY_NODE_PHYSICAL_PLAN_STREAM_SEMI_SESSION: + case QUERY_NODE_PHYSICAL_PLAN_STREAM_FINAL_SESSION: return physiSessionWindowNodeToJson(pObj, pJson); case QUERY_NODE_PHYSICAL_PLAN_MERGE_STATE: case QUERY_NODE_PHYSICAL_PLAN_STREAM_STATE: @@ -4131,6 +4135,8 @@ static int32_t jsonToSpecificNode(const SJson* pJson, void* pObj) { return jsonToPhysiFillNode(pJson, pObj); case QUERY_NODE_PHYSICAL_PLAN_MERGE_SESSION: case QUERY_NODE_PHYSICAL_PLAN_STREAM_SESSION: + case QUERY_NODE_PHYSICAL_PLAN_STREAM_SEMI_SESSION: + case QUERY_NODE_PHYSICAL_PLAN_STREAM_FINAL_SESSION: return jsonToPhysiSessionWindowNode(pJson, pObj); case QUERY_NODE_PHYSICAL_PLAN_MERGE_STATE: case QUERY_NODE_PHYSICAL_PLAN_STREAM_STATE: diff --git a/source/libs/nodes/src/nodesUtilFuncs.c b/source/libs/nodes/src/nodesUtilFuncs.c index a45ba53ad173b3bbb88881e3d3db8624ea94e51e..6ebd0f2d39082fc1ccf6337477fc4211c84df88d 100644 --- a/source/libs/nodes/src/nodesUtilFuncs.c +++ b/source/libs/nodes/src/nodesUtilFuncs.c @@ -88,6 +88,8 @@ SNode* nodesMakeNode(ENodeType type) { return makeNode(type, sizeof(SStreamOptions)); case QUERY_NODE_LEFT_VALUE: return makeNode(type, sizeof(SLeftValueNode)); + case QUERY_NODE_COLUMN_REF: + return makeNode(type, sizeof(SColumnDefNode)); case QUERY_NODE_SET_OPERATOR: return makeNode(type, sizeof(SSetOperator)); case QUERY_NODE_SELECT_STMT: @@ -148,6 +150,8 @@ SNode* nodesMakeNode(ENodeType type) { return makeNode(type, sizeof(SDropTopicStmt)); case QUERY_NODE_DROP_CGROUP_STMT: return makeNode(type, sizeof(SDropCGroupStmt)); + case QUERY_NODE_ALTER_LOCAL_STMT: + return makeNode(type, sizeof(SAlterLocalStmt)); case QUERY_NODE_EXPLAIN_STMT: return makeNode(type, sizeof(SExplainStmt)); case QUERY_NODE_DESCRIBE_STMT: @@ -198,18 +202,19 @@ SNode* nodesMakeNode(ENodeType type) { case QUERY_NODE_SHOW_CONSUMERS_STMT: case QUERY_NODE_SHOW_SUBSCRIBES_STMT: case QUERY_NODE_SHOW_SMAS_STMT: - case QUERY_NODE_SHOW_CONFIGS_STMT: case QUERY_NODE_SHOW_CONNECTIONS_STMT: case QUERY_NODE_SHOW_QUERIES_STMT: case QUERY_NODE_SHOW_VNODES_STMT: case QUERY_NODE_SHOW_APPS_STMT: case QUERY_NODE_SHOW_SCORES_STMT: case QUERY_NODE_SHOW_VARIABLE_STMT: + case QUERY_NODE_SHOW_TRANSACTIONS_STMT: + return makeNode(type, sizeof(SShowStmt)); case QUERY_NODE_SHOW_CREATE_DATABASE_STMT: + return makeNode(type, sizeof(SShowCreateDatabaseStmt)); case QUERY_NODE_SHOW_CREATE_TABLE_STMT: case QUERY_NODE_SHOW_CREATE_STABLE_STMT: - case QUERY_NODE_SHOW_TRANSACTIONS_STMT: - return makeNode(type, sizeof(SShowStmt)); + return makeNode(type, sizeof(SShowCreateTableStmt)); case QUERY_NODE_KILL_QUERY_STMT: return makeNode(type, sizeof(SKillQueryStmt)); case QUERY_NODE_KILL_TRANSACTION_STMT: @@ -287,6 +292,10 @@ SNode* nodesMakeNode(ENodeType type) { return makeNode(type, sizeof(SSessionWinodwPhysiNode)); case QUERY_NODE_PHYSICAL_PLAN_STREAM_SESSION: return makeNode(type, sizeof(SStreamSessionWinodwPhysiNode)); + case QUERY_NODE_PHYSICAL_PLAN_STREAM_SEMI_SESSION: + return makeNode(type, sizeof(SStreamSemiSessionWinodwPhysiNode)); + case QUERY_NODE_PHYSICAL_PLAN_STREAM_FINAL_SESSION: + return makeNode(type, sizeof(SStreamFinalSessionWinodwPhysiNode)); case QUERY_NODE_PHYSICAL_PLAN_MERGE_STATE: return makeNode(type, sizeof(SStateWinodwPhysiNode)); case QUERY_NODE_PHYSICAL_PLAN_STREAM_STATE: @@ -613,7 +622,6 @@ void nodesDestroyNode(SNode* pNode) { case QUERY_NODE_SHOW_CONSUMERS_STMT: case QUERY_NODE_SHOW_SUBSCRIBES_STMT: case QUERY_NODE_SHOW_SMAS_STMT: - case QUERY_NODE_SHOW_CONFIGS_STMT: case QUERY_NODE_SHOW_CONNECTIONS_STMT: case QUERY_NODE_SHOW_QUERIES_STMT: case QUERY_NODE_SHOW_VNODES_STMT: @@ -804,6 +812,7 @@ void nodesDestroyNode(SNode* pNode) { } case QUERY_NODE_PHYSICAL_PLAN_MERGE_SESSION: case QUERY_NODE_PHYSICAL_PLAN_STREAM_SESSION: + case QUERY_NODE_PHYSICAL_PLAN_STREAM_SEMI_SESSION: case QUERY_NODE_PHYSICAL_PLAN_STREAM_FINAL_SESSION: destroyWinodwPhysiNode((SWinodwPhysiNode*)pNode); break; @@ -1463,6 +1472,26 @@ int32_t nodesCollectSpecialNodes(SSelectStmt* pSelect, ESqlClause clause, ENodeT return TSDB_CODE_SUCCESS; } +static EDealRes hasColumn(SNode* pNode, void* pContext) { + if (QUERY_NODE_COLUMN == nodeType(pNode)) { + *(bool*)pContext = true; + return DEAL_RES_END; + } + return DEAL_RES_CONTINUE; +} + +bool nodesExprHasColumn(SNode* pNode) { + bool hasCol = false; + nodesWalkExprPostOrder(pNode, hasColumn, &hasCol); + return hasCol; +} + +bool nodesExprsHasColumn(SNodeList* pList) { + bool hasCol = false; + nodesWalkExprsPostOrder(pList, hasColumn, &hasCol); + return hasCol; +} + char* nodesGetFillModeString(EFillMode mode) { switch (mode) { case FILL_MODE_NONE: diff --git a/source/libs/parser/inc/parAst.h b/source/libs/parser/inc/parAst.h index b8c6ffc8434a863396ded781a3dfe49638b082d3..b4f49b3e88579436f2a491d498e59cf862e51d86 100644 --- a/source/libs/parser/inc/parAst.h +++ b/source/libs/parser/inc/parAst.h @@ -59,8 +59,8 @@ typedef enum EDatabaseOptionType { typedef enum ETableOptionType { TABLE_OPTION_COMMENT = 1, - TABLE_OPTION_FILE_FACTOR, - TABLE_OPTION_DELAY, + TABLE_OPTION_MAXDELAY, + TABLE_OPTION_WATERMARK, TABLE_OPTION_ROLLUP, TABLE_OPTION_TTL, TABLE_OPTION_SMA @@ -152,7 +152,7 @@ SNode* createAlterTableRenameCol(SAstCreateContext* pCxt, SNode* pRealTable, int SNode* createAlterTableSetTag(SAstCreateContext* pCxt, SNode* pRealTable, SToken* pTagName, SNode* pVal); SNode* createUseDatabaseStmt(SAstCreateContext* pCxt, SToken* pDbName); SNode* createShowStmt(SAstCreateContext* pCxt, ENodeType type, SNode* pDbName, SNode* pTbNamePattern); -SNode* createShowCreateDatabaseStmt(SAstCreateContext* pCxt, const SToken* pDbName); +SNode* createShowCreateDatabaseStmt(SAstCreateContext* pCxt, SToken* pDbName); SNode* createShowCreateTableStmt(SAstCreateContext* pCxt, ENodeType type, SNode* pRealTable); SNode* createCreateUserStmt(SAstCreateContext* pCxt, SToken* pUserName, const SToken* pPassword); SNode* createAlterUserStmt(SAstCreateContext* pCxt, SToken* pUserName, int8_t alterType, const SToken* pVal); diff --git a/source/libs/parser/inc/sql.y b/source/libs/parser/inc/sql.y index 0d9e6e4a73542a5c948c7b88d446a08bc7546151..476485904c8076001818ee7804182b9f411c2768 100644 --- a/source/libs/parser/inc/sql.y +++ b/source/libs/parser/inc/sql.y @@ -111,7 +111,7 @@ priv_level(A) ::= db_name(B) NK_DOT NK_STAR. /************************************************ create/drop/alter dnode *********************************************/ cmd ::= CREATE DNODE dnode_endpoint(A). { pCxt->pRootNode = createCreateDnodeStmt(pCxt, &A, NULL); } -cmd ::= CREATE DNODE dnode_host_name(A) PORT NK_INTEGER(B). { pCxt->pRootNode = createCreateDnodeStmt(pCxt, &A, &B); } +cmd ::= CREATE DNODE dnode_endpoint(A) PORT NK_INTEGER(B). { pCxt->pRootNode = createCreateDnodeStmt(pCxt, &A, &B); } cmd ::= DROP DNODE NK_INTEGER(A). { pCxt->pRootNode = createDropDnodeStmt(pCxt, &A); } cmd ::= DROP DNODE dnode_endpoint(A). { pCxt->pRootNode = createDropDnodeStmt(pCxt, &A); } cmd ::= ALTER DNODE NK_INTEGER(A) NK_STRING(B). { pCxt->pRootNode = createAlterDnodeStmt(pCxt, &A, &B, NULL); } @@ -122,11 +122,8 @@ cmd ::= ALTER ALL DNODES NK_STRING(A) NK_STRING(B). %type dnode_endpoint { SToken } %destructor dnode_endpoint { } dnode_endpoint(A) ::= NK_STRING(B). { A = B; } - -%type dnode_host_name { SToken } -%destructor dnode_host_name { } -dnode_host_name(A) ::= NK_ID(B). { A = B; } -dnode_host_name(A) ::= NK_IPTOKEN(B). { A = B; } +dnode_endpoint(A) ::= NK_ID(B). { A = B; } +dnode_endpoint(A) ::= NK_IPTOKEN(B). { A = B; } /************************************************ alter local *********************************************************/ cmd ::= ALTER LOCAL NK_STRING(A). { pCxt->pRootNode = createAlterLocalStmt(pCxt, &A, NULL); } @@ -317,8 +314,8 @@ tags_def(A) ::= TAGS NK_LP column_def_list(B) NK_RP. table_options(A) ::= . { A = createDefaultTableOptions(pCxt); } table_options(A) ::= table_options(B) COMMENT NK_STRING(C). { A = setTableOption(pCxt, B, TABLE_OPTION_COMMENT, &C); } -//table_options(A) ::= table_options(B) DELAY NK_INTEGER(C). { A = setTableOption(pCxt, B, TABLE_OPTION_DELAY, &C); } -table_options(A) ::= table_options(B) FILE_FACTOR NK_FLOAT(C). { A = setTableOption(pCxt, B, TABLE_OPTION_FILE_FACTOR, &C); } +table_options(A) ::= table_options(B) MAX_DELAY duration_list(C). { A = setTableOption(pCxt, B, TABLE_OPTION_MAXDELAY, C); } +table_options(A) ::= table_options(B) WATERMARK duration_list(C). { A = setTableOption(pCxt, B, TABLE_OPTION_WATERMARK, C); } table_options(A) ::= table_options(B) ROLLUP NK_LP rollup_func_list(C) NK_RP. { A = setTableOption(pCxt, B, TABLE_OPTION_ROLLUP, C); } table_options(A) ::= table_options(B) TTL NK_INTEGER(C). { A = setTableOption(pCxt, B, TABLE_OPTION_TTL, &C); } table_options(A) ::= table_options(B) SMA NK_LP col_name_list(C) NK_RP. { A = setTableOption(pCxt, B, TABLE_OPTION_SMA, C); } @@ -331,6 +328,11 @@ alter_table_options(A) ::= alter_table_options(B) alter_table_option(C). alter_table_option(A) ::= COMMENT NK_STRING(B). { A.type = TABLE_OPTION_COMMENT; A.val = B; } alter_table_option(A) ::= TTL NK_INTEGER(B). { A.type = TABLE_OPTION_TTL; A.val = B; } +%type duration_list { SNodeList* } +%destructor duration_list { nodesDestroyList($$); } +duration_list(A) ::= duration_literal(B). { A = createNodeList(pCxt, releaseRawExprNode(pCxt, B)); } +duration_list(A) ::= duration_list(B) NK_COMMA duration_literal(C). { A = addNodeToList(pCxt, B, releaseRawExprNode(pCxt, C)); } + %type rollup_func_list { SNodeList* } %destructor rollup_func_list { nodesDestroyList($$); } rollup_func_list(A) ::= rollup_func_name(B). { A = createNodeList(pCxt, B); } diff --git a/source/libs/parser/src/parAstCreater.c b/source/libs/parser/src/parAstCreater.c index 8a7987e4ec0b389619c76b8a9bff95e06b4239f9..6ea1b090ad6d48a46bdf87f0872463ed7e2f9b71 100644 --- a/source/libs/parser/src/parAstCreater.c +++ b/source/libs/parser/src/parAstCreater.c @@ -18,6 +18,7 @@ #include "parAst.h" #include "parUtil.h" +#include "tglobal.h" #include "ttime.h" #define CHECK_OUT_OF_MEM(p) \ @@ -36,6 +37,14 @@ } \ } while (0) +#define COPY_STRING_FORM_ID_TOKEN(buf, pToken) strncpy(buf, (pToken)->z, TMIN((pToken)->n, sizeof(buf) - 1)) +#define COPY_STRING_FORM_STR_TOKEN(buf, pToken) \ + do { \ + if ((pToken)->n > 2) { \ + strncpy(buf, (pToken)->z + 1, TMIN((pToken)->n - 2, sizeof(buf) - 1)); \ + } \ + } while (0) + SToken nil_token = {.type = TK_NK_NIL, .n = 0, .z = NULL}; void initAstCreateContext(SParseContext* pParseCxt, SAstCreateContext* pCxt) { @@ -50,12 +59,6 @@ void initAstCreateContext(SParseContext* pParseCxt, SAstCreateContext* pCxt) { pCxt->errCode = TSDB_CODE_SUCCESS; } -static void copyStringFormStringToken(SToken* pToken, char* pBuf, int32_t len) { - if (pToken->n > 2) { - strncpy(pBuf, pToken->z + 1, TMIN(pToken->n - 2, len - 1)); - } -} - static void trimEscape(SToken* pName) { // todo need to deal with `ioo``ii` -> ioo`ii if (NULL != pName && pName->n > 1 && '`' == pName->z[0]) { @@ -108,50 +111,52 @@ static bool checkPassword(SAstCreateContext* pCxt, const SToken* pPasswordToken, return TSDB_CODE_SUCCESS == pCxt->errCode; } -static bool checkAndSplitEndpoint(SAstCreateContext* pCxt, const SToken* pEp, char* pFqdn, int32_t* pPort) { - if (NULL == pEp) { - pCxt->errCode = TSDB_CODE_PAR_SYNTAX_ERROR; - } else if (pEp->n >= TSDB_FQDN_LEN + 2 + 6) { // format 'fqdn:port' - pCxt->errCode = generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_NAME_OR_PASSWD_TOO_LONG); - } else { - char ep[TSDB_FQDN_LEN + 2 + 6]; - strncpy(ep, pEp->z, pEp->n); - strdequote(ep); - strtrim(ep); - char* pColon = strchr(ep, ':'); - if (NULL == pColon) { - pCxt->errCode = generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_INVALID_ENDPOINT); - } else { - strncpy(pFqdn, ep, pColon - ep); - *pPort = taosStr2Int32(pColon + 1, NULL, 10); - if (*pPort >= UINT16_MAX || *pPort <= 0) { - pCxt->errCode = generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_INVALID_PORT); - } - } +static int32_t parsePort(SAstCreateContext* pCxt, const char* p, int32_t* pPort) { + *pPort = taosStr2Int32(p, NULL, 10); + if (*pPort >= UINT16_MAX || *pPort <= 0) { + return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_INVALID_PORT); } - return TSDB_CODE_SUCCESS == pCxt->errCode; + return TSDB_CODE_SUCCESS; } -static bool checkFqdn(SAstCreateContext* pCxt, const SToken* pFqdn) { - if (NULL == pFqdn) { - pCxt->errCode = TSDB_CODE_PAR_SYNTAX_ERROR; - } else { - if (pFqdn->n >= TSDB_FQDN_LEN) { - pCxt->errCode = generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_NAME_OR_PASSWD_TOO_LONG); - } +static int32_t parseEndpoint(SAstCreateContext* pCxt, const SToken* pEp, char* pFqdn, int32_t* pPort) { + if (pEp->n >= (NULL == pPort ? (TSDB_FQDN_LEN + 1 + 5) : TSDB_FQDN_LEN)) { // format 'fqdn:port' or 'fqdn' + return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_NAME_OR_PASSWD_TOO_LONG); } - return TSDB_CODE_SUCCESS == pCxt->errCode; + + char ep[TSDB_FQDN_LEN + 1 + 5] = {0}; + COPY_STRING_FORM_ID_TOKEN(ep, pEp); + strdequote(ep); + strtrim(ep); + if (NULL == pPort) { + strcpy(pFqdn, ep); + return TSDB_CODE_SUCCESS; + } + char* pColon = strchr(ep, ':'); + if (NULL == pColon) { + *pPort = tsServerPort; + strcpy(pFqdn, ep); + return TSDB_CODE_SUCCESS; + } + strncpy(pFqdn, ep, pColon - ep); + return parsePort(pCxt, pColon + 1, pPort); } -static bool checkPort(SAstCreateContext* pCxt, const SToken* pPortToken, int32_t* pPort) { - if (NULL == pPortToken) { +static bool checkAndSplitEndpoint(SAstCreateContext* pCxt, const SToken* pEp, const SToken* pPortToken, char* pFqdn, + int32_t* pPort) { + if (NULL == pEp) { pCxt->errCode = TSDB_CODE_PAR_SYNTAX_ERROR; - } else { - *pPort = taosStr2Int32(pPortToken->z, NULL, 10); - if (*pPort >= UINT16_MAX || *pPort <= 0) { - pCxt->errCode = generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_INVALID_PORT); - } + return false; } + + if (NULL != pPortToken) { + pCxt->errCode = parsePort(pCxt, pPortToken->z, pPort); + } + + if (TSDB_CODE_SUCCESS == pCxt->errCode) { + pCxt->errCode = parseEndpoint(pCxt, pEp, pFqdn, (NULL != pPortToken ? NULL : pPort)); + } + return TSDB_CODE_SUCCESS == pCxt->errCode; } @@ -274,9 +279,9 @@ SNode* createColumnNode(SAstCreateContext* pCxt, SToken* pTableAlias, SToken* pC SColumnNode* col = (SColumnNode*)nodesMakeNode(QUERY_NODE_COLUMN); CHECK_OUT_OF_MEM(col); if (NULL != pTableAlias) { - strncpy(col->tableAlias, pTableAlias->z, pTableAlias->n); + COPY_STRING_FORM_ID_TOKEN(col->tableAlias, pTableAlias); } - strncpy(col->colName, pColumnName->z, pColumnName->n); + COPY_STRING_FORM_ID_TOKEN(col->colName, pColumnName); return (SNode*)col; } @@ -417,7 +422,7 @@ SNode* createFunctionNode(SAstCreateContext* pCxt, const SToken* pFuncName, SNod } SFunctionNode* func = (SFunctionNode*)nodesMakeNode(QUERY_NODE_FUNCTION); CHECK_OUT_OF_MEM(func); - strncpy(func->functionName, pFuncName->z, pFuncName->n); + COPY_STRING_FORM_ID_TOKEN(func->functionName, pFuncName); func->pParameterList = pParameterList; return (SNode*)func; } @@ -464,16 +469,16 @@ SNode* createRealTableNode(SAstCreateContext* pCxt, SToken* pDbName, SToken* pTa SRealTableNode* realTable = (SRealTableNode*)nodesMakeNode(QUERY_NODE_REAL_TABLE); CHECK_OUT_OF_MEM(realTable); if (NULL != pDbName) { - strncpy(realTable->table.dbName, pDbName->z, pDbName->n); + COPY_STRING_FORM_ID_TOKEN(realTable->table.dbName, pDbName); } else { strcpy(realTable->table.dbName, pCxt->pQueryCxt->db); } if (NULL != pTableAlias && TK_NK_NIL != pTableAlias->type) { - strncpy(realTable->table.tableAlias, pTableAlias->z, pTableAlias->n); + COPY_STRING_FORM_ID_TOKEN(realTable->table.tableAlias, pTableAlias); } else { - strncpy(realTable->table.tableAlias, pTableName->z, pTableName->n); + COPY_STRING_FORM_ID_TOKEN(realTable->table.tableAlias, pTableName); } - strncpy(realTable->table.tableName, pTableName->z, pTableName->n); + COPY_STRING_FORM_ID_TOKEN(realTable->table.tableName, pTableName); return (SNode*)realTable; } @@ -483,7 +488,7 @@ SNode* createTempTableNode(SAstCreateContext* pCxt, SNode* pSubquery, const STok CHECK_OUT_OF_MEM(tempTable); tempTable->pSubquery = pSubquery; if (NULL != pTableAlias && TK_NK_NIL != pTableAlias->type) { - strncpy(tempTable->table.tableAlias, pTableAlias->z, pTableAlias->n); + COPY_STRING_FORM_ID_TOKEN(tempTable->table.tableAlias, pTableAlias); } else { sprintf(tempTable->table.tableAlias, "%p", tempTable); } @@ -648,6 +653,8 @@ SNode* addOrderByClause(SAstCreateContext* pCxt, SNode* pStmt, SNodeList* pOrder CHECK_PARSER_STATUS(pCxt); if (QUERY_NODE_SELECT_STMT == nodeType(pStmt)) { ((SSelectStmt*)pStmt)->pOrderByList = pOrderByList; + } else { + ((SSetOperator*)pStmt)->pOrderByList = pOrderByList; } return pStmt; } @@ -783,8 +790,7 @@ SNode* setDatabaseOption(SAstCreateContext* pCxt, SNode* pOptions, EDatabaseOpti ((SDatabaseOptions*)pOptions)->pagesize = taosStr2Int32(((SToken*)pVal)->z, NULL, 10); break; case DB_OPTION_PRECISION: - copyStringFormStringToken((SToken*)pVal, ((SDatabaseOptions*)pOptions)->precisionStr, - sizeof(((SDatabaseOptions*)pOptions)->precisionStr)); + COPY_STRING_FORM_STR_TOKEN(((SDatabaseOptions*)pOptions)->precisionStr, (SToken*)pVal); break; case DB_OPTION_REPLICA: ((SDatabaseOptions*)pOptions)->replica = taosStr2Int8(((SToken*)pVal)->z, NULL, 10); @@ -833,7 +839,7 @@ SNode* createCreateDatabaseStmt(SAstCreateContext* pCxt, bool ignoreExists, STok } SCreateDatabaseStmt* pStmt = (SCreateDatabaseStmt*)nodesMakeNode(QUERY_NODE_CREATE_DATABASE_STMT); CHECK_OUT_OF_MEM(pStmt); - strncpy(pStmt->dbName, pDbName->z, pDbName->n); + COPY_STRING_FORM_ID_TOKEN(pStmt->dbName, pDbName); pStmt->ignoreExists = ignoreExists; pStmt->pOptions = (SDatabaseOptions*)pOptions; return (SNode*)pStmt; @@ -846,7 +852,7 @@ SNode* createDropDatabaseStmt(SAstCreateContext* pCxt, bool ignoreNotExists, STo } SDropDatabaseStmt* pStmt = (SDropDatabaseStmt*)nodesMakeNode(QUERY_NODE_DROP_DATABASE_STMT); CHECK_OUT_OF_MEM(pStmt); - strncpy(pStmt->dbName, pDbName->z, pDbName->n); + COPY_STRING_FORM_ID_TOKEN(pStmt->dbName, pDbName); pStmt->ignoreNotExists = ignoreNotExists; return (SNode*)pStmt; } @@ -858,7 +864,7 @@ SNode* createAlterDatabaseStmt(SAstCreateContext* pCxt, SToken* pDbName, SNode* } SAlterDatabaseStmt* pStmt = (SAlterDatabaseStmt*)nodesMakeNode(QUERY_NODE_ALTER_DATABASE_STMT); CHECK_OUT_OF_MEM(pStmt); - strncpy(pStmt->dbName, pDbName->z, pDbName->n); + COPY_STRING_FORM_ID_TOKEN(pStmt->dbName, pDbName); pStmt->pOptions = (SDatabaseOptions*)pOptions; return (SNode*)pStmt; } @@ -867,8 +873,10 @@ SNode* createDefaultTableOptions(SAstCreateContext* pCxt) { CHECK_PARSER_STATUS(pCxt); STableOptions* pOptions = (STableOptions*)nodesMakeNode(QUERY_NODE_TABLE_OPTIONS); CHECK_OUT_OF_MEM(pOptions); - // pOptions->delay = TSDB_DEFAULT_ROLLUP_DELAY; - pOptions->filesFactor = TSDB_DEFAULT_ROLLUP_FILE_FACTOR; + pOptions->maxDelay1 = -1; + pOptions->maxDelay2 = -1; + pOptions->watermark1 = TSDB_DEFAULT_ROLLUP_WATERMARK; + pOptions->watermark2 = TSDB_DEFAULT_ROLLUP_WATERMARK; pOptions->ttl = TSDB_DEFAULT_TABLE_TTL; return (SNode*)pOptions; } @@ -877,7 +885,6 @@ SNode* createAlterTableOptions(SAstCreateContext* pCxt) { CHECK_PARSER_STATUS(pCxt); STableOptions* pOptions = (STableOptions*)nodesMakeNode(QUERY_NODE_TABLE_OPTIONS); CHECK_OUT_OF_MEM(pOptions); - pOptions->delay = -1; pOptions->ttl = -1; return (SNode*)pOptions; } @@ -887,12 +894,14 @@ SNode* setTableOption(SAstCreateContext* pCxt, SNode* pOptions, ETableOptionType switch (type) { case TABLE_OPTION_COMMENT: if (checkComment(pCxt, (SToken*)pVal, true)) { - copyStringFormStringToken((SToken*)pVal, ((STableOptions*)pOptions)->comment, - sizeof(((STableOptions*)pOptions)->comment)); + COPY_STRING_FORM_STR_TOKEN(((STableOptions*)pOptions)->comment, (SToken*)pVal); } break; - case TABLE_OPTION_DELAY: - ((STableOptions*)pOptions)->delay = taosStr2Int32(((SToken*)pVal)->z, NULL, 10); + case TABLE_OPTION_MAXDELAY: + ((STableOptions*)pOptions)->pMaxDelay = pVal; + break; + case TABLE_OPTION_WATERMARK: + ((STableOptions*)pOptions)->pWatermark = pVal; break; case TABLE_OPTION_ROLLUP: ((STableOptions*)pOptions)->pRollupFuncs = pVal; @@ -916,7 +925,7 @@ SNode* createColumnDefNode(SAstCreateContext* pCxt, SToken* pColName, SDataType } SColumnDefNode* pCol = (SColumnDefNode*)nodesMakeNode(QUERY_NODE_COLUMN_DEF); CHECK_OUT_OF_MEM(pCol); - strncpy(pCol->colName, pColName->z, pColName->n); + COPY_STRING_FORM_ID_TOKEN(pCol->colName, pColName); pCol->dataType = dataType; if (NULL != pComment) { trimString(pComment->z, pComment->n, pCol->comments, sizeof(pCol->comments)); @@ -1031,7 +1040,7 @@ SNode* createAlterTableAddModifyCol(SAstCreateContext* pCxt, SNode* pRealTable, SAlterTableStmt* pStmt = (SAlterTableStmt*)nodesMakeNode(QUERY_NODE_ALTER_TABLE_STMT); CHECK_OUT_OF_MEM(pStmt); pStmt->alterType = alterType; - strncpy(pStmt->colName, pColName->z, pColName->n); + COPY_STRING_FORM_ID_TOKEN(pStmt->colName, pColName); pStmt->dataType = dataType; return createAlterTableStmtFinalize(pRealTable, pStmt); } @@ -1044,7 +1053,7 @@ SNode* createAlterTableDropCol(SAstCreateContext* pCxt, SNode* pRealTable, int8_ SAlterTableStmt* pStmt = (SAlterTableStmt*)nodesMakeNode(QUERY_NODE_ALTER_TABLE_STMT); CHECK_OUT_OF_MEM(pStmt); pStmt->alterType = alterType; - strncpy(pStmt->colName, pColName->z, pColName->n); + COPY_STRING_FORM_ID_TOKEN(pStmt->colName, pColName); return createAlterTableStmtFinalize(pRealTable, pStmt); } @@ -1057,8 +1066,8 @@ SNode* createAlterTableRenameCol(SAstCreateContext* pCxt, SNode* pRealTable, int SAlterTableStmt* pStmt = (SAlterTableStmt*)nodesMakeNode(QUERY_NODE_ALTER_TABLE_STMT); CHECK_OUT_OF_MEM(pStmt); pStmt->alterType = alterType; - strncpy(pStmt->colName, pOldColName->z, pOldColName->n); - strncpy(pStmt->newColName, pNewColName->z, pNewColName->n); + COPY_STRING_FORM_ID_TOKEN(pStmt->colName, pOldColName); + COPY_STRING_FORM_ID_TOKEN(pStmt->newColName, pNewColName); return createAlterTableStmtFinalize(pRealTable, pStmt); } @@ -1070,7 +1079,7 @@ SNode* createAlterTableSetTag(SAstCreateContext* pCxt, SNode* pRealTable, SToken SAlterTableStmt* pStmt = (SAlterTableStmt*)nodesMakeNode(QUERY_NODE_ALTER_TABLE_STMT); CHECK_OUT_OF_MEM(pStmt); pStmt->alterType = TSDB_ALTER_TABLE_UPDATE_TAG_VAL; - strncpy(pStmt->colName, pTagName->z, pTagName->n); + COPY_STRING_FORM_ID_TOKEN(pStmt->colName, pTagName); pStmt->pVal = (SValueNode*)pVal; return createAlterTableStmtFinalize(pRealTable, pStmt); } @@ -1082,7 +1091,7 @@ SNode* createUseDatabaseStmt(SAstCreateContext* pCxt, SToken* pDbName) { } SUseDatabaseStmt* pStmt = (SUseDatabaseStmt*)nodesMakeNode(QUERY_NODE_USE_DATABASE_STMT); CHECK_OUT_OF_MEM(pStmt); - strncpy(pStmt->dbName, pDbName->z, pDbName->n); + COPY_STRING_FORM_ID_TOKEN(pStmt->dbName, pDbName); return (SNode*)pStmt; } @@ -1105,18 +1114,24 @@ SNode* createShowStmt(SAstCreateContext* pCxt, ENodeType type, SNode* pDbName, S return (SNode*)pStmt; } -SNode* createShowCreateDatabaseStmt(SAstCreateContext* pCxt, const SToken* pDbName) { +SNode* createShowCreateDatabaseStmt(SAstCreateContext* pCxt, SToken* pDbName) { CHECK_PARSER_STATUS(pCxt); - SNode* pStmt = nodesMakeNode(QUERY_NODE_SHOW_CREATE_DATABASE_STMT); + if (!checkDbName(pCxt, pDbName, true)) { + return NULL; + } + SShowCreateDatabaseStmt* pStmt = (SShowCreateDatabaseStmt*)nodesMakeNode(QUERY_NODE_SHOW_CREATE_DATABASE_STMT); CHECK_OUT_OF_MEM(pStmt); - return pStmt; + COPY_STRING_FORM_ID_TOKEN(pStmt->dbName, pDbName); + return (SNode*)pStmt; } SNode* createShowCreateTableStmt(SAstCreateContext* pCxt, ENodeType type, SNode* pRealTable) { CHECK_PARSER_STATUS(pCxt); - SNode* pStmt = nodesMakeNode(type); + SShowCreateTableStmt* pStmt = (SShowCreateTableStmt*)nodesMakeNode(type); CHECK_OUT_OF_MEM(pStmt); - return pStmt; + strcpy(pStmt->dbName, ((SRealTableNode*)pRealTable)->table.dbName); + strcpy(pStmt->tableName, ((SRealTableNode*)pRealTable)->table.tableName); + return (SNode*)pStmt; } SNode* createCreateUserStmt(SAstCreateContext* pCxt, SToken* pUserName, const SToken* pPassword) { @@ -1127,7 +1142,7 @@ SNode* createCreateUserStmt(SAstCreateContext* pCxt, SToken* pUserName, const ST } SCreateUserStmt* pStmt = (SCreateUserStmt*)nodesMakeNode(QUERY_NODE_CREATE_USER_STMT); CHECK_OUT_OF_MEM(pStmt); - strncpy(pStmt->useName, pUserName->z, pUserName->n); + COPY_STRING_FORM_ID_TOKEN(pStmt->useName, pUserName); strcpy(pStmt->password, password); return (SNode*)pStmt; } @@ -1139,7 +1154,7 @@ SNode* createAlterUserStmt(SAstCreateContext* pCxt, SToken* pUserName, int8_t al } SAlterUserStmt* pStmt = (SAlterUserStmt*)nodesMakeNode(QUERY_NODE_ALTER_USER_STMT); CHECK_OUT_OF_MEM(pStmt); - strncpy(pStmt->useName, pUserName->z, pUserName->n); + COPY_STRING_FORM_ID_TOKEN(pStmt->useName, pUserName); if (TSDB_ALTER_USER_PASSWD == alterType) { char password[TSDB_USET_PASSWORD_LEN] = {0}; if (!checkPassword(pCxt, pVal, password)) { @@ -1159,29 +1174,18 @@ SNode* createDropUserStmt(SAstCreateContext* pCxt, SToken* pUserName) { } SDropUserStmt* pStmt = (SDropUserStmt*)nodesMakeNode(QUERY_NODE_DROP_USER_STMT); CHECK_OUT_OF_MEM(pStmt); - strncpy(pStmt->useName, pUserName->z, pUserName->n); + COPY_STRING_FORM_ID_TOKEN(pStmt->useName, pUserName); return (SNode*)pStmt; } SNode* createCreateDnodeStmt(SAstCreateContext* pCxt, const SToken* pFqdn, const SToken* pPort) { CHECK_PARSER_STATUS(pCxt); - int32_t port = 0; - char fqdn[TSDB_FQDN_LEN] = {0}; - if (NULL == pPort) { - if (!checkAndSplitEndpoint(pCxt, pFqdn, fqdn, &port)) { - return NULL; - } - } else if (!checkFqdn(pCxt, pFqdn) || !checkPort(pCxt, pPort, &port)) { - return NULL; - } SCreateDnodeStmt* pStmt = (SCreateDnodeStmt*)nodesMakeNode(QUERY_NODE_CREATE_DNODE_STMT); CHECK_OUT_OF_MEM(pStmt); - if (NULL == pPort) { - strcpy(pStmt->fqdn, fqdn); - } else { - strncpy(pStmt->fqdn, pFqdn->z, pFqdn->n); + if (!checkAndSplitEndpoint(pCxt, pFqdn, pPort, pStmt->fqdn, &pStmt->port)) { + nodesDestroyNode((SNode*)pStmt); + return NULL; } - pStmt->port = port; return (SNode*)pStmt; } @@ -1192,7 +1196,7 @@ SNode* createDropDnodeStmt(SAstCreateContext* pCxt, const SToken* pDnode) { if (TK_NK_INTEGER == pDnode->type) { pStmt->dnodeId = taosStr2Int32(pDnode->z, NULL, 10); } else { - if (!checkAndSplitEndpoint(pCxt, pDnode, pStmt->fqdn, &pStmt->port)) { + if (!checkAndSplitEndpoint(pCxt, pDnode, NULL, pStmt->fqdn, &pStmt->port)) { nodesDestroyNode((SNode*)pStmt); return NULL; } @@ -1223,8 +1227,8 @@ SNode* createCreateIndexStmt(SAstCreateContext* pCxt, EIndexType type, bool igno CHECK_OUT_OF_MEM(pStmt); pStmt->indexType = type; pStmt->ignoreExists = ignoreExists; - strncpy(pStmt->indexName, pIndexName->z, pIndexName->n); - strncpy(pStmt->tableName, pTableName->z, pTableName->n); + COPY_STRING_FORM_ID_TOKEN(pStmt->indexName, pIndexName); + COPY_STRING_FORM_ID_TOKEN(pStmt->tableName, pTableName); pStmt->pCols = pCols; pStmt->pOptions = (SIndexOptions*)pOptions; return (SNode*)pStmt; @@ -1250,8 +1254,8 @@ SNode* createDropIndexStmt(SAstCreateContext* pCxt, bool ignoreNotExists, SToken SDropIndexStmt* pStmt = (SDropIndexStmt*)nodesMakeNode(QUERY_NODE_DROP_INDEX_STMT); CHECK_OUT_OF_MEM(pStmt); pStmt->ignoreNotExists = ignoreNotExists; - strncpy(pStmt->indexName, pIndexName->z, pIndexName->n); - strncpy(pStmt->tableName, pTableName->z, pTableName->n); + COPY_STRING_FORM_ID_TOKEN(pStmt->indexName, pIndexName); + COPY_STRING_FORM_ID_TOKEN(pStmt->tableName, pTableName); return (SNode*)pStmt; } @@ -1276,14 +1280,14 @@ SNode* createCreateTopicStmt(SAstCreateContext* pCxt, bool ignoreExists, const S CHECK_PARSER_STATUS(pCxt); SCreateTopicStmt* pStmt = (SCreateTopicStmt*)nodesMakeNode(QUERY_NODE_CREATE_TOPIC_STMT); CHECK_OUT_OF_MEM(pStmt); - strncpy(pStmt->topicName, pTopicName->z, pTopicName->n); + COPY_STRING_FORM_ID_TOKEN(pStmt->topicName, pTopicName); pStmt->ignoreExists = ignoreExists; if (NULL != pRealTable) { strcpy(pStmt->subDbName, ((SRealTableNode*)pRealTable)->table.dbName); strcpy(pStmt->subSTbName, ((SRealTableNode*)pRealTable)->table.tableName); nodesDestroyNode(pRealTable); } else if (NULL != pSubDbName) { - strncpy(pStmt->subDbName, pSubDbName->z, pSubDbName->n); + COPY_STRING_FORM_ID_TOKEN(pStmt->subDbName, pSubDbName); } else { pStmt->pQuery = pQuery; } @@ -1294,7 +1298,7 @@ SNode* createDropTopicStmt(SAstCreateContext* pCxt, bool ignoreNotExists, const CHECK_PARSER_STATUS(pCxt); SDropTopicStmt* pStmt = (SDropTopicStmt*)nodesMakeNode(QUERY_NODE_DROP_TOPIC_STMT); CHECK_OUT_OF_MEM(pStmt); - strncpy(pStmt->topicName, pTopicName->z, pTopicName->n); + COPY_STRING_FORM_ID_TOKEN(pStmt->topicName, pTopicName); pStmt->ignoreNotExists = ignoreNotExists; return (SNode*)pStmt; } @@ -1305,8 +1309,8 @@ SNode* createDropCGroupStmt(SAstCreateContext* pCxt, bool ignoreNotExists, const SDropCGroupStmt* pStmt = (SDropCGroupStmt*)nodesMakeNode(QUERY_NODE_DROP_CGROUP_STMT); CHECK_OUT_OF_MEM(pStmt); pStmt->ignoreNotExists = ignoreNotExists; - strncpy(pStmt->topicName, pTopicName->z, pTopicName->n); - strncpy(pStmt->cgroup, pCGroupId->z, pCGroupId->n); + COPY_STRING_FORM_ID_TOKEN(pStmt->topicName, pTopicName); + COPY_STRING_FORM_ID_TOKEN(pStmt->cgroup, pCGroupId); return (SNode*)pStmt; } @@ -1386,9 +1390,9 @@ SNode* createCreateFunctionStmt(SAstCreateContext* pCxt, bool ignoreExists, bool SCreateFunctionStmt* pStmt = (SCreateFunctionStmt*)nodesMakeNode(QUERY_NODE_CREATE_FUNCTION_STMT); CHECK_OUT_OF_MEM(pStmt); pStmt->ignoreExists = ignoreExists; - strncpy(pStmt->funcName, pFuncName->z, pFuncName->n); + COPY_STRING_FORM_ID_TOKEN(pStmt->funcName, pFuncName); pStmt->isAgg = aggFunc; - strncpy(pStmt->libraryPath, pLibPath->z + 1, pLibPath->n - 2); + COPY_STRING_FORM_STR_TOKEN(pStmt->libraryPath, pLibPath); pStmt->outputDt = dataType; pStmt->bufSize = bufSize; return (SNode*)pStmt; @@ -1399,7 +1403,7 @@ SNode* createDropFunctionStmt(SAstCreateContext* pCxt, bool ignoreNotExists, con SDropFunctionStmt* pStmt = (SDropFunctionStmt*)nodesMakeNode(QUERY_NODE_DROP_FUNCTION_STMT); CHECK_OUT_OF_MEM(pStmt); pStmt->ignoreNotExists = ignoreNotExists; - strncpy(pStmt->funcName, pFuncName->z, pFuncName->n); + COPY_STRING_FORM_ID_TOKEN(pStmt->funcName, pFuncName); return (SNode*)pStmt; } @@ -1416,7 +1420,7 @@ SNode* createCreateStreamStmt(SAstCreateContext* pCxt, bool ignoreExists, const CHECK_PARSER_STATUS(pCxt); SCreateStreamStmt* pStmt = (SCreateStreamStmt*)nodesMakeNode(QUERY_NODE_CREATE_STREAM_STMT); CHECK_OUT_OF_MEM(pStmt); - strncpy(pStmt->streamName, pStreamName->z, pStreamName->n); + COPY_STRING_FORM_ID_TOKEN(pStmt->streamName, pStreamName); if (NULL != pRealTable) { strcpy(pStmt->targetDbName, ((SRealTableNode*)pRealTable)->table.dbName); strcpy(pStmt->targetTabName, ((SRealTableNode*)pRealTable)->table.tableName); @@ -1432,7 +1436,7 @@ SNode* createDropStreamStmt(SAstCreateContext* pCxt, bool ignoreNotExists, const CHECK_PARSER_STATUS(pCxt); SDropStreamStmt* pStmt = (SDropStreamStmt*)nodesMakeNode(QUERY_NODE_DROP_STREAM_STMT); CHECK_OUT_OF_MEM(pStmt); - strncpy(pStmt->streamName, pStreamName->z, TMIN(pStreamName->n, sizeof(pStmt->streamName) - 1)); + COPY_STRING_FORM_ID_TOKEN(pStmt->streamName, pStreamName); pStmt->ignoreNotExists = ignoreNotExists; return (SNode*)pStmt; } @@ -1449,7 +1453,7 @@ SNode* createKillQueryStmt(SAstCreateContext* pCxt, const SToken* pQueryId) { CHECK_PARSER_STATUS(pCxt); SKillQueryStmt* pStmt = (SKillQueryStmt*)nodesMakeNode(QUERY_NODE_KILL_QUERY_STMT); CHECK_OUT_OF_MEM(pStmt); - strncpy(pStmt->queryId, pQueryId->z, TMIN(pQueryId->n, sizeof(pStmt->queryId) - 1)); + trimString(pQueryId->z, pQueryId->n, pStmt->queryId, sizeof(pStmt->queryId) - 1); return (SNode*)pStmt; } @@ -1501,8 +1505,8 @@ SNode* createGrantStmt(SAstCreateContext* pCxt, int64_t privileges, SToken* pDbN SGrantStmt* pStmt = (SGrantStmt*)nodesMakeNode(QUERY_NODE_GRANT_STMT); CHECK_OUT_OF_MEM(pStmt); pStmt->privileges = privileges; - strncpy(pStmt->dbName, pDbName->z, pDbName->n); - strncpy(pStmt->userName, pUserName->z, pUserName->n); + COPY_STRING_FORM_ID_TOKEN(pStmt->dbName, pDbName); + COPY_STRING_FORM_ID_TOKEN(pStmt->userName, pUserName); return (SNode*)pStmt; } @@ -1514,8 +1518,8 @@ SNode* createRevokeStmt(SAstCreateContext* pCxt, int64_t privileges, SToken* pDb SRevokeStmt* pStmt = (SRevokeStmt*)nodesMakeNode(QUERY_NODE_REVOKE_STMT); CHECK_OUT_OF_MEM(pStmt); pStmt->privileges = privileges; - strncpy(pStmt->dbName, pDbName->z, pDbName->n); - strncpy(pStmt->userName, pUserName->z, pUserName->n); + COPY_STRING_FORM_ID_TOKEN(pStmt->dbName, pDbName); + COPY_STRING_FORM_ID_TOKEN(pStmt->userName, pUserName); return (SNode*)pStmt; } diff --git a/source/libs/parser/src/parAstParser.c b/source/libs/parser/src/parAstParser.c index 1aa1320a3c3c4ed291792b450148b870606f71ec..e3218f972b0076e2d6ca3f7f2e303383658b24cb 100644 --- a/source/libs/parser/src/parAstParser.c +++ b/source/libs/parser/src/parAstParser.c @@ -387,6 +387,29 @@ static int32_t collectMetaKeyFromShowQueries(SCollectMetaKeyCxt* pCxt, SShowStmt pCxt->pMetaCache); } +static int32_t collectMetaKeyFromShowConfigs(SCollectMetaKeyCxt* pCxt, SShowStmt* pStmt) { + return reserveTableMetaInCache(pCxt->pParseCxt->acctId, TSDB_INFORMATION_SCHEMA_DB, TSDB_INS_TABLE_CONFIGS, + pCxt->pMetaCache); +} + +static int32_t collectMetaKeyFromShowVariables(SCollectMetaKeyCxt* pCxt, SShowStmt* pStmt) { + return reserveTableMetaInCache(pCxt->pParseCxt->acctId, TSDB_INFORMATION_SCHEMA_DB, TSDB_INS_TABLE_CONFIGS, + pCxt->pMetaCache); +} + +static int32_t collectMetaKeyFromShowCreateDatabase(SCollectMetaKeyCxt* pCxt, SShowCreateDatabaseStmt* pStmt) { + return reserveDbCfgInCache(pCxt->pParseCxt->acctId, pStmt->dbName, pCxt->pMetaCache); +} + +static int32_t collectMetaKeyFromShowCreateTable(SCollectMetaKeyCxt* pCxt, SShowCreateTableStmt* pStmt) { + return reserveTableMetaInCache(pCxt->pParseCxt->acctId, pStmt->dbName, pStmt->tableName, pCxt->pMetaCache); +} + +static int32_t collectMetaKeyFromShowApps(SCollectMetaKeyCxt* pCxt, SShowStmt* pStmt) { + return reserveTableMetaInCache(pCxt->pParseCxt->acctId, TSDB_PERFORMANCE_SCHEMA_DB, TSDB_PERFS_TABLE_APPS, + pCxt->pMetaCache); +} + static int32_t collectMetaKeyFromShowTransactions(SCollectMetaKeyCxt* pCxt, SShowStmt* pStmt) { return reserveTableMetaInCache(pCxt->pParseCxt->acctId, TSDB_PERFORMANCE_SCHEMA_DB, TSDB_PERFS_TABLE_TRANS, pCxt->pMetaCache); @@ -461,6 +484,15 @@ static int32_t collectMetaKeyFromQuery(SCollectMetaKeyCxt* pCxt, SNode* pStmt) { return collectMetaKeyFromShowConnections(pCxt, (SShowStmt*)pStmt); case QUERY_NODE_SHOW_QUERIES_STMT: return collectMetaKeyFromShowQueries(pCxt, (SShowStmt*)pStmt); + case QUERY_NODE_SHOW_VARIABLE_STMT: + return collectMetaKeyFromShowVariables(pCxt, (SShowStmt*)pStmt); + case QUERY_NODE_SHOW_CREATE_DATABASE_STMT: + return collectMetaKeyFromShowCreateDatabase(pCxt, (SShowCreateDatabaseStmt*)pStmt); + case QUERY_NODE_SHOW_CREATE_TABLE_STMT: + case QUERY_NODE_SHOW_CREATE_STABLE_STMT: + return collectMetaKeyFromShowCreateTable(pCxt, (SShowCreateTableStmt*)pStmt); + case QUERY_NODE_SHOW_APPS_STMT: + return collectMetaKeyFromShowApps(pCxt, (SShowStmt*)pStmt); case QUERY_NODE_SHOW_TRANSACTIONS_STMT: return collectMetaKeyFromShowTransactions(pCxt, (SShowStmt*)pStmt); case QUERY_NODE_DELETE_STMT: diff --git a/source/libs/parser/src/parTokenizer.c b/source/libs/parser/src/parTokenizer.c index 55ea7171fa41c185453ade03655d0d70c936f72b..41d32204dac66a4ec7fb65dd1415764f85b1b8cb 100644 --- a/source/libs/parser/src/parTokenizer.c +++ b/source/libs/parser/src/parTokenizer.c @@ -81,7 +81,7 @@ static SKeyword keywordTable[] = { {"DURATION", TK_DURATION}, {"EXISTS", TK_EXISTS}, {"EXPLAIN", TK_EXPLAIN}, - {"FILE_FACTOR", TK_FILE_FACTOR}, + // {"FILE_FACTOR", TK_FILE_FACTOR}, {"FILL", TK_FILL}, {"FIRST", TK_FIRST}, {"FLOAT", TK_FLOAT}, diff --git a/source/libs/parser/src/parTranslater.c b/source/libs/parser/src/parTranslater.c index 5d26a98dc868d80f3230dfd5595e0af377e55ece..3cb885c8e177b3bcdd857ccdf97537266f94be84 100644 --- a/source/libs/parser/src/parTranslater.c +++ b/source/libs/parser/src/parTranslater.c @@ -36,6 +36,7 @@ typedef struct STranslateContext { int32_t currLevel; ESqlClause currClause; SSelectStmt* pCurrSelectStmt; + SSetOperator* pCurrSetOperator; SCmdMsgInfo* pCmdMsg; SHashObj* pDbs; SHashObj* pTables; @@ -432,7 +433,7 @@ static void setColumnInfoBySchema(const SRealTableNode* pTable, const SSchema* p static void setColumnInfoByExpr(const STableNode* pTable, SExprNode* pExpr, SColumnNode** pColRef) { SColumnNode* pCol = *pColRef; - pCol->pProjectRef = (SNode*)pExpr; + // pCol->pProjectRef = (SNode*)pExpr; if (NULL == pExpr->pAssociation) { pExpr->pAssociation = taosArrayInit(TARRAY_MIN_SIZE, POINTER_BYTES); } @@ -613,17 +614,36 @@ static EDealRes translateColumnWithoutPrefix(STranslateContext* pCxt, SColumnNod return DEAL_RES_CONTINUE; } -static bool translateColumnUseAlias(STranslateContext* pCxt, SColumnNode** pCol) { - SNodeList* pProjectionList = pCxt->pCurrSelectStmt->pProjectionList; +static SNodeList* getProjectListFromCxt(STranslateContext* pCxt) { + if (NULL != pCxt->pCurrSelectStmt) { + return pCxt->pCurrSelectStmt->pProjectionList; + } else if (NULL != pCxt->pCurrSetOperator) { + return pCxt->pCurrSetOperator->pProjectionList; + } else { + return NULL; + } +} + +static EDealRes translateColumnUseAlias(STranslateContext* pCxt, SColumnNode** pCol, bool* pFound) { + SNodeList* pProjectionList = getProjectListFromCxt(pCxt); SNode* pNode; FOREACH(pNode, pProjectionList) { SExprNode* pExpr = (SExprNode*)pNode; if (0 == strcmp((*pCol)->colName, pExpr->aliasName)) { - setColumnInfoByExpr(NULL, pExpr, pCol); - return true; + SColumnRefNode* pColRef = (SColumnRefNode*)nodesMakeNode(QUERY_NODE_COLUMN_REF); + if (NULL == pColRef) { + pCxt->errCode = TSDB_CODE_OUT_OF_MEMORY; + return DEAL_RES_ERROR; + } + strcpy(pColRef->colName, pExpr->aliasName); + nodesDestroyNode(*(SNode**)pCol); + *(SNode**)pCol = (SNode*)pColRef; + *pFound = true; + return DEAL_RES_CONTINUE; } } - return false; + *pFound = false; + return DEAL_RES_CONTINUE; } static EDealRes translateColumn(STranslateContext* pCxt, SColumnNode** pCol) { @@ -638,9 +658,11 @@ static EDealRes translateColumn(STranslateContext* pCxt, SColumnNode** pCol) { } else { bool found = false; if (SQL_CLAUSE_ORDER_BY == pCxt->currClause) { - found = translateColumnUseAlias(pCxt, pCol); + res = translateColumnUseAlias(pCxt, pCol, &found); + } + if (DEAL_RES_ERROR != res && !found) { + res = translateColumnWithoutPrefix(pCxt, pCol); } - res = (found ? DEAL_RES_CONTINUE : translateColumnWithoutPrefix(pCxt, pCol)); } return res; } @@ -1098,11 +1120,43 @@ static int32_t translateWindowPseudoColumnFunc(STranslateContext* pCxt, SFunctio return TSDB_CODE_SUCCESS; } +static int32_t translateForbidWindowFunc(STranslateContext* pCxt, SFunctionNode* pFunc) { + if (!fmIsForbidWindowFunc(pFunc->funcId)) { + return TSDB_CODE_SUCCESS; + } + if (NULL != pCxt->pCurrSelectStmt->pWindow) { + return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_WINDOW_NOT_ALLOWED_FUNC, pFunc->functionName); + } + return TSDB_CODE_SUCCESS; +} + +static int32_t translateForbidStreamFunc(STranslateContext* pCxt, SFunctionNode* pFunc) { + if (!fmIsForbidStreamFunc(pFunc->funcId)) { + return TSDB_CODE_SUCCESS; + } + if (pCxt->createStream) { + return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_STREAM_NOT_ALLOWED_FUNC, pFunc->functionName); + } + return TSDB_CODE_SUCCESS; +} + +static int32_t translateForbidGroupByFunc(STranslateContext* pCxt, SFunctionNode* pFunc) { + if (!fmIsForbidGroupByFunc(pFunc->funcId)) { + return TSDB_CODE_SUCCESS; + } + if (NULL != pCxt->pCurrSelectStmt->pGroupByList) { + return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_GROUP_BY_NOT_ALLOWED_FUNC, pFunc->functionName); + } + return TSDB_CODE_SUCCESS; +} + static void setFuncClassification(SSelectStmt* pSelect, SFunctionNode* pFunc) { if (NULL != pSelect) { pSelect->hasAggFuncs = pSelect->hasAggFuncs ? true : fmIsAggFunc(pFunc->funcId); pSelect->hasRepeatScanFuncs = pSelect->hasRepeatScanFuncs ? true : fmIsRepeatScanFunc(pFunc->funcId); pSelect->hasIndefiniteRowsFunc = pSelect->hasIndefiniteRowsFunc ? true : fmIsIndefiniteRowsFunc(pFunc->funcId); + pSelect->hasUniqueFunc = pSelect->hasUniqueFunc ? true : (FUNCTION_TYPE_UNIQUE == pFunc->funcType); + pSelect->hasTailFunc = pSelect->hasTailFunc ? true : (FUNCTION_TYPE_TAIL == pFunc->funcType); } } @@ -1130,6 +1184,15 @@ static EDealRes translateFunction(STranslateContext* pCxt, SFunctionNode* pFunc) if (TSDB_CODE_SUCCESS == pCxt->errCode) { pCxt->errCode = translateWindowPseudoColumnFunc(pCxt, pFunc); } + if (TSDB_CODE_SUCCESS == pCxt->errCode) { + pCxt->errCode = translateForbidWindowFunc(pCxt, pFunc); + } + if (TSDB_CODE_SUCCESS == pCxt->errCode) { + pCxt->errCode = translateForbidStreamFunc(pCxt, pFunc); + } + if (TSDB_CODE_SUCCESS == pCxt->errCode) { + pCxt->errCode = translateForbidGroupByFunc(pCxt, pFunc); + } if (TSDB_CODE_SUCCESS == pCxt->errCode) { setFuncClassification(pCxt->pCurrSelectStmt, pFunc); } @@ -1387,7 +1450,8 @@ static int32_t addMnodeToVgroupList(const SEpSet* pEpSet, SArray** pVgroupList) } static int32_t setSysTableVgroupList(STranslateContext* pCxt, SName* pName, SRealTableNode* pRealTable) { - if (0 != strcmp(pRealTable->table.tableName, TSDB_INS_TABLE_USER_TABLES)) { + if (0 != strcmp(pRealTable->table.tableName, TSDB_INS_TABLE_USER_TABLES) && + 0 != strcmp(pRealTable->table.tableName, TSDB_INS_TABLE_USER_TABLE_DISTRIBUTED)) { return TSDB_CODE_SUCCESS; } @@ -1468,7 +1532,8 @@ static bool joinTableIsSingleTable(SJoinTableNode* pJoinTable) { static bool isSingleTable(SRealTableNode* pRealTable) { int8_t tableType = pRealTable->pMeta->tableType; if (TSDB_SYSTEM_TABLE == tableType) { - return 0 != strcmp(pRealTable->table.tableName, TSDB_INS_TABLE_USER_TABLES); + return 0 != strcmp(pRealTable->table.tableName, TSDB_INS_TABLE_USER_TABLES) && + 0 != strcmp(pRealTable->table.tableName, TSDB_INS_TABLE_USER_TABLE_DISTRIBUTED); } return (TSDB_CHILD_TABLE == tableType || TSDB_NORMAL_TABLE == tableType); } @@ -1762,11 +1827,11 @@ static int32_t translateOrderByPosition(STranslateContext* pCxt, SNodeList* pPro } else if (0 == pos || pos > LIST_LENGTH(pProjectionList)) { return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_WRONG_NUMBER_OF_SELECT); } else { - SColumnNode* pCol = (SColumnNode*)nodesMakeNode(QUERY_NODE_COLUMN); + SColumnRefNode* pCol = (SColumnRefNode*)nodesMakeNode(QUERY_NODE_COLUMN_REF); if (NULL == pCol) { return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_OUT_OF_MEMORY); } - setColumnInfoByExpr(NULL, (SExprNode*)nodesListGetNode(pProjectionList, pos - 1), &pCol); + strcpy(pCol->colName, ((SExprNode*)nodesListGetNode(pProjectionList, pos - 1))->aliasName); ((SOrderByExprNode*)pNode)->pExpr = (SNode*)pCol; nodesDestroyNode(pExpr); } @@ -1781,16 +1846,15 @@ static int32_t translateOrderByPosition(STranslateContext* pCxt, SNodeList* pPro static int32_t translateOrderBy(STranslateContext* pCxt, SSelectStmt* pSelect) { bool other; int32_t code = translateOrderByPosition(pCxt, pSelect->pProjectionList, pSelect->pOrderByList, &other); - if (TSDB_CODE_SUCCESS != code) { - return code; - } - if (!other) { - return TSDB_CODE_SUCCESS; - } - pCxt->currClause = SQL_CLAUSE_ORDER_BY; - code = translateExprList(pCxt, pSelect->pOrderByList); if (TSDB_CODE_SUCCESS == code) { - code = checkExprListForGroupBy(pCxt, pSelect->pOrderByList); + if (!other) { + return TSDB_CODE_SUCCESS; + } + pCxt->currClause = SQL_CLAUSE_ORDER_BY; + code = translateExprList(pCxt, pSelect->pOrderByList); + if (TSDB_CODE_SUCCESS == code) { + code = checkExprListForGroupBy(pCxt, pSelect->pOrderByList); + } } return code; } @@ -2116,6 +2180,172 @@ static int32_t rewriteTimelineFunc(STranslateContext* pCxt, SSelectStmt* pSelect return pCxt->errCode; } +typedef struct SRwriteUniqueCxt { + STranslateContext* pTranslateCxt; + SNode* pExpr; +} SRwriteUniqueCxt; + +static EDealRes rewriteSeletcValueFunc(STranslateContext* pCxt, SNode** pNode) { + SFunctionNode* pFirst = (SFunctionNode*)nodesMakeNode(QUERY_NODE_FUNCTION); + if (NULL == pFirst) { + pCxt->errCode = TSDB_CODE_OUT_OF_MEMORY; + return DEAL_RES_ERROR; + } + strcpy(pFirst->functionName, "first"); + TSWAP(pFirst->pParameterList, ((SFunctionNode*)*pNode)->pParameterList); + strcpy(pFirst->node.aliasName, ((SExprNode*)*pNode)->aliasName); + nodesDestroyNode(*pNode); + *pNode = (SNode*)pFirst; + pCxt->errCode = fmGetFuncInfo(pFirst, pCxt->msgBuf.buf, pCxt->msgBuf.len); + pCxt->pCurrSelectStmt->hasAggFuncs = true; + return TSDB_CODE_SUCCESS == pCxt->errCode ? DEAL_RES_IGNORE_CHILD : DEAL_RES_ERROR; +} + +static EDealRes rewriteUniqueFunc(SNode** pNode, void* pContext) { + SRwriteUniqueCxt* pCxt = pContext; + if (QUERY_NODE_FUNCTION == nodeType(*pNode)) { + SFunctionNode* pFunc = (SFunctionNode*)*pNode; + if (FUNCTION_TYPE_UNIQUE == pFunc->funcType) { + SNode* pExpr = nodesListGetNode(pFunc->pParameterList, 0); + NODES_CLEAR_LIST(pFunc->pParameterList); + strcpy(((SExprNode*)pExpr)->aliasName, ((SExprNode*)*pNode)->aliasName); + nodesDestroyNode(*pNode); + *pNode = pExpr; + pCxt->pExpr = pExpr; + return DEAL_RES_IGNORE_CHILD; + } else if (FUNCTION_TYPE_SELECT_VALUE == pFunc->funcType) { + return rewriteSeletcValueFunc(pCxt->pTranslateCxt, pNode); + } + } + return DEAL_RES_CONTINUE; +} + +static SNode* createGroupingSet(SNode* pExpr) { + SGroupingSetNode* pGroupingSet = (SGroupingSetNode*)nodesMakeNode(QUERY_NODE_GROUPING_SET); + if (NULL == pGroupingSet) { + return NULL; + } + pGroupingSet->groupingSetType = GP_TYPE_NORMAL; + if (TSDB_CODE_SUCCESS != nodesListMakeStrictAppend(&pGroupingSet->pParameterList, nodesCloneNode(pExpr))) { + nodesDestroyNode((SNode*)pGroupingSet); + return NULL; + } + return (SNode*)pGroupingSet; +} + +static int32_t rewriteUniqueStmt(STranslateContext* pCxt, SSelectStmt* pSelect) { + if (!pSelect->hasUniqueFunc) { + return TSDB_CODE_SUCCESS; + } + + SRwriteUniqueCxt cxt = {.pTranslateCxt = pCxt, .pExpr = NULL}; + nodesRewriteExprs(pSelect->pProjectionList, rewriteUniqueFunc, &cxt); + if (TSDB_CODE_SUCCESS == cxt.pTranslateCxt->errCode) { + cxt.pTranslateCxt->errCode = nodesListMakeStrictAppend(&pSelect->pGroupByList, createGroupingSet(cxt.pExpr)); + } + pSelect->hasIndefiniteRowsFunc = false; + return cxt.pTranslateCxt->errCode; +} + +typedef struct SRwriteTailCxt { + STranslateContext* pTranslateCxt; + int64_t limit; + int64_t offset; +} SRwriteTailCxt; + +static EDealRes rewriteTailFunc(SNode** pNode, void* pContext) { + SRwriteTailCxt* pCxt = pContext; + if (QUERY_NODE_FUNCTION == nodeType(*pNode)) { + SFunctionNode* pFunc = (SFunctionNode*)*pNode; + if (FUNCTION_TYPE_TAIL == pFunc->funcType) { + pCxt->limit = ((SValueNode*)nodesListGetNode(pFunc->pParameterList, 1))->datum.i; + if (3 == LIST_LENGTH(pFunc->pParameterList)) { + pCxt->offset = ((SValueNode*)nodesListGetNode(pFunc->pParameterList, 2))->datum.i; + } + SNode* pExpr = nodesListGetNode(pFunc->pParameterList, 0); + strcpy(((SExprNode*)pExpr)->aliasName, ((SExprNode*)*pNode)->aliasName); + NODES_CLEAR_LIST(pFunc->pParameterList); + nodesDestroyNode(*pNode); + *pNode = pExpr; + return DEAL_RES_IGNORE_CHILD; + } + } + return DEAL_RES_CONTINUE; +} + +static int32_t createLimieNode(SRwriteTailCxt* pCxt, SLimitNode** pOutput) { + *pOutput = (SLimitNode*)nodesMakeNode(QUERY_NODE_LIMIT); + if (NULL == *pOutput) { + return TSDB_CODE_OUT_OF_MEMORY; + } + (*pOutput)->limit = pCxt->limit; + (*pOutput)->offset = pCxt->offset; + return TSDB_CODE_SUCCESS; +} + +static SNode* createOrderByExpr(STranslateContext* pCxt) { + SOrderByExprNode* pOrder = (SOrderByExprNode*)nodesMakeNode(QUERY_NODE_ORDER_BY_EXPR); + if (NULL == pOrder) { + return NULL; + } + pCxt->errCode = createPrimaryKeyCol(pCxt, &pOrder->pExpr); + if (TSDB_CODE_SUCCESS != pCxt->errCode) { + nodesDestroyNode((SNode*)pOrder); + return NULL; + } + pOrder->order = ORDER_DESC; + pOrder->nullOrder = NULL_ORDER_FIRST; + return (SNode*)pOrder; +} + +static int32_t rewriteTailStmt(STranslateContext* pCxt, SSelectStmt* pSelect) { + if (!pSelect->hasTailFunc) { + return TSDB_CODE_SUCCESS; + } + + SRwriteTailCxt cxt = {.pTranslateCxt = pCxt, .limit = -1, .offset = -1}; + nodesRewriteExprs(pSelect->pProjectionList, rewriteTailFunc, &cxt); + int32_t code = nodesListMakeStrictAppend(&pSelect->pOrderByList, createOrderByExpr(pCxt)); + if (TSDB_CODE_SUCCESS == code) { + code = createLimieNode(&cxt, &pSelect->pLimit); + } + pSelect->hasIndefiniteRowsFunc = false; + return code; +} + +typedef struct SReplaceOrderByAliasCxt { + STranslateContext* pTranslateCxt; + SNodeList* pProjectionList; +} SReplaceOrderByAliasCxt; + +static EDealRes replaceOrderByAliasImpl(SNode** pNode, void* pContext) { + SReplaceOrderByAliasCxt* pCxt = pContext; + if (QUERY_NODE_COLUMN_REF == nodeType(*pNode)) { + SNodeList* pProjectionList = pCxt->pProjectionList; + SNode* pProject = NULL; + FOREACH(pProject, pProjectionList) { + SExprNode* pExpr = (SExprNode*)pProject; + if (0 == strcmp(((SColumnRefNode*)*pNode)->colName, pExpr->aliasName)) { + SNode* pNew = nodesCloneNode(pProject); + if (NULL == pNew) { + pCxt->pTranslateCxt->errCode = TSDB_CODE_OUT_OF_MEMORY; + return DEAL_RES_ERROR; + } + nodesDestroyNode(*pNode); + *pNode = pNew; + return DEAL_RES_CONTINUE; + } + } + } + return DEAL_RES_CONTINUE; +} + +static int32_t replaceOrderByAlias(STranslateContext* pCxt, SNodeList* pProjectionList, SNodeList* pOrderByList) { + SReplaceOrderByAliasCxt cxt = {.pTranslateCxt = pCxt, .pProjectionList = pProjectionList}; + nodesRewriteExprsPostOrder(pOrderByList, replaceOrderByAliasImpl, &cxt); + return pCxt->errCode; +} + static int32_t translateSelect(STranslateContext* pCxt, SSelectStmt* pSelect) { pCxt->pCurrSelectStmt = pSelect; int32_t code = translateFrom(pCxt, pSelect->pFromTable); @@ -2147,9 +2377,18 @@ static int32_t translateSelect(STranslateContext* pCxt, SSelectStmt* pSelect) { if (TSDB_CODE_SUCCESS == code) { code = checkLimit(pCxt, pSelect); } + if (TSDB_CODE_SUCCESS == code) { + code = rewriteUniqueStmt(pCxt, pSelect); + } + // if (TSDB_CODE_SUCCESS == code) { + // code = rewriteTailStmt(pCxt, pSelect); + // } if (TSDB_CODE_SUCCESS == code) { code = rewriteTimelineFunc(pCxt, pSelect); } + if (TSDB_CODE_SUCCESS == code) { + code = replaceOrderByAlias(pCxt, pSelect->pProjectionList, pSelect->pOrderByList); + } return code; } @@ -2186,7 +2425,7 @@ static int32_t createCastFunc(STranslateContext* pCxt, SNode* pExpr, SDataType d return TSDB_CODE_SUCCESS; } -static int32_t translateSetOperatorImpl(STranslateContext* pCxt, SSetOperator* pSetOperator) { +static int32_t translateSetOperProject(STranslateContext* pCxt, SSetOperator* pSetOperator) { SNodeList* pLeftProjections = getProjectList(pSetOperator->pLeft); SNodeList* pRightProjections = getProjectList(pSetOperator->pRight); if (LIST_LENGTH(pLeftProjections) != LIST_LENGTH(pRightProjections)) { @@ -2221,6 +2460,23 @@ static uint8_t calcSetOperatorPrecision(SSetOperator* pSetOperator) { return calcPrecision(getStmtPrecision(pSetOperator->pLeft), getStmtPrecision(pSetOperator->pRight)); } +static int32_t translateSetOperOrderBy(STranslateContext* pCxt, SSetOperator* pSetOperator) { + bool other; + int32_t code = translateOrderByPosition(pCxt, pSetOperator->pProjectionList, pSetOperator->pOrderByList, &other); + if (TSDB_CODE_SUCCESS == code) { + if (other) { + pCxt->currClause = SQL_CLAUSE_ORDER_BY; + pCxt->pCurrSelectStmt = NULL; + pCxt->pCurrSetOperator = pSetOperator; + code = translateExprList(pCxt, pSetOperator->pOrderByList); + } + } + if (TSDB_CODE_SUCCESS == code) { + code = replaceOrderByAlias(pCxt, pSetOperator->pProjectionList, pSetOperator->pOrderByList); + } + return code; +} + static int32_t translateSetOperator(STranslateContext* pCxt, SSetOperator* pSetOperator) { int32_t code = translateQuery(pCxt, pSetOperator->pLeft); if (TSDB_CODE_SUCCESS == code) { @@ -2231,7 +2487,10 @@ static int32_t translateSetOperator(STranslateContext* pCxt, SSetOperator* pSetO } if (TSDB_CODE_SUCCESS == code) { pSetOperator->precision = calcSetOperatorPrecision(pSetOperator); - code = translateSetOperatorImpl(pCxt, pSetOperator); + code = translateSetOperProject(pCxt, pSetOperator); + } + if (TSDB_CODE_SUCCESS == code) { + code = translateSetOperOrderBy(pCxt, pSetOperator); } return code; } @@ -2366,7 +2625,7 @@ static int32_t checkDbDaysOption(STranslateContext* pCxt, SDatabaseOptions* pOpt if (TIME_UNIT_MINUTE != pOptions->pDaysPerFile->unit && TIME_UNIT_HOUR != pOptions->pDaysPerFile->unit && TIME_UNIT_DAY != pOptions->pDaysPerFile->unit) { return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_INVALID_OPTION_UNIT, "daysPerFile", - pOptions->pDaysPerFile->unit); + pOptions->pDaysPerFile->unit, TIME_UNIT_MINUTE, TIME_UNIT_HOUR, TIME_UNIT_DAY); } pOptions->daysPerFile = getBigintFromValueNode(pOptions->pDaysPerFile); } @@ -2668,14 +2927,6 @@ static SColumnDefNode* findColDef(SNodeList* pCols, const SColumnNode* pCol) { return NULL; } -static int32_t checTableFactorOption(STranslateContext* pCxt, float val) { - if (val < TSDB_MIN_ROLLUP_FILE_FACTOR || val > TSDB_MAX_ROLLUP_FILE_FACTOR) { - return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_INVALID_F_RANGE_OPTION, "file_factor", val, - TSDB_MIN_ROLLUP_FILE_FACTOR, TSDB_MAX_ROLLUP_FILE_FACTOR); - } - return TSDB_CODE_SUCCESS; -} - static int32_t checkTableSmaOption(STranslateContext* pCxt, SCreateTableStmt* pStmt) { if (NULL != pStmt->pOptions->pSma) { SNode* pNode = NULL; @@ -2825,27 +3076,74 @@ static int32_t checkTableSchema(STranslateContext* pCxt, SCreateTableStmt* pStmt return code; } -static int32_t checkSchemalessDb(STranslateContext* pCxt, const char* pDbName) { - // if (0 != pCxt->pParseCxt->schemalessType) { - // return TSDB_CODE_SUCCESS; - // } - // SDbCfgInfo info = {0}; - // int32_t code = getDBCfg(pCxt, pDbName, &info); - // if (TSDB_CODE_SUCCESS == code) { - // code = info.schemaless ? TSDB_CODE_SML_INVALID_DB_CONF : TSDB_CODE_SUCCESS; - // } - // return code; - return TSDB_CODE_SUCCESS; +static int32_t getTableDelayOrWatermarkOption(STranslateContext* pCxt, const char* pName, int64_t minVal, + int64_t maxVal, SValueNode* pVal, int64_t* pMaxDelay) { + int32_t code = (DEAL_RES_ERROR == translateValue(pCxt, pVal) ? pCxt->errCode : TSDB_CODE_SUCCESS); + if (TSDB_CODE_SUCCESS == code && TIME_UNIT_MILLISECOND != pVal->unit && TIME_UNIT_SECOND != pVal->unit && + TIME_UNIT_MINUTE != pVal->unit) { + code = generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_INVALID_OPTION_UNIT, pName, pVal->unit, + TIME_UNIT_MILLISECOND, TIME_UNIT_SECOND, TIME_UNIT_MINUTE); + } + if (TSDB_CODE_SUCCESS == code) { + code = checkRangeOption(pCxt, pName, pVal->datum.i, minVal, maxVal); + } + if (TSDB_CODE_SUCCESS == code) { + *pMaxDelay = pVal->datum.i; + } + return code; +} + +static int32_t getTableMaxDelayOption(STranslateContext* pCxt, SValueNode* pVal, int64_t* pMaxDelay) { + return getTableDelayOrWatermarkOption(pCxt, "maxDelay", TSDB_MIN_ROLLUP_MAX_DELAY, TSDB_MAX_ROLLUP_MAX_DELAY, pVal, + pMaxDelay); +} + +static int32_t checkTableMaxDelayOption(STranslateContext* pCxt, STableOptions* pOptions) { + if (NULL == pOptions->pMaxDelay) { + return TSDB_CODE_SUCCESS; + } + + if (LIST_LENGTH(pOptions->pMaxDelay) > 2) { + return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_INVALID_TABLE_OPTION, "maxdelay"); + } + + int32_t code = + getTableMaxDelayOption(pCxt, (SValueNode*)nodesListGetNode(pOptions->pMaxDelay, 0), &pOptions->maxDelay1); + if (TSDB_CODE_SUCCESS == code && 2 == LIST_LENGTH(pOptions->pMaxDelay)) { + code = getTableMaxDelayOption(pCxt, (SValueNode*)nodesListGetNode(pOptions->pMaxDelay, 1), &pOptions->maxDelay2); + } + + return code; +} + +static int32_t getTableWatermarkOption(STranslateContext* pCxt, SValueNode* pVal, int64_t* pMaxDelay) { + return getTableDelayOrWatermarkOption(pCxt, "watermark", TSDB_MIN_ROLLUP_WATERMARK, TSDB_MAX_ROLLUP_WATERMARK, pVal, + pMaxDelay); +} + +static int32_t checkTableWatermarkOption(STranslateContext* pCxt, STableOptions* pOptions) { + if (NULL == pOptions->pWatermark) { + return TSDB_CODE_SUCCESS; + } + + if (LIST_LENGTH(pOptions->pWatermark) > 2) { + return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_INVALID_TABLE_OPTION, "watermark"); + } + + int32_t code = + getTableWatermarkOption(pCxt, (SValueNode*)nodesListGetNode(pOptions->pWatermark, 0), &pOptions->watermark1); + if (TSDB_CODE_SUCCESS == code && 2 == LIST_LENGTH(pOptions->pWatermark)) { + code = getTableWatermarkOption(pCxt, (SValueNode*)nodesListGetNode(pOptions->pWatermark, 1), &pOptions->watermark2); + } + + return code; } static int32_t checkCreateTable(STranslateContext* pCxt, SCreateTableStmt* pStmt) { - int32_t code = checkSchemalessDb(pCxt, pStmt->dbName); + int32_t code = checkTableMaxDelayOption(pCxt, pStmt->pOptions); if (TSDB_CODE_SUCCESS == code) { - code = checTableFactorOption(pCxt, pStmt->pOptions->filesFactor); + code = checkTableWatermarkOption(pCxt, pStmt->pOptions); } - // if (TSDB_CODE_SUCCESS == code) { - // code = checkRangeOption(pCxt, "delay", pStmt->pOptions->delay, TSDB_MIN_ROLLUP_DELAY, TSDB_MAX_ROLLUP_DELAY); - // } if (TSDB_CODE_SUCCESS == code) { code = checkTableRollupOption(pCxt, pStmt->pOptions->pRollupFuncs); } @@ -3088,8 +3386,10 @@ static int32_t buildRollupAst(STranslateContext* pCxt, SCreateTableStmt* pStmt, static int32_t buildCreateStbReq(STranslateContext* pCxt, SCreateTableStmt* pStmt, SMCreateStbReq* pReq) { pReq->igExists = pStmt->ignoreExists; - // pReq->delay = pStmt->pOptions->delay; - pReq->xFilesFactor = pStmt->pOptions->filesFactor; + pReq->delay1 = pStmt->pOptions->maxDelay1; + pReq->delay2 = pStmt->pOptions->maxDelay2; + pReq->watermark1 = pStmt->pOptions->watermark1; + pReq->watermark2 = pStmt->pOptions->watermark2; pReq->ttl = pStmt->pOptions->ttl; columnDefNodeToField(pStmt->pCols, &pReq->pColumns); columnDefNodeToField(pStmt->pTags, &pReq->pTags); @@ -3314,27 +3614,6 @@ static int32_t translateAlterDnode(STranslateContext* pCxt, SAlterDnodeStmt* pSt return buildCmdMsg(pCxt, TDMT_MND_CONFIG_DNODE, (FSerializeFunc)tSerializeSMCfgDnodeReq, &cfgReq); } -static int32_t nodeTypeToShowType(ENodeType nt) { - switch (nt) { - case QUERY_NODE_SHOW_CONNECTIONS_STMT: - return TSDB_MGMT_TABLE_CONNS; - case QUERY_NODE_SHOW_LICENCE_STMT: - return TSDB_MGMT_TABLE_GRANTS; - case QUERY_NODE_SHOW_QUERIES_STMT: - return TSDB_MGMT_TABLE_QUERIES; - case QUERY_NODE_SHOW_VARIABLE_STMT: - return TSDB_MGMT_TABLE_CONFIGS; - default: - break; - } - return 0; -} - -static int32_t translateShow(STranslateContext* pCxt, SShowStmt* pStmt) { - SShowReq showReq = {.type = nodeTypeToShowType(nodeType(pStmt))}; - return buildCmdMsg(pCxt, TDMT_MND_SHOW, (FSerializeFunc)tSerializeSShowReq, &showReq); -} - static int32_t getSmaIndexDstVgId(STranslateContext* pCxt, char* pTableName, int32_t* pVgId) { SVgroupInfo vg = {0}; int32_t code = getTableHashVgroup(pCxt, pCxt->pParseCxt->db, pTableName, &vg); @@ -3633,7 +3912,7 @@ static int32_t translateDescribe(STranslateContext* pCxt, SDescribeStmt* pStmt) static int32_t translateKillConnection(STranslateContext* pCxt, SKillStmt* pStmt) { SKillConnReq killReq = {0}; killReq.connId = pStmt->targetId; - return buildCmdMsg(pCxt, TDMT_MND_KILL_CONN, (FSerializeFunc)tSerializeSKillQueryReq, &killReq); + return buildCmdMsg(pCxt, TDMT_MND_KILL_CONN, (FSerializeFunc)tSerializeSKillConnReq, &killReq); } static int32_t translateKillQuery(STranslateContext* pCxt, SKillQueryStmt* pStmt) { @@ -3880,6 +4159,18 @@ static int32_t translateSplitVgroup(STranslateContext* pCxt, SSplitVgroupStmt* p return buildCmdMsg(pCxt, TDMT_MND_SPLIT_VGROUP, (FSerializeFunc)tSerializeSSplitVgroupReq, &req); } +static int32_t translateShowCreateDatabase(STranslateContext* pCxt, SShowCreateDatabaseStmt* pStmt) { + pStmt->pCfg = taosMemoryCalloc(1, sizeof(SDbCfgInfo)); + if (NULL == pStmt->pCfg) { + return TSDB_CODE_OUT_OF_MEMORY; + } + return getDBCfg(pCxt, pStmt->dbName, (SDbCfgInfo*)pStmt->pCfg); +} + +static int32_t translateShowCreateTable(STranslateContext* pCxt, SShowCreateTableStmt* pStmt) { + return getTableMeta(pCxt, pStmt->dbName, pStmt->tableName, &pStmt->pMeta); +} + static int32_t translateQuery(STranslateContext* pCxt, SNode* pNode) { int32_t code = TSDB_CODE_SUCCESS; switch (nodeType(pNode)) { @@ -3934,12 +4225,6 @@ static int32_t translateQuery(STranslateContext* pCxt, SNode* pNode) { case QUERY_NODE_ALTER_DNODE_STMT: code = translateAlterDnode(pCxt, (SAlterDnodeStmt*)pNode); break; - case QUERY_NODE_SHOW_CONNECTIONS_STMT: - case QUERY_NODE_SHOW_QUERIES_STMT: - case QUERY_NODE_SHOW_TOPICS_STMT: - case QUERY_NODE_SHOW_VARIABLE_STMT: - code = translateShow(pCxt, (SShowStmt*)pNode); - break; case QUERY_NODE_CREATE_INDEX_STMT: code = translateCreateIndex(pCxt, (SCreateIndexStmt*)pNode); break; @@ -4015,6 +4300,13 @@ static int32_t translateQuery(STranslateContext* pCxt, SNode* pNode) { case QUERY_NODE_SPLIT_VGROUP_STMT: code = translateSplitVgroup(pCxt, (SSplitVgroupStmt*)pNode); break; + case QUERY_NODE_SHOW_CREATE_DATABASE_STMT: + code = translateShowCreateDatabase(pCxt, (SShowCreateDatabaseStmt*)pNode); + break; + case QUERY_NODE_SHOW_CREATE_TABLE_STMT: + case QUERY_NODE_SHOW_CREATE_STABLE_STMT: + code = translateShowCreateTable(pCxt, (SShowCreateTableStmt*)pNode); + break; default: break; } @@ -4097,6 +4389,42 @@ static int32_t extractDescribeResultSchema(int32_t* numOfCols, SSchema** pSchema return TSDB_CODE_SUCCESS; } +static int32_t extractShowCreateDatabaseResultSchema(int32_t* numOfCols, SSchema** pSchema) { + *numOfCols = 2; + *pSchema = taosMemoryCalloc((*numOfCols), sizeof(SSchema)); + if (NULL == (*pSchema)) { + return TSDB_CODE_OUT_OF_MEMORY; + } + + (*pSchema)[0].type = TSDB_DATA_TYPE_BINARY; + (*pSchema)[0].bytes = TSDB_DB_NAME_LEN; + strcpy((*pSchema)[0].name, "Database"); + + (*pSchema)[1].type = TSDB_DATA_TYPE_BINARY; + (*pSchema)[1].bytes = TSDB_MAX_BINARY_LEN; + strcpy((*pSchema)[1].name, "Create Database"); + + return TSDB_CODE_SUCCESS; +} + +static int32_t extractShowCreateTableResultSchema(int32_t* numOfCols, SSchema** pSchema) { + *numOfCols = 2; + *pSchema = taosMemoryCalloc((*numOfCols), sizeof(SSchema)); + if (NULL == (*pSchema)) { + return TSDB_CODE_OUT_OF_MEMORY; + } + + (*pSchema)[0].type = TSDB_DATA_TYPE_BINARY; + (*pSchema)[0].bytes = TSDB_TABLE_NAME_LEN; + strcpy((*pSchema)[0].name, "Table"); + + (*pSchema)[1].type = TSDB_DATA_TYPE_BINARY; + (*pSchema)[1].bytes = TSDB_MAX_BINARY_LEN; + strcpy((*pSchema)[1].name, "Create Table"); + + return TSDB_CODE_SUCCESS; +} + int32_t extractResultSchema(const SNode* pRoot, int32_t* numOfCols, SSchema** pSchema) { if (NULL == pRoot) { return TSDB_CODE_SUCCESS; @@ -4110,6 +4438,11 @@ int32_t extractResultSchema(const SNode* pRoot, int32_t* numOfCols, SSchema** pS return extractExplainResultSchema(numOfCols, pSchema); case QUERY_NODE_DESCRIBE_STMT: return extractDescribeResultSchema(numOfCols, pSchema); + case QUERY_NODE_SHOW_CREATE_DATABASE_STMT: + return extractShowCreateDatabaseResultSchema(numOfCols, pSchema); + case QUERY_NODE_SHOW_CREATE_TABLE_STMT: + case QUERY_NODE_SHOW_CREATE_STABLE_STMT: + return extractShowCreateTableResultSchema(numOfCols, pSchema); default: break; } @@ -4757,10 +5090,7 @@ static int32_t rewriteCreateMultiTable(STranslateContext* pCxt, SQuery* pQuery) SNode* pNode; FOREACH(pNode, pStmt->pSubTables) { SCreateSubTableClause* pClause = (SCreateSubTableClause*)pNode; - code = checkSchemalessDb(pCxt, pClause->dbName); - if (TSDB_CODE_SUCCESS == code) { - code = rewriteCreateSubTable(pCxt, pClause, pVgroupHashmap); - } + code = rewriteCreateSubTable(pCxt, pClause, pVgroupHashmap); if (TSDB_CODE_SUCCESS != code) { taosHashCleanup(pVgroupHashmap); return code; @@ -5333,10 +5663,14 @@ static int32_t setQuery(STranslateContext* pCxt, SQuery* pQuery) { pQuery->msgType = toMsgType(((SVnodeModifOpStmt*)pQuery->pRoot)->sqlNodeType); break; case QUERY_NODE_DESCRIBE_STMT: + case QUERY_NODE_SHOW_CREATE_DATABASE_STMT: + case QUERY_NODE_SHOW_CREATE_TABLE_STMT: + case QUERY_NODE_SHOW_CREATE_STABLE_STMT: pQuery->execMode = QUERY_EXEC_MODE_LOCAL; pQuery->haveResultSet = true; break; case QUERY_NODE_RESET_QUERY_CACHE_STMT: + case QUERY_NODE_ALTER_LOCAL_STMT: pQuery->execMode = QUERY_EXEC_MODE_LOCAL; break; default: diff --git a/source/libs/parser/src/parUtil.c b/source/libs/parser/src/parUtil.c index 42e887bb9d69c1a20bf877f8d230dfae668bdbbd..95cdf58d5a5bc3d6e02e0bf67154839562d4a6e5 100644 --- a/source/libs/parser/src/parUtil.c +++ b/source/libs/parser/src/parUtil.c @@ -90,7 +90,7 @@ static char* getSyntaxErrFormat(int32_t errCode) { case TSDB_CODE_PAR_GROUPBY_WINDOW_COEXIST: return "GROUP BY and WINDOW-clause can't be used together"; case TSDB_CODE_PAR_INVALID_OPTION_UNIT: - return "Invalid option %s unit: %c, only m, h, d allowed"; + return "Invalid option %s unit: %c, only %c, %c, %c allowed"; case TSDB_CODE_PAR_INVALID_KEEP_UNIT: return "Invalid option keep unit: %c, only m, h, d allowed"; case TSDB_CODE_PAR_AGG_FUNC_NESTING: @@ -185,9 +185,17 @@ static char* getSyntaxErrFormat(int32_t errCode) { case TSDB_CODE_PAR_INVALID_REDISTRIBUTE_VG: return "The REDISTRIBUTE VGROUP statement only support 1 to 3 dnodes"; case TSDB_CODE_PAR_FILL_NOT_ALLOWED_FUNC: - return "%s function not allowed in fill query"; + return "%s function does not supportted in fill query"; case TSDB_CODE_PAR_INVALID_WINDOW_PC: - return "_WSTARTTS, _WENDTS and _WDURATION can only be used in window queries"; + return "_WSTARTTS, _WENDTS and _WDURATION can only be used in window query"; + case TSDB_CODE_PAR_WINDOW_NOT_ALLOWED_FUNC: + return "%s function does not supportted in time window query"; + case TSDB_CODE_PAR_STREAM_NOT_ALLOWED_FUNC: + return "%s function does not supportted in stream query"; + case TSDB_CODE_PAR_GROUP_BY_NOT_ALLOWED_FUNC: + return "%s function does not supportted in group query"; + case TSDB_CODE_PAR_INVALID_TABLE_OPTION: + return "Invalid option %s"; case TSDB_CODE_OUT_OF_MEMORY: return "Out of memory"; default: diff --git a/source/libs/parser/src/sql.c b/source/libs/parser/src/sql.c index e81610c1b459a93540cc70f5145eb762718ee0be..a950b4efb0e4a3b41f297c62462c411868676d2a 100644 --- a/source/libs/parser/src/sql.c +++ b/source/libs/parser/src/sql.c @@ -104,25 +104,25 @@ #endif /************* Begin control #defines *****************************************/ #define YYCODETYPE unsigned short int -#define YYNOCODE 360 +#define YYNOCODE 359 #define YYACTIONTYPE unsigned short int #define ParseTOKENTYPE SToken typedef union { int yyinit; ParseTOKENTYPE yy0; - EFillMode yy54; - int32_t yy100; - bool yy137; - int64_t yy189; - SToken yy209; - ENullOrder yy217; - SDataType yy304; - EOperatorType yy380; - SNodeList* yy424; - EOrder yy578; - SAlterOption yy605; - EJoinType yy612; - SNode* yy632; + EOperatorType yy28; + int32_t yy42; + ENullOrder yy107; + EFillMode yy320; + SToken yy421; + SNodeList* yy530; + SAlterOption yy557; + EOrder yy610; + bool yy621; + EJoinType yy636; + int64_t yy669; + SNode* yy674; + SDataType yy690; } YYMINORTYPE; #ifndef YYSTACKDEPTH #define YYSTACKDEPTH 100 @@ -138,17 +138,17 @@ typedef union { #define ParseCTX_FETCH #define ParseCTX_STORE #define YYFALLBACK 1 -#define YYNSTATE 619 -#define YYNRULE 454 -#define YYNTOKEN 240 -#define YY_MAX_SHIFT 618 -#define YY_MIN_SHIFTREDUCE 906 -#define YY_MAX_SHIFTREDUCE 1359 -#define YY_ERROR_ACTION 1360 -#define YY_ACCEPT_ACTION 1361 -#define YY_NO_ACTION 1362 -#define YY_MIN_REDUCE 1363 -#define YY_MAX_REDUCE 1816 +#define YYNSTATE 622 +#define YYNRULE 459 +#define YYNTOKEN 239 +#define YY_MAX_SHIFT 621 +#define YY_MIN_SHIFTREDUCE 912 +#define YY_MAX_SHIFTREDUCE 1370 +#define YY_ERROR_ACTION 1371 +#define YY_ACCEPT_ACTION 1372 +#define YY_NO_ACTION 1373 +#define YY_MIN_REDUCE 1374 +#define YY_MAX_REDUCE 1832 /************* End control #defines *******************************************/ #define YY_NLOOKAHEAD ((int)(sizeof(yy_lookahead)/sizeof(yy_lookahead[0]))) @@ -215,607 +215,594 @@ typedef union { ** yy_default[] Default action for each state. ** *********** Begin parsing tables **********************************************/ -#define YY_ACTTAB_COUNT (2152) +#define YY_ACTTAB_COUNT (2098) static const YYACTIONTYPE yy_action[] = { - /* 0 */ 28, 231, 1663, 1650, 611, 610, 297, 1650, 358, 1485, - /* 10 */ 314, 1483, 35, 33, 352, 36, 34, 32, 31, 30, - /* 20 */ 306, 24, 1173, 1647, 533, 442, 441, 1647, 79, 1647, - /* 30 */ 1679, 36, 34, 32, 31, 30, 152, 493, 517, 1643, - /* 40 */ 1649, 115, 278, 1643, 1649, 1643, 1649, 1171, 516, 1486, - /* 50 */ 536, 533, 1633, 1494, 536, 1794, 536, 497, 14, 1746, - /* 60 */ 35, 33, 1300, 356, 1179, 1663, 114, 147, 306, 1692, - /* 70 */ 1173, 1791, 82, 1664, 519, 1666, 1667, 515, 532, 536, - /* 80 */ 1494, 1, 1732, 1743, 1794, 1314, 280, 1728, 36, 34, - /* 90 */ 32, 31, 30, 1679, 410, 1171, 1793, 1651, 1794, 520, - /* 100 */ 1791, 517, 309, 615, 112, 1583, 14, 1746, 35, 33, - /* 110 */ 149, 516, 1179, 1172, 1791, 1633, 306, 1647, 1173, 145, - /* 120 */ 1739, 1740, 532, 1744, 36, 34, 32, 31, 30, 2, - /* 130 */ 63, 1742, 1692, 1643, 1649, 84, 1664, 519, 1666, 1667, - /* 140 */ 515, 1364, 536, 1171, 536, 1732, 953, 1794, 952, 1731, - /* 150 */ 1728, 615, 1490, 72, 14, 392, 1174, 393, 1395, 148, - /* 160 */ 1179, 1172, 96, 1791, 532, 95, 94, 93, 92, 91, - /* 170 */ 90, 89, 88, 87, 1487, 954, 56, 2, 132, 201, - /* 180 */ 1375, 1177, 1178, 39, 1224, 1225, 1227, 1228, 1229, 1230, - /* 190 */ 1231, 512, 534, 1239, 1240, 1241, 1242, 1243, 1244, 615, - /* 200 */ 400, 952, 393, 1395, 1174, 55, 1197, 66, 133, 1172, - /* 210 */ 96, 150, 1451, 95, 94, 93, 92, 91, 90, 89, - /* 220 */ 88, 87, 65, 279, 1356, 38, 429, 194, 567, 1177, - /* 230 */ 1178, 1363, 1224, 1225, 1227, 1228, 1229, 1230, 1231, 512, - /* 240 */ 534, 1239, 1240, 1241, 1242, 1243, 1244, 566, 487, 565, - /* 250 */ 564, 563, 1174, 55, 63, 105, 104, 103, 102, 101, - /* 260 */ 100, 99, 98, 97, 936, 35, 33, 110, 36, 34, - /* 270 */ 32, 31, 30, 306, 1746, 1173, 1489, 1177, 1178, 1198, - /* 280 */ 1224, 1225, 1227, 1228, 1229, 1230, 1231, 512, 534, 1239, - /* 290 */ 1240, 1241, 1242, 1243, 1244, 55, 445, 444, 1741, 1538, - /* 300 */ 1171, 443, 940, 941, 111, 440, 296, 1355, 439, 438, - /* 310 */ 437, 1536, 569, 35, 33, 1245, 1679, 1179, 26, 310, - /* 320 */ 1386, 306, 391, 1173, 486, 395, 290, 130, 36, 34, - /* 330 */ 32, 31, 30, 1196, 8, 150, 1496, 1043, 559, 558, - /* 340 */ 557, 1047, 556, 1049, 1050, 555, 1052, 552, 1171, 1058, - /* 350 */ 549, 1060, 1061, 546, 543, 150, 615, 450, 482, 618, - /* 360 */ 485, 35, 33, 585, 583, 1179, 1172, 142, 397, 306, - /* 370 */ 1633, 1173, 458, 248, 1195, 291, 1324, 289, 288, 1532, - /* 380 */ 433, 318, 9, 150, 435, 107, 193, 399, 1574, 1576, - /* 390 */ 395, 607, 603, 599, 595, 247, 1171, 1472, 453, 157, - /* 400 */ 533, 129, 1361, 447, 615, 1425, 434, 572, 192, 1174, - /* 410 */ 1794, 1385, 357, 1179, 1172, 479, 1322, 1323, 1325, 1326, - /* 420 */ 80, 1571, 1792, 242, 61, 150, 1791, 60, 159, 1494, - /* 430 */ 9, 488, 483, 51, 1177, 1178, 50, 1224, 1225, 1227, - /* 440 */ 1228, 1229, 1230, 1231, 512, 534, 1239, 1240, 1241, 1242, - /* 450 */ 1243, 1244, 615, 55, 410, 316, 529, 1174, 54, 319, - /* 460 */ 322, 1633, 1172, 130, 150, 445, 444, 130, 1538, 562, - /* 470 */ 443, 382, 1496, 111, 440, 311, 1496, 439, 438, 437, - /* 480 */ 1536, 474, 1177, 1178, 203, 1224, 1225, 1227, 1228, 1229, - /* 490 */ 1230, 1231, 512, 534, 1239, 1240, 1241, 1242, 1243, 1244, - /* 500 */ 1384, 1794, 1147, 1199, 197, 1174, 1575, 1576, 36, 34, - /* 510 */ 32, 31, 30, 147, 1307, 161, 160, 1791, 35, 33, - /* 520 */ 1197, 1663, 1155, 1156, 195, 1470, 306, 351, 1173, 350, - /* 530 */ 1177, 1178, 493, 1224, 1225, 1227, 1228, 1229, 1230, 1231, - /* 540 */ 512, 534, 1239, 1240, 1241, 1242, 1243, 1244, 253, 1679, - /* 550 */ 1633, 1524, 533, 1171, 1383, 1621, 533, 517, 1382, 1250, - /* 560 */ 277, 114, 1195, 1005, 367, 1197, 1479, 516, 106, 375, - /* 570 */ 1179, 1633, 387, 466, 533, 431, 497, 1422, 1211, 469, - /* 580 */ 1007, 1494, 569, 150, 1381, 1494, 368, 2, 1692, 343, - /* 590 */ 388, 82, 1664, 519, 1666, 1667, 515, 1481, 536, 112, - /* 600 */ 331, 1732, 1477, 1494, 1633, 280, 1728, 198, 1633, 615, - /* 610 */ 345, 341, 533, 495, 144, 1739, 1740, 1794, 1744, 1172, - /* 620 */ 1794, 940, 941, 1538, 106, 1376, 11, 10, 222, 147, - /* 630 */ 317, 436, 147, 1791, 1633, 1536, 1791, 591, 590, 589, - /* 640 */ 321, 1494, 588, 587, 586, 116, 581, 580, 579, 578, - /* 650 */ 577, 576, 575, 574, 123, 570, 32, 31, 30, 1380, - /* 660 */ 386, 1299, 1174, 381, 380, 379, 378, 377, 374, 373, - /* 670 */ 372, 371, 370, 366, 365, 364, 363, 362, 361, 360, - /* 680 */ 359, 499, 1663, 520, 1379, 1226, 1197, 1177, 1178, 1584, - /* 690 */ 1224, 1225, 1227, 1228, 1229, 1230, 1231, 512, 534, 1239, - /* 700 */ 1240, 1241, 1242, 1243, 1244, 1264, 131, 1276, 1378, 1633, - /* 710 */ 1679, 259, 573, 533, 1466, 7, 511, 533, 496, 1471, - /* 720 */ 1200, 533, 459, 257, 53, 409, 1269, 52, 516, 1491, - /* 730 */ 1226, 533, 1633, 1610, 1633, 533, 36, 34, 32, 31, - /* 740 */ 30, 1377, 1494, 244, 162, 1374, 1494, 467, 1663, 1692, - /* 750 */ 1494, 501, 83, 1664, 519, 1666, 1667, 515, 1633, 536, - /* 760 */ 1494, 1373, 1732, 1794, 1494, 25, 299, 1728, 143, 55, - /* 770 */ 36, 34, 32, 31, 30, 147, 1679, 1469, 533, 1791, - /* 780 */ 223, 533, 281, 130, 517, 475, 1759, 435, 533, 1372, - /* 790 */ 530, 1633, 1497, 531, 516, 1633, 185, 1371, 1633, 183, - /* 800 */ 320, 476, 1370, 1369, 1663, 81, 1211, 1494, 1368, 434, - /* 810 */ 1494, 1633, 567, 504, 1262, 1692, 281, 1494, 274, 1664, - /* 820 */ 519, 1666, 1667, 515, 509, 536, 1751, 1295, 493, 561, - /* 830 */ 457, 566, 1679, 565, 564, 563, 59, 58, 355, 1633, - /* 840 */ 514, 156, 1452, 455, 1538, 207, 349, 1633, 1262, 584, - /* 850 */ 516, 1226, 1633, 1633, 1633, 1367, 1537, 114, 1633, 276, - /* 860 */ 1173, 226, 339, 1263, 337, 333, 329, 153, 324, 1298, - /* 870 */ 567, 1692, 1412, 1182, 273, 1664, 519, 1666, 1667, 515, - /* 880 */ 513, 536, 510, 1704, 1268, 1171, 1366, 1407, 1295, 566, - /* 890 */ 346, 565, 564, 563, 446, 112, 120, 1263, 187, 150, - /* 900 */ 189, 186, 1179, 188, 191, 1633, 1663, 190, 46, 448, - /* 910 */ 146, 1739, 1740, 1405, 1744, 11, 10, 1653, 1268, 1358, - /* 920 */ 1359, 480, 1181, 27, 304, 1257, 1258, 1259, 1260, 1261, - /* 930 */ 1265, 1266, 1267, 210, 1679, 451, 1633, 471, 502, 1185, - /* 940 */ 78, 615, 496, 37, 37, 460, 37, 1254, 233, 1321, - /* 950 */ 74, 1172, 516, 217, 1655, 1680, 1633, 27, 304, 1257, - /* 960 */ 1258, 1259, 1260, 1261, 1265, 1266, 1267, 118, 1663, 1396, - /* 970 */ 119, 1533, 120, 1692, 212, 46, 83, 1664, 519, 1666, - /* 980 */ 1667, 515, 977, 536, 1270, 1232, 1732, 1128, 1184, 235, - /* 990 */ 299, 1728, 143, 541, 1174, 428, 1679, 1762, 494, 978, - /* 1000 */ 1663, 225, 505, 119, 517, 228, 120, 230, 525, 3, - /* 1010 */ 1760, 241, 5, 1036, 516, 121, 252, 119, 1633, 1177, - /* 1020 */ 1178, 323, 1195, 326, 330, 286, 287, 1005, 1679, 249, - /* 1030 */ 1139, 369, 1573, 376, 1064, 1692, 517, 158, 83, 1664, - /* 1040 */ 519, 1666, 1667, 515, 1068, 536, 516, 1074, 1732, 384, - /* 1050 */ 1633, 389, 299, 1728, 1807, 383, 1072, 1201, 122, 385, - /* 1060 */ 1663, 390, 1204, 1766, 401, 398, 165, 1692, 402, 167, - /* 1070 */ 83, 1664, 519, 1666, 1667, 515, 1203, 536, 1205, 404, - /* 1080 */ 1732, 403, 170, 406, 299, 1728, 1807, 172, 1679, 407, - /* 1090 */ 1202, 175, 1663, 408, 62, 1789, 517, 411, 178, 430, - /* 1100 */ 432, 1484, 182, 86, 1480, 184, 516, 295, 124, 1179, - /* 1110 */ 1633, 125, 1482, 1478, 1615, 126, 127, 1614, 196, 461, - /* 1120 */ 1679, 199, 250, 202, 465, 462, 1200, 1692, 517, 468, - /* 1130 */ 83, 1664, 519, 1666, 1667, 515, 472, 536, 516, 205, - /* 1140 */ 1732, 481, 1633, 1663, 299, 1728, 1807, 497, 470, 478, - /* 1150 */ 473, 523, 1773, 1772, 298, 1750, 490, 208, 1763, 1692, - /* 1160 */ 211, 6, 264, 1664, 519, 1666, 1667, 515, 1753, 536, - /* 1170 */ 484, 1679, 216, 1663, 477, 113, 1295, 218, 1199, 517, - /* 1180 */ 40, 300, 1747, 506, 503, 18, 527, 1582, 1794, 516, - /* 1190 */ 137, 521, 522, 1633, 219, 1581, 526, 308, 497, 237, - /* 1200 */ 149, 1679, 528, 224, 1791, 251, 1663, 1495, 1713, 517, - /* 1210 */ 1692, 1810, 1790, 264, 1664, 519, 1666, 1667, 515, 516, - /* 1220 */ 536, 239, 71, 1633, 73, 539, 246, 254, 500, 614, - /* 1230 */ 136, 227, 1467, 229, 1679, 507, 1663, 265, 47, 1794, - /* 1240 */ 1692, 256, 517, 84, 1664, 519, 1666, 1667, 515, 258, - /* 1250 */ 536, 147, 516, 1732, 275, 1791, 1633, 508, 1728, 266, - /* 1260 */ 1627, 1626, 57, 1625, 1679, 325, 1622, 327, 328, 332, - /* 1270 */ 1166, 1167, 517, 1692, 154, 1620, 134, 1664, 519, 1666, - /* 1280 */ 1667, 515, 516, 536, 334, 335, 1633, 1663, 336, 1619, - /* 1290 */ 338, 1618, 340, 1617, 342, 1616, 1663, 344, 1600, 155, - /* 1300 */ 1142, 347, 1141, 1692, 348, 1594, 84, 1664, 519, 1666, - /* 1310 */ 1667, 515, 1593, 536, 353, 1679, 1732, 354, 1592, 498, - /* 1320 */ 1808, 1729, 1591, 517, 1679, 1111, 1566, 1565, 1564, 1563, - /* 1330 */ 1562, 1561, 517, 516, 1560, 1559, 1558, 1633, 1557, 1556, - /* 1340 */ 1555, 1554, 516, 1553, 1552, 1551, 1633, 1550, 1549, 117, - /* 1350 */ 1663, 1548, 1547, 1546, 1692, 1545, 1544, 269, 1664, 519, - /* 1360 */ 1666, 1667, 515, 1692, 536, 1543, 134, 1664, 519, 1666, - /* 1370 */ 1667, 515, 1113, 536, 1542, 1541, 1540, 1539, 1679, 1424, - /* 1380 */ 1392, 943, 163, 108, 942, 1391, 517, 1608, 1602, 1590, - /* 1390 */ 1663, 171, 1589, 394, 489, 140, 516, 164, 109, 169, - /* 1400 */ 1633, 396, 1579, 303, 45, 174, 1663, 1473, 1423, 1421, - /* 1410 */ 1809, 1419, 1417, 412, 413, 414, 417, 1692, 1679, 971, - /* 1420 */ 274, 1664, 519, 1666, 1667, 515, 514, 536, 416, 420, - /* 1430 */ 418, 421, 422, 1415, 1679, 426, 516, 424, 1404, 425, - /* 1440 */ 1633, 1403, 517, 1390, 1475, 1077, 181, 1078, 1474, 1413, - /* 1450 */ 1004, 582, 516, 1003, 1663, 584, 1633, 1692, 292, 305, - /* 1460 */ 273, 1664, 519, 1666, 1667, 515, 1408, 536, 1002, 1705, - /* 1470 */ 449, 1001, 998, 1692, 997, 996, 274, 1664, 519, 1666, - /* 1480 */ 1667, 515, 1679, 536, 293, 1406, 1663, 294, 1389, 452, - /* 1490 */ 517, 180, 1388, 454, 456, 85, 1607, 1149, 1601, 463, - /* 1500 */ 516, 1588, 1587, 141, 1633, 1586, 1578, 307, 49, 427, - /* 1510 */ 423, 419, 415, 179, 1679, 67, 1663, 204, 464, 4, - /* 1520 */ 37, 1692, 517, 206, 274, 1664, 519, 1666, 1667, 515, - /* 1530 */ 15, 536, 516, 128, 209, 43, 1633, 1320, 64, 200, - /* 1540 */ 135, 177, 213, 215, 1679, 22, 48, 1653, 214, 1313, - /* 1550 */ 68, 23, 517, 1692, 42, 1292, 260, 1664, 519, 1666, - /* 1560 */ 1667, 515, 516, 536, 221, 1291, 1633, 1663, 138, 1349, - /* 1570 */ 17, 1338, 1344, 1343, 301, 1348, 10, 1347, 302, 19, - /* 1580 */ 1255, 139, 1234, 1692, 151, 29, 268, 1664, 519, 1666, - /* 1590 */ 1667, 515, 12, 536, 1233, 1679, 20, 1219, 176, 16, - /* 1600 */ 168, 518, 173, 517, 405, 41, 13, 1663, 1577, 238, - /* 1610 */ 74, 524, 1652, 516, 232, 21, 234, 1633, 1318, 1189, - /* 1620 */ 236, 240, 166, 69, 70, 243, 1695, 540, 1236, 535, - /* 1630 */ 44, 1071, 1065, 538, 1692, 1679, 315, 270, 1664, 519, - /* 1640 */ 1666, 1667, 515, 517, 536, 542, 544, 1663, 1062, 1059, - /* 1650 */ 545, 547, 548, 516, 550, 1053, 553, 1633, 551, 1057, - /* 1660 */ 1051, 1056, 1663, 554, 1055, 1054, 75, 1042, 76, 560, - /* 1670 */ 1073, 77, 1070, 568, 1692, 1679, 969, 261, 1664, 519, - /* 1680 */ 1666, 1667, 515, 517, 536, 993, 1011, 245, 991, 571, - /* 1690 */ 1679, 986, 1008, 516, 990, 989, 988, 1633, 517, 987, - /* 1700 */ 985, 984, 1006, 981, 980, 979, 976, 975, 516, 974, - /* 1710 */ 1420, 592, 1633, 1663, 1692, 593, 594, 271, 1664, 519, - /* 1720 */ 1666, 1667, 515, 1418, 536, 1663, 597, 596, 598, 1692, - /* 1730 */ 1416, 600, 262, 1664, 519, 1666, 1667, 515, 601, 536, - /* 1740 */ 602, 1679, 604, 605, 606, 1402, 608, 609, 1401, 517, - /* 1750 */ 1414, 1387, 613, 1679, 612, 1175, 255, 616, 617, 516, - /* 1760 */ 1362, 517, 1362, 1633, 1362, 1663, 1362, 1362, 1362, 1362, - /* 1770 */ 1362, 516, 1362, 1362, 1362, 1633, 1663, 1362, 1362, 1362, - /* 1780 */ 1692, 1362, 1362, 272, 1664, 519, 1666, 1667, 515, 1362, - /* 1790 */ 536, 1362, 1692, 1679, 1362, 263, 1664, 519, 1666, 1667, - /* 1800 */ 515, 517, 536, 1362, 1679, 1362, 1362, 1362, 1362, 1362, - /* 1810 */ 1362, 516, 517, 1362, 1362, 1633, 1362, 1362, 1362, 1362, - /* 1820 */ 1362, 1362, 516, 1362, 1362, 1362, 1633, 1663, 1362, 1362, - /* 1830 */ 1362, 1362, 1692, 1362, 1362, 1675, 1664, 519, 1666, 1667, - /* 1840 */ 515, 1362, 536, 1692, 1362, 1362, 1674, 1664, 519, 1666, - /* 1850 */ 1667, 515, 1362, 536, 1362, 1679, 1362, 1663, 1362, 1362, - /* 1860 */ 1362, 1362, 1362, 517, 1362, 1362, 1362, 1362, 1362, 1362, - /* 1870 */ 1362, 1362, 1362, 516, 1362, 1362, 1362, 1633, 1362, 1362, - /* 1880 */ 1362, 1362, 1362, 1362, 1362, 1679, 1362, 1663, 1362, 1362, - /* 1890 */ 1362, 1362, 1362, 517, 1692, 1362, 1362, 1673, 1664, 519, - /* 1900 */ 1666, 1667, 515, 516, 536, 1362, 1362, 1633, 1362, 1362, - /* 1910 */ 1362, 1362, 1362, 1362, 1362, 1679, 1362, 1362, 1362, 1663, - /* 1920 */ 1362, 1362, 1362, 517, 1692, 1362, 1362, 284, 1664, 519, - /* 1930 */ 1666, 1667, 515, 516, 536, 1362, 1362, 1633, 1362, 1362, - /* 1940 */ 1362, 1362, 1362, 313, 312, 1362, 1362, 1679, 1362, 1362, - /* 1950 */ 1362, 1362, 1663, 1187, 1692, 517, 1362, 283, 1664, 519, - /* 1960 */ 1666, 1667, 515, 1362, 536, 516, 1362, 1362, 1362, 1633, - /* 1970 */ 1362, 1362, 1362, 1362, 1362, 1362, 1362, 1362, 1180, 1362, - /* 1980 */ 1679, 1362, 1362, 1362, 1362, 1663, 1692, 1362, 517, 285, - /* 1990 */ 1664, 519, 1666, 1667, 515, 1179, 536, 1362, 516, 1362, - /* 2000 */ 1362, 1362, 1633, 1362, 1362, 1362, 1362, 1362, 1362, 1362, - /* 2010 */ 1362, 1362, 1362, 1679, 1362, 1362, 493, 1362, 1362, 1692, - /* 2020 */ 1362, 517, 282, 1664, 519, 1666, 1667, 515, 1362, 536, - /* 2030 */ 1362, 516, 1362, 1362, 537, 1633, 1362, 1362, 1362, 1362, - /* 2040 */ 1362, 1362, 1362, 1362, 1183, 114, 493, 1362, 1362, 1362, - /* 2050 */ 1362, 1362, 1692, 1362, 1362, 267, 1664, 519, 1666, 1667, - /* 2060 */ 515, 1362, 536, 1362, 497, 1362, 1362, 1362, 1362, 1362, - /* 2070 */ 1362, 1362, 1362, 1362, 1362, 114, 1362, 1362, 1362, 1362, - /* 2080 */ 1362, 1362, 1362, 112, 1362, 1362, 1362, 1188, 1362, 1362, - /* 2090 */ 1362, 1362, 1362, 1362, 497, 1362, 1362, 1362, 220, 1739, - /* 2100 */ 492, 1362, 491, 1362, 1362, 1794, 1362, 1362, 1362, 1362, - /* 2110 */ 1362, 1362, 1191, 112, 1362, 1362, 1362, 149, 1362, 1362, - /* 2120 */ 1362, 1791, 1362, 534, 1239, 1240, 1362, 1362, 220, 1739, - /* 2130 */ 492, 1362, 491, 1362, 1362, 1794, 1362, 1362, 1362, 1362, - /* 2140 */ 1362, 1362, 1362, 1362, 1362, 1362, 1362, 147, 1362, 1362, - /* 2150 */ 1362, 1791, + /* 0 */ 1557, 321, 1679, 1666, 1550, 1552, 300, 396, 1666, 397, + /* 10 */ 1406, 317, 35, 33, 1663, 404, 1556, 397, 1406, 1663, + /* 20 */ 309, 1483, 1184, 1557, 36, 34, 32, 31, 30, 1695, + /* 30 */ 299, 536, 36, 34, 32, 31, 30, 521, 1810, 1555, + /* 40 */ 1659, 1665, 1649, 1810, 520, 1659, 1665, 1182, 28, 233, + /* 50 */ 1809, 540, 362, 1372, 1807, 150, 540, 501, 14, 1807, + /* 60 */ 35, 33, 1311, 395, 1190, 1679, 399, 536, 309, 1708, + /* 70 */ 1184, 159, 82, 1680, 523, 1682, 1683, 519, 414, 540, + /* 80 */ 57, 1, 1748, 536, 401, 1810, 283, 1744, 497, 281, + /* 90 */ 1206, 537, 1695, 537, 1325, 1182, 62, 1808, 1810, 61, + /* 100 */ 521, 1807, 106, 618, 154, 1649, 14, 520, 942, 435, + /* 110 */ 151, 326, 1190, 24, 1807, 414, 114, 1253, 1254, 1505, + /* 120 */ 501, 1505, 356, 36, 34, 32, 31, 30, 39, 2, + /* 130 */ 134, 497, 1708, 1695, 1462, 82, 1680, 523, 1682, 1683, + /* 140 */ 519, 490, 540, 524, 38, 1748, 946, 947, 1318, 283, + /* 150 */ 1744, 618, 1810, 1600, 1208, 112, 196, 56, 1185, 114, + /* 160 */ 1183, 1810, 1208, 1810, 149, 1253, 1254, 130, 1807, 499, + /* 170 */ 146, 1755, 1756, 149, 1760, 149, 574, 1807, 489, 1807, + /* 180 */ 508, 486, 1188, 1189, 347, 1235, 1236, 1238, 1239, 1240, + /* 190 */ 1241, 1242, 516, 538, 1250, 1251, 1252, 1255, 112, 36, + /* 200 */ 34, 32, 31, 30, 349, 345, 1185, 133, 1183, 1386, + /* 210 */ 152, 66, 282, 147, 1755, 1756, 1209, 1760, 524, 35, + /* 220 */ 33, 36, 34, 32, 31, 30, 312, 309, 1599, 1184, + /* 230 */ 1188, 1189, 1375, 1235, 1236, 1238, 1239, 1240, 1241, 1242, + /* 240 */ 516, 538, 1250, 1251, 1252, 1255, 36, 34, 32, 31, + /* 250 */ 30, 492, 487, 96, 1182, 73, 95, 94, 93, 92, + /* 260 */ 91, 90, 89, 88, 87, 14, 491, 35, 33, 1222, + /* 270 */ 26, 1190, 1433, 152, 1496, 309, 1498, 1184, 1193, 56, + /* 280 */ 36, 34, 32, 31, 30, 1663, 152, 403, 2, 1335, + /* 290 */ 399, 1049, 563, 562, 561, 1053, 560, 1055, 1056, 559, + /* 300 */ 1058, 556, 1182, 1064, 553, 1066, 1067, 550, 547, 1261, + /* 310 */ 618, 1659, 1665, 1091, 1092, 1208, 32, 31, 30, 1190, + /* 320 */ 1237, 589, 540, 1397, 1253, 1254, 483, 1333, 1334, 1336, + /* 330 */ 1337, 1209, 596, 595, 594, 324, 8, 593, 592, 591, + /* 340 */ 116, 586, 585, 584, 583, 582, 581, 580, 579, 124, + /* 350 */ 575, 1436, 578, 313, 1477, 355, 56, 354, 618, 1367, + /* 360 */ 1207, 131, 350, 1649, 572, 1185, 1396, 1183, 509, 1395, + /* 370 */ 1507, 96, 1253, 1254, 95, 94, 93, 92, 91, 90, + /* 380 */ 89, 88, 87, 122, 121, 569, 568, 567, 64, 1188, + /* 390 */ 1189, 1196, 1235, 1236, 1238, 1239, 1240, 1241, 1242, 516, + /* 400 */ 538, 1250, 1251, 1252, 1255, 497, 1649, 537, 152, 1649, + /* 410 */ 1501, 449, 448, 1185, 1557, 1183, 447, 1394, 360, 111, + /* 420 */ 444, 314, 1393, 443, 442, 441, 35, 33, 1256, 570, + /* 430 */ 1555, 1679, 1548, 114, 309, 1505, 1184, 1188, 1189, 1366, + /* 440 */ 1235, 1236, 1238, 1239, 1240, 1241, 1242, 516, 538, 1250, + /* 450 */ 1251, 1252, 1255, 1392, 1184, 449, 448, 1649, 1695, 1391, + /* 460 */ 447, 1182, 1649, 111, 444, 152, 521, 443, 442, 441, + /* 470 */ 197, 1649, 112, 520, 35, 33, 1551, 1552, 1190, 1182, + /* 480 */ 1190, 1237, 309, 1679, 1184, 152, 537, 148, 1755, 1756, + /* 490 */ 293, 1760, 959, 1649, 958, 9, 1190, 361, 1708, 1649, + /* 500 */ 64, 84, 1680, 523, 1682, 1683, 519, 1390, 540, 1182, + /* 510 */ 1695, 1748, 203, 110, 1505, 1747, 1744, 618, 500, 433, + /* 520 */ 473, 960, 1500, 1649, 571, 520, 1190, 1548, 1166, 1167, + /* 530 */ 187, 1253, 1254, 185, 537, 618, 316, 315, 56, 294, + /* 540 */ 67, 292, 291, 9, 437, 106, 1198, 1649, 439, 1482, + /* 550 */ 1708, 1310, 440, 83, 1680, 523, 1682, 1683, 519, 1011, + /* 560 */ 540, 1810, 1505, 1748, 959, 618, 958, 302, 1744, 145, + /* 570 */ 438, 1191, 1185, 149, 1183, 463, 1013, 1807, 1557, 1253, + /* 580 */ 1254, 225, 446, 445, 537, 320, 479, 1775, 1190, 386, + /* 590 */ 1185, 7, 1183, 960, 1555, 371, 1188, 1189, 1208, 1235, + /* 600 */ 1236, 1238, 1239, 1240, 1241, 1242, 516, 538, 1250, 1251, + /* 610 */ 1252, 1255, 1505, 319, 1188, 1189, 1810, 1275, 946, 947, + /* 620 */ 1185, 131, 1183, 322, 144, 79, 1389, 541, 149, 1388, + /* 630 */ 1507, 131, 1807, 35, 33, 163, 162, 1544, 115, 1280, + /* 640 */ 1507, 309, 572, 1184, 1188, 1189, 1497, 1235, 1236, 1238, + /* 650 */ 1239, 1240, 1241, 1242, 516, 538, 1250, 1251, 1252, 1255, + /* 660 */ 1385, 122, 121, 569, 568, 567, 1649, 152, 1182, 1649, + /* 670 */ 537, 1384, 11, 10, 25, 1667, 1383, 537, 280, 54, + /* 680 */ 1206, 372, 1199, 1762, 1194, 1190, 1663, 379, 413, 1382, + /* 690 */ 391, 36, 34, 32, 31, 30, 590, 588, 1505, 537, + /* 700 */ 1649, 1679, 2, 1210, 132, 1505, 1202, 1759, 392, 262, + /* 710 */ 1502, 1649, 1659, 1665, 1192, 1381, 1649, 538, 1250, 1251, + /* 720 */ 1374, 260, 53, 540, 618, 52, 1380, 1505, 1695, 1649, + /* 730 */ 1379, 256, 1762, 1762, 1535, 505, 521, 284, 1253, 1254, + /* 740 */ 1480, 1649, 164, 520, 105, 104, 103, 102, 101, 100, + /* 750 */ 99, 98, 97, 1287, 480, 1649, 1758, 1757, 1309, 1767, + /* 760 */ 1306, 1222, 1378, 1306, 1237, 566, 1649, 56, 1708, 1273, + /* 770 */ 1649, 278, 1680, 523, 1682, 1683, 519, 577, 540, 1185, + /* 780 */ 390, 1183, 1481, 385, 384, 383, 382, 381, 378, 377, + /* 790 */ 376, 375, 374, 370, 369, 368, 367, 366, 365, 364, + /* 800 */ 363, 1211, 1649, 1188, 1189, 81, 1235, 1236, 1238, 1239, + /* 810 */ 1240, 1241, 1242, 516, 538, 1250, 1251, 1252, 1255, 131, + /* 820 */ 1274, 537, 36, 34, 32, 31, 30, 1195, 1508, 537, + /* 830 */ 224, 1637, 1626, 572, 537, 1377, 60, 59, 359, 574, + /* 840 */ 471, 158, 1279, 189, 513, 534, 188, 353, 191, 1505, + /* 850 */ 439, 190, 122, 121, 569, 568, 567, 1505, 279, 537, + /* 860 */ 537, 343, 1505, 341, 337, 333, 155, 328, 284, 1494, + /* 870 */ 535, 246, 438, 1423, 1490, 1649, 335, 27, 307, 1268, + /* 880 */ 1269, 1270, 1271, 1272, 1276, 1277, 1278, 1505, 1505, 1418, + /* 890 */ 1679, 1590, 537, 1416, 193, 450, 152, 192, 1669, 461, + /* 900 */ 1273, 503, 161, 323, 120, 621, 46, 983, 212, 1369, + /* 910 */ 1370, 452, 459, 11, 10, 455, 78, 1695, 37, 251, + /* 920 */ 1505, 506, 1492, 1488, 984, 500, 75, 200, 209, 515, + /* 930 */ 1649, 142, 520, 565, 182, 1671, 37, 612, 608, 604, + /* 940 */ 600, 250, 470, 1387, 37, 475, 137, 1332, 235, 214, + /* 950 */ 118, 1274, 431, 427, 423, 419, 181, 1708, 228, 1281, + /* 960 */ 83, 1680, 523, 1682, 1683, 519, 80, 540, 1265, 244, + /* 970 */ 1748, 484, 1463, 1279, 302, 1744, 145, 1243, 119, 120, + /* 980 */ 46, 65, 545, 119, 179, 1139, 120, 107, 1412, 237, + /* 990 */ 464, 529, 119, 432, 1776, 1696, 219, 325, 1407, 1545, + /* 1000 */ 1778, 498, 533, 3, 1679, 5, 227, 230, 27, 307, + /* 1010 */ 1268, 1269, 1270, 1271, 1272, 1276, 1277, 1278, 327, 243, + /* 1020 */ 1042, 255, 1206, 1070, 1074, 232, 330, 1081, 1079, 478, + /* 1030 */ 1679, 1695, 205, 123, 614, 334, 289, 1150, 1011, 521, + /* 1040 */ 290, 252, 380, 178, 1649, 170, 520, 175, 373, 409, + /* 1050 */ 1592, 1158, 160, 199, 388, 387, 1679, 1695, 389, 1212, + /* 1060 */ 393, 394, 402, 1215, 405, 521, 167, 406, 168, 169, + /* 1070 */ 1649, 1708, 520, 1214, 83, 1680, 523, 1682, 1683, 519, + /* 1080 */ 407, 540, 1216, 1695, 1748, 408, 172, 410, 302, 1744, + /* 1090 */ 1823, 521, 174, 1213, 412, 177, 1649, 1708, 520, 1782, + /* 1100 */ 83, 1680, 523, 1682, 1683, 519, 411, 540, 1679, 63, + /* 1110 */ 1748, 415, 180, 434, 302, 1744, 1823, 436, 1495, 184, + /* 1120 */ 1491, 186, 1679, 1708, 125, 1805, 83, 1680, 523, 1682, + /* 1130 */ 1683, 519, 298, 540, 86, 1695, 1748, 126, 198, 1493, + /* 1140 */ 302, 1744, 1823, 521, 1489, 127, 1631, 253, 1649, 1695, + /* 1150 */ 520, 1766, 128, 1630, 472, 465, 469, 521, 1211, 466, + /* 1160 */ 527, 201, 1649, 501, 520, 204, 207, 476, 485, 477, + /* 1170 */ 474, 1789, 1779, 1679, 482, 1708, 301, 501, 269, 1680, + /* 1180 */ 523, 1682, 1683, 519, 210, 540, 488, 1788, 6, 1708, + /* 1190 */ 454, 213, 269, 1680, 523, 1682, 1683, 519, 494, 540, + /* 1200 */ 1695, 1306, 1769, 220, 1810, 462, 481, 113, 521, 1210, + /* 1210 */ 1763, 40, 510, 1649, 303, 520, 151, 18, 1810, 195, + /* 1220 */ 1807, 1598, 507, 530, 531, 218, 525, 1597, 221, 526, + /* 1230 */ 149, 457, 239, 241, 1807, 311, 451, 1679, 139, 532, + /* 1240 */ 1708, 194, 1826, 84, 1680, 523, 1682, 1683, 519, 1729, + /* 1250 */ 540, 1679, 254, 1748, 72, 1806, 1506, 512, 1744, 504, + /* 1260 */ 1679, 511, 74, 226, 1695, 1478, 51, 543, 229, 50, + /* 1270 */ 231, 257, 518, 617, 249, 47, 138, 1649, 1695, 520, + /* 1280 */ 1549, 261, 263, 270, 264, 259, 521, 1695, 1643, 1642, + /* 1290 */ 58, 1649, 1641, 520, 329, 521, 1638, 332, 331, 1177, + /* 1300 */ 1649, 1178, 520, 156, 1708, 336, 1636, 277, 1680, 523, + /* 1310 */ 1682, 1683, 519, 517, 540, 514, 1720, 1679, 1708, 338, + /* 1320 */ 339, 135, 1680, 523, 1682, 1683, 519, 1708, 540, 340, + /* 1330 */ 84, 1680, 523, 1682, 1683, 519, 1635, 540, 342, 1679, + /* 1340 */ 1748, 1634, 344, 1633, 1695, 1745, 346, 1632, 348, 1616, + /* 1350 */ 157, 351, 521, 352, 1153, 1152, 1610, 1649, 1609, 520, + /* 1360 */ 357, 358, 1608, 1607, 502, 1824, 1695, 1125, 1585, 1584, + /* 1370 */ 1583, 1582, 1581, 1580, 521, 1579, 1578, 1577, 1576, 1649, + /* 1380 */ 1575, 520, 1574, 1573, 1708, 1572, 1571, 273, 1680, 523, + /* 1390 */ 1682, 1683, 519, 1679, 540, 1570, 1569, 1568, 1567, 117, + /* 1400 */ 1566, 1565, 1564, 1563, 1562, 1679, 1708, 1561, 1560, 135, + /* 1410 */ 1680, 523, 1682, 1683, 519, 1127, 540, 1559, 1558, 1435, + /* 1420 */ 1695, 1403, 143, 949, 493, 108, 948, 165, 521, 1402, + /* 1430 */ 1624, 1618, 1695, 1649, 398, 520, 400, 166, 1606, 173, + /* 1440 */ 518, 1605, 109, 1595, 1484, 1649, 306, 520, 171, 1434, + /* 1450 */ 176, 1432, 418, 1825, 1430, 1428, 1679, 422, 1426, 426, + /* 1460 */ 1708, 416, 430, 278, 1680, 523, 1682, 1683, 519, 977, + /* 1470 */ 540, 417, 1708, 421, 420, 277, 1680, 523, 1682, 1683, + /* 1480 */ 519, 424, 540, 1695, 1721, 425, 428, 429, 1415, 1414, + /* 1490 */ 1401, 521, 1486, 1085, 1084, 1485, 1649, 1010, 520, 1009, + /* 1500 */ 1008, 587, 589, 1007, 1004, 1424, 45, 1679, 295, 308, + /* 1510 */ 1003, 183, 1002, 1419, 453, 296, 1417, 456, 297, 1400, + /* 1520 */ 458, 1679, 1399, 1708, 460, 1623, 278, 1680, 523, 1682, + /* 1530 */ 1683, 519, 85, 540, 1695, 1617, 49, 467, 1160, 1604, + /* 1540 */ 1603, 1602, 521, 129, 1594, 68, 206, 1649, 1695, 520, + /* 1550 */ 211, 4, 37, 15, 136, 43, 521, 1331, 215, 22, + /* 1560 */ 310, 1649, 1324, 520, 216, 217, 69, 23, 1669, 16, + /* 1570 */ 1679, 468, 223, 41, 1708, 208, 202, 278, 1680, 523, + /* 1580 */ 1682, 1683, 519, 48, 540, 1303, 1302, 1679, 1708, 42, + /* 1590 */ 140, 265, 1680, 523, 1682, 1683, 519, 1695, 540, 1360, + /* 1600 */ 17, 1355, 1349, 1354, 10, 521, 19, 304, 1359, 1358, + /* 1610 */ 1649, 305, 520, 1245, 1695, 29, 141, 153, 1244, 1266, + /* 1620 */ 234, 1230, 521, 12, 20, 1593, 21, 1649, 236, 520, + /* 1630 */ 240, 1329, 522, 238, 70, 1668, 1679, 1708, 245, 528, + /* 1640 */ 272, 1680, 523, 1682, 1683, 519, 75, 540, 1679, 13, + /* 1650 */ 71, 1711, 1200, 1247, 1708, 539, 44, 274, 1680, 523, + /* 1660 */ 1682, 1683, 519, 1695, 540, 1071, 544, 318, 542, 242, + /* 1670 */ 546, 521, 1068, 548, 1065, 1695, 1649, 549, 520, 551, + /* 1680 */ 554, 552, 555, 521, 557, 1059, 1057, 558, 1649, 1679, + /* 1690 */ 520, 1063, 1062, 1048, 76, 77, 1080, 55, 564, 247, + /* 1700 */ 1061, 1076, 1060, 1708, 975, 573, 266, 1680, 523, 1682, + /* 1710 */ 1683, 519, 999, 540, 1017, 1708, 1695, 576, 275, 1680, + /* 1720 */ 523, 1682, 1683, 519, 521, 540, 248, 997, 996, 1649, + /* 1730 */ 995, 520, 992, 994, 993, 991, 990, 1014, 1012, 987, + /* 1740 */ 986, 1679, 985, 982, 981, 980, 1431, 597, 598, 599, + /* 1750 */ 1679, 1429, 601, 602, 1427, 603, 1708, 1425, 605, 267, + /* 1760 */ 1680, 523, 1682, 1683, 519, 606, 540, 607, 1695, 609, + /* 1770 */ 610, 611, 1413, 613, 1398, 615, 521, 1695, 1186, 616, + /* 1780 */ 619, 1649, 258, 520, 620, 521, 1373, 1373, 1373, 1373, + /* 1790 */ 1649, 1373, 520, 1373, 1373, 1373, 1373, 1373, 1373, 1679, + /* 1800 */ 1373, 1373, 1373, 1373, 1373, 1373, 1373, 1373, 1708, 1373, + /* 1810 */ 1373, 276, 1680, 523, 1682, 1683, 519, 1708, 540, 1373, + /* 1820 */ 268, 1680, 523, 1682, 1683, 519, 1695, 540, 1373, 1373, + /* 1830 */ 1373, 1373, 1373, 1373, 521, 1373, 1373, 497, 1373, 1649, + /* 1840 */ 1679, 520, 1373, 1373, 1373, 1373, 1373, 1373, 1373, 1373, + /* 1850 */ 1373, 1373, 1373, 1373, 1373, 1373, 1373, 1679, 1373, 1373, + /* 1860 */ 1373, 1373, 1373, 1373, 1373, 114, 1708, 1695, 1373, 1691, + /* 1870 */ 1680, 523, 1682, 1683, 519, 521, 540, 1373, 1373, 1373, + /* 1880 */ 1649, 1373, 520, 1373, 1695, 501, 1373, 1373, 1373, 1373, + /* 1890 */ 1373, 1373, 521, 1373, 1373, 1373, 1373, 1649, 1373, 520, + /* 1900 */ 1373, 1373, 1373, 1373, 112, 1373, 1679, 1708, 1373, 1373, + /* 1910 */ 1690, 1680, 523, 1682, 1683, 519, 1373, 540, 1679, 222, + /* 1920 */ 1755, 496, 1373, 495, 1708, 1373, 1810, 1689, 1680, 523, + /* 1930 */ 1682, 1683, 519, 1695, 540, 1373, 1373, 1373, 151, 497, + /* 1940 */ 1373, 521, 1807, 1373, 1373, 1695, 1649, 1373, 520, 1373, + /* 1950 */ 1373, 1373, 1373, 521, 1373, 1373, 1373, 1373, 1649, 1679, + /* 1960 */ 520, 1373, 1373, 1373, 1373, 1373, 1373, 114, 1373, 1373, + /* 1970 */ 1373, 1373, 1373, 1708, 1373, 1373, 287, 1680, 523, 1682, + /* 1980 */ 1683, 519, 1373, 540, 1373, 1708, 1695, 501, 286, 1680, + /* 1990 */ 523, 1682, 1683, 519, 521, 540, 1373, 1373, 1373, 1649, + /* 2000 */ 1373, 520, 1373, 1373, 1373, 1373, 112, 1373, 1373, 1373, + /* 2010 */ 1373, 1679, 1373, 1373, 1373, 1373, 1373, 1373, 1373, 1373, + /* 2020 */ 1679, 222, 1755, 496, 1373, 495, 1708, 1373, 1810, 288, + /* 2030 */ 1680, 523, 1682, 1683, 519, 1373, 540, 1373, 1695, 1373, + /* 2040 */ 149, 1373, 1373, 1373, 1807, 1373, 521, 1695, 1373, 1373, + /* 2050 */ 1373, 1649, 1373, 520, 1373, 521, 1373, 1373, 1373, 1373, + /* 2060 */ 1649, 1373, 520, 1373, 1373, 1373, 1373, 1373, 1373, 1373, + /* 2070 */ 1373, 1373, 1373, 1373, 1373, 1373, 1373, 1373, 1708, 1373, + /* 2080 */ 1373, 285, 1680, 523, 1682, 1683, 519, 1708, 540, 1373, + /* 2090 */ 271, 1680, 523, 1682, 1683, 519, 1373, 540, }; static const YYCODETYPE yy_lookahead[] = { - /* 0 */ 324, 325, 243, 273, 251, 252, 276, 273, 250, 273, - /* 10 */ 276, 272, 12, 13, 298, 12, 13, 14, 15, 16, - /* 20 */ 20, 2, 22, 293, 250, 257, 258, 293, 253, 293, - /* 30 */ 271, 12, 13, 14, 15, 16, 262, 250, 279, 309, - /* 40 */ 310, 266, 284, 309, 310, 309, 310, 47, 289, 274, - /* 50 */ 320, 250, 293, 279, 320, 339, 320, 298, 58, 311, - /* 60 */ 12, 13, 14, 262, 64, 243, 279, 351, 20, 310, - /* 70 */ 22, 355, 313, 314, 315, 316, 317, 318, 20, 320, - /* 80 */ 279, 81, 323, 335, 339, 82, 327, 328, 12, 13, - /* 90 */ 14, 15, 16, 271, 57, 47, 351, 273, 339, 289, - /* 100 */ 355, 279, 292, 103, 317, 295, 58, 311, 12, 13, - /* 110 */ 351, 289, 64, 113, 355, 293, 20, 293, 22, 332, - /* 120 */ 333, 334, 20, 336, 12, 13, 14, 15, 16, 81, - /* 130 */ 255, 335, 310, 309, 310, 313, 314, 315, 316, 317, - /* 140 */ 318, 0, 320, 47, 320, 323, 20, 339, 22, 327, - /* 150 */ 328, 103, 277, 253, 58, 246, 156, 248, 249, 351, - /* 160 */ 64, 113, 21, 355, 20, 24, 25, 26, 27, 28, - /* 170 */ 29, 30, 31, 32, 274, 49, 4, 81, 242, 55, - /* 180 */ 244, 181, 182, 81, 184, 185, 186, 187, 188, 189, - /* 190 */ 190, 191, 192, 193, 194, 195, 196, 197, 198, 103, - /* 200 */ 246, 22, 248, 249, 156, 81, 20, 83, 256, 113, - /* 210 */ 21, 211, 260, 24, 25, 26, 27, 28, 29, 30, - /* 220 */ 31, 32, 165, 166, 148, 81, 47, 170, 93, 181, - /* 230 */ 182, 0, 184, 185, 186, 187, 188, 189, 190, 191, - /* 240 */ 192, 193, 194, 195, 196, 197, 198, 112, 20, 114, - /* 250 */ 115, 116, 156, 81, 255, 24, 25, 26, 27, 28, - /* 260 */ 29, 30, 31, 32, 4, 12, 13, 268, 12, 13, - /* 270 */ 14, 15, 16, 20, 311, 22, 277, 181, 182, 20, - /* 280 */ 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, - /* 290 */ 194, 195, 196, 197, 198, 81, 60, 61, 335, 271, - /* 300 */ 47, 65, 42, 43, 68, 69, 278, 231, 72, 73, - /* 310 */ 74, 283, 57, 12, 13, 14, 271, 64, 2, 263, - /* 320 */ 243, 20, 247, 22, 279, 250, 35, 271, 12, 13, - /* 330 */ 14, 15, 16, 20, 81, 211, 280, 94, 95, 96, - /* 340 */ 97, 98, 99, 100, 101, 102, 103, 104, 47, 106, - /* 350 */ 107, 108, 109, 110, 111, 211, 103, 4, 143, 19, - /* 360 */ 315, 12, 13, 257, 258, 64, 113, 270, 14, 20, - /* 370 */ 293, 22, 19, 33, 20, 84, 181, 86, 87, 282, - /* 380 */ 89, 281, 81, 211, 93, 45, 33, 247, 288, 289, - /* 390 */ 250, 51, 52, 53, 54, 55, 47, 0, 45, 55, - /* 400 */ 250, 145, 240, 50, 103, 0, 115, 64, 55, 156, - /* 410 */ 339, 243, 262, 64, 113, 220, 221, 222, 223, 224, - /* 420 */ 80, 279, 351, 83, 80, 211, 355, 83, 286, 279, - /* 430 */ 81, 216, 217, 80, 181, 182, 83, 184, 185, 186, - /* 440 */ 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, - /* 450 */ 197, 198, 103, 81, 57, 263, 116, 156, 3, 263, - /* 460 */ 298, 293, 113, 271, 211, 60, 61, 271, 271, 92, - /* 470 */ 65, 75, 280, 68, 69, 278, 280, 72, 73, 74, - /* 480 */ 283, 141, 181, 182, 144, 184, 185, 186, 187, 188, - /* 490 */ 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, - /* 500 */ 243, 339, 162, 20, 164, 156, 288, 289, 12, 13, - /* 510 */ 14, 15, 16, 351, 14, 119, 120, 355, 12, 13, - /* 520 */ 20, 243, 167, 168, 169, 0, 20, 155, 22, 157, - /* 530 */ 181, 182, 250, 184, 185, 186, 187, 188, 189, 190, - /* 540 */ 191, 192, 193, 194, 195, 196, 197, 198, 264, 271, - /* 550 */ 293, 267, 250, 47, 243, 0, 250, 279, 243, 14, - /* 560 */ 18, 279, 20, 47, 262, 20, 272, 289, 262, 27, - /* 570 */ 64, 293, 30, 302, 250, 269, 298, 0, 82, 298, - /* 580 */ 64, 279, 57, 211, 243, 279, 262, 81, 310, 151, - /* 590 */ 48, 313, 314, 315, 316, 317, 318, 272, 320, 317, - /* 600 */ 45, 323, 272, 279, 293, 327, 328, 272, 293, 103, - /* 610 */ 172, 173, 250, 331, 332, 333, 334, 339, 336, 113, - /* 620 */ 339, 42, 43, 271, 262, 244, 1, 2, 145, 351, - /* 630 */ 278, 269, 351, 355, 293, 283, 355, 60, 61, 62, - /* 640 */ 63, 279, 65, 66, 67, 68, 69, 70, 71, 72, - /* 650 */ 73, 74, 75, 76, 77, 78, 14, 15, 16, 243, - /* 660 */ 118, 4, 156, 121, 122, 123, 124, 125, 126, 127, - /* 670 */ 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, - /* 680 */ 138, 226, 243, 289, 243, 185, 20, 181, 182, 295, - /* 690 */ 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, - /* 700 */ 194, 195, 196, 197, 198, 139, 18, 82, 243, 293, - /* 710 */ 271, 23, 259, 250, 261, 37, 272, 250, 279, 0, - /* 720 */ 20, 250, 298, 35, 36, 262, 160, 39, 289, 262, - /* 730 */ 185, 250, 293, 262, 293, 250, 12, 13, 14, 15, - /* 740 */ 16, 243, 279, 262, 56, 243, 279, 262, 243, 310, - /* 750 */ 279, 41, 313, 314, 315, 316, 317, 318, 293, 320, - /* 760 */ 279, 243, 323, 339, 279, 199, 327, 328, 329, 81, - /* 770 */ 12, 13, 14, 15, 16, 351, 271, 0, 250, 355, - /* 780 */ 341, 250, 58, 271, 279, 346, 347, 93, 250, 243, - /* 790 */ 262, 293, 280, 262, 289, 293, 85, 243, 293, 88, - /* 800 */ 262, 296, 243, 243, 243, 117, 82, 279, 243, 115, - /* 810 */ 279, 293, 93, 41, 90, 310, 58, 279, 313, 314, - /* 820 */ 315, 316, 317, 318, 58, 320, 209, 210, 250, 272, - /* 830 */ 21, 112, 271, 114, 115, 116, 148, 149, 150, 293, - /* 840 */ 279, 153, 260, 34, 271, 145, 158, 293, 90, 41, - /* 850 */ 289, 185, 293, 293, 293, 243, 283, 279, 293, 171, - /* 860 */ 22, 358, 174, 139, 176, 177, 178, 179, 180, 212, - /* 870 */ 93, 310, 0, 47, 313, 314, 315, 316, 317, 318, - /* 880 */ 319, 320, 321, 322, 160, 47, 243, 0, 210, 112, - /* 890 */ 82, 114, 115, 116, 22, 317, 41, 139, 85, 211, - /* 900 */ 85, 88, 64, 88, 85, 293, 243, 88, 41, 22, - /* 910 */ 332, 333, 334, 0, 336, 1, 2, 44, 160, 196, - /* 920 */ 197, 349, 47, 199, 200, 201, 202, 203, 204, 205, - /* 930 */ 206, 207, 208, 41, 271, 22, 293, 82, 228, 113, - /* 940 */ 81, 103, 279, 41, 41, 306, 41, 181, 41, 82, - /* 950 */ 91, 113, 289, 343, 81, 271, 293, 199, 200, 201, - /* 960 */ 202, 203, 204, 205, 206, 207, 208, 41, 243, 249, - /* 970 */ 41, 282, 41, 310, 82, 41, 313, 314, 315, 316, - /* 980 */ 317, 318, 47, 320, 82, 82, 323, 82, 113, 82, - /* 990 */ 327, 328, 329, 41, 156, 251, 271, 312, 337, 64, - /* 1000 */ 243, 352, 230, 41, 279, 352, 41, 352, 82, 340, - /* 1010 */ 347, 82, 213, 82, 289, 41, 82, 41, 293, 181, - /* 1020 */ 182, 308, 20, 250, 45, 307, 257, 47, 271, 300, - /* 1030 */ 154, 250, 250, 287, 82, 310, 279, 40, 313, 314, - /* 1040 */ 315, 316, 317, 318, 82, 320, 289, 82, 323, 139, - /* 1050 */ 293, 250, 327, 328, 329, 285, 82, 20, 82, 285, - /* 1060 */ 243, 245, 20, 338, 304, 245, 255, 310, 289, 255, - /* 1070 */ 313, 314, 315, 316, 317, 318, 20, 320, 20, 299, - /* 1080 */ 323, 297, 255, 297, 327, 328, 329, 255, 271, 279, - /* 1090 */ 20, 255, 243, 290, 255, 338, 279, 250, 255, 245, - /* 1100 */ 271, 271, 271, 250, 271, 271, 289, 245, 271, 64, - /* 1110 */ 293, 271, 271, 271, 293, 271, 271, 293, 253, 163, - /* 1120 */ 271, 253, 304, 253, 289, 303, 20, 310, 279, 250, - /* 1130 */ 313, 314, 315, 316, 317, 318, 279, 320, 289, 253, - /* 1140 */ 323, 219, 293, 243, 327, 328, 329, 298, 297, 293, - /* 1150 */ 290, 218, 348, 348, 293, 338, 147, 294, 312, 310, - /* 1160 */ 294, 225, 313, 314, 315, 316, 317, 318, 345, 320, - /* 1170 */ 293, 271, 344, 243, 214, 279, 210, 308, 20, 279, - /* 1180 */ 40, 232, 311, 229, 227, 81, 291, 294, 339, 289, - /* 1190 */ 342, 293, 293, 293, 330, 294, 142, 293, 298, 279, - /* 1200 */ 351, 271, 290, 353, 355, 267, 243, 279, 326, 279, - /* 1210 */ 310, 359, 354, 313, 314, 315, 316, 317, 318, 289, - /* 1220 */ 320, 253, 253, 293, 81, 275, 253, 250, 354, 245, - /* 1230 */ 305, 353, 261, 353, 271, 354, 243, 265, 301, 339, - /* 1240 */ 310, 254, 279, 313, 314, 315, 316, 317, 318, 241, - /* 1250 */ 320, 351, 289, 323, 265, 355, 293, 327, 328, 265, - /* 1260 */ 0, 0, 40, 0, 271, 72, 0, 47, 175, 175, - /* 1270 */ 47, 47, 279, 310, 47, 0, 313, 314, 315, 316, - /* 1280 */ 317, 318, 289, 320, 47, 47, 293, 243, 175, 0, - /* 1290 */ 175, 0, 47, 0, 22, 0, 243, 47, 0, 81, - /* 1300 */ 113, 160, 156, 310, 159, 0, 313, 314, 315, 316, - /* 1310 */ 317, 318, 0, 320, 152, 271, 323, 151, 0, 356, - /* 1320 */ 357, 328, 0, 279, 271, 44, 0, 0, 0, 0, - /* 1330 */ 0, 0, 279, 289, 0, 0, 0, 293, 0, 0, - /* 1340 */ 0, 0, 289, 0, 0, 0, 293, 0, 0, 40, - /* 1350 */ 243, 0, 0, 0, 310, 0, 0, 313, 314, 315, - /* 1360 */ 316, 317, 318, 310, 320, 0, 313, 314, 315, 316, - /* 1370 */ 317, 318, 22, 320, 0, 0, 0, 0, 271, 0, - /* 1380 */ 0, 14, 40, 37, 14, 0, 279, 0, 0, 0, - /* 1390 */ 243, 147, 0, 44, 350, 41, 289, 38, 37, 37, - /* 1400 */ 293, 44, 0, 296, 90, 37, 243, 0, 0, 0, - /* 1410 */ 357, 0, 0, 47, 45, 37, 45, 310, 271, 59, - /* 1420 */ 313, 314, 315, 316, 317, 318, 279, 320, 47, 47, - /* 1430 */ 37, 45, 37, 0, 271, 37, 289, 47, 0, 45, - /* 1440 */ 293, 0, 279, 0, 0, 22, 88, 47, 0, 0, - /* 1450 */ 47, 41, 289, 47, 243, 41, 293, 310, 22, 296, - /* 1460 */ 313, 314, 315, 316, 317, 318, 0, 320, 47, 322, - /* 1470 */ 48, 47, 47, 310, 47, 47, 313, 314, 315, 316, - /* 1480 */ 317, 318, 271, 320, 22, 0, 243, 22, 0, 47, - /* 1490 */ 279, 33, 0, 22, 22, 20, 0, 47, 0, 22, - /* 1500 */ 289, 0, 0, 45, 293, 0, 0, 296, 145, 51, - /* 1510 */ 52, 53, 54, 55, 271, 81, 243, 37, 145, 41, - /* 1520 */ 41, 310, 279, 140, 313, 314, 315, 316, 317, 318, - /* 1530 */ 215, 320, 289, 161, 82, 41, 293, 82, 80, 142, - /* 1540 */ 81, 83, 81, 44, 271, 81, 145, 44, 41, 82, - /* 1550 */ 81, 41, 279, 310, 41, 82, 313, 314, 315, 316, - /* 1560 */ 317, 318, 289, 320, 44, 82, 293, 243, 44, 82, - /* 1570 */ 41, 82, 47, 47, 47, 47, 2, 47, 47, 41, - /* 1580 */ 181, 44, 82, 310, 44, 81, 313, 314, 315, 316, - /* 1590 */ 317, 318, 81, 320, 82, 271, 81, 22, 140, 215, - /* 1600 */ 142, 183, 144, 279, 146, 209, 215, 243, 0, 37, - /* 1610 */ 91, 143, 44, 289, 82, 81, 81, 293, 82, 22, - /* 1620 */ 81, 140, 164, 81, 81, 44, 81, 47, 82, 81, - /* 1630 */ 81, 113, 82, 92, 310, 271, 47, 313, 314, 315, - /* 1640 */ 316, 317, 318, 279, 320, 81, 47, 243, 82, 82, - /* 1650 */ 81, 47, 81, 289, 47, 82, 47, 293, 81, 105, - /* 1660 */ 82, 105, 243, 81, 105, 105, 81, 22, 81, 93, - /* 1670 */ 47, 81, 22, 58, 310, 271, 59, 313, 314, 315, - /* 1680 */ 316, 317, 318, 279, 320, 47, 64, 41, 47, 79, - /* 1690 */ 271, 22, 64, 289, 47, 47, 47, 293, 279, 47, - /* 1700 */ 47, 47, 47, 47, 47, 47, 47, 47, 289, 47, - /* 1710 */ 0, 47, 293, 243, 310, 45, 37, 313, 314, 315, - /* 1720 */ 316, 317, 318, 0, 320, 243, 45, 47, 37, 310, - /* 1730 */ 0, 47, 313, 314, 315, 316, 317, 318, 45, 320, - /* 1740 */ 37, 271, 47, 45, 37, 0, 47, 46, 0, 279, - /* 1750 */ 0, 0, 21, 271, 22, 22, 22, 21, 20, 289, - /* 1760 */ 360, 279, 360, 293, 360, 243, 360, 360, 360, 360, - /* 1770 */ 360, 289, 360, 360, 360, 293, 243, 360, 360, 360, - /* 1780 */ 310, 360, 360, 313, 314, 315, 316, 317, 318, 360, - /* 1790 */ 320, 360, 310, 271, 360, 313, 314, 315, 316, 317, - /* 1800 */ 318, 279, 320, 360, 271, 360, 360, 360, 360, 360, - /* 1810 */ 360, 289, 279, 360, 360, 293, 360, 360, 360, 360, - /* 1820 */ 360, 360, 289, 360, 360, 360, 293, 243, 360, 360, - /* 1830 */ 360, 360, 310, 360, 360, 313, 314, 315, 316, 317, - /* 1840 */ 318, 360, 320, 310, 360, 360, 313, 314, 315, 316, - /* 1850 */ 317, 318, 360, 320, 360, 271, 360, 243, 360, 360, - /* 1860 */ 360, 360, 360, 279, 360, 360, 360, 360, 360, 360, - /* 1870 */ 360, 360, 360, 289, 360, 360, 360, 293, 360, 360, - /* 1880 */ 360, 360, 360, 360, 360, 271, 360, 243, 360, 360, - /* 1890 */ 360, 360, 360, 279, 310, 360, 360, 313, 314, 315, - /* 1900 */ 316, 317, 318, 289, 320, 360, 360, 293, 360, 360, - /* 1910 */ 360, 360, 360, 360, 360, 271, 360, 360, 360, 243, - /* 1920 */ 360, 360, 360, 279, 310, 360, 360, 313, 314, 315, - /* 1930 */ 316, 317, 318, 289, 320, 360, 360, 293, 360, 360, - /* 1940 */ 360, 360, 360, 12, 13, 360, 360, 271, 360, 360, - /* 1950 */ 360, 360, 243, 22, 310, 279, 360, 313, 314, 315, - /* 1960 */ 316, 317, 318, 360, 320, 289, 360, 360, 360, 293, - /* 1970 */ 360, 360, 360, 360, 360, 360, 360, 360, 47, 360, - /* 1980 */ 271, 360, 360, 360, 360, 243, 310, 360, 279, 313, - /* 1990 */ 314, 315, 316, 317, 318, 64, 320, 360, 289, 360, - /* 2000 */ 360, 360, 293, 360, 360, 360, 360, 360, 360, 360, - /* 2010 */ 360, 360, 360, 271, 360, 360, 250, 360, 360, 310, - /* 2020 */ 360, 279, 313, 314, 315, 316, 317, 318, 360, 320, - /* 2030 */ 360, 289, 360, 360, 103, 293, 360, 360, 360, 360, - /* 2040 */ 360, 360, 360, 360, 113, 279, 250, 360, 360, 360, - /* 2050 */ 360, 360, 310, 360, 360, 313, 314, 315, 316, 317, - /* 2060 */ 318, 360, 320, 360, 298, 360, 360, 360, 360, 360, - /* 2070 */ 360, 360, 360, 360, 360, 279, 360, 360, 360, 360, - /* 2080 */ 360, 360, 360, 317, 360, 360, 360, 156, 360, 360, - /* 2090 */ 360, 360, 360, 360, 298, 360, 360, 360, 332, 333, - /* 2100 */ 334, 360, 336, 360, 360, 339, 360, 360, 360, 360, - /* 2110 */ 360, 360, 181, 317, 360, 360, 360, 351, 360, 360, - /* 2120 */ 360, 355, 360, 192, 193, 194, 360, 360, 332, 333, - /* 2130 */ 334, 360, 336, 360, 360, 339, 360, 360, 360, 360, - /* 2140 */ 360, 360, 360, 360, 360, 360, 360, 351, 360, 360, - /* 2150 */ 360, 355, 360, 360, 360, 360, 360, 360, 360, 360, - /* 2160 */ 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, - /* 2170 */ 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, - /* 2180 */ 360, 360, 360, 360, + /* 0 */ 269, 280, 242, 271, 283, 284, 274, 245, 271, 247, + /* 10 */ 248, 274, 12, 13, 282, 245, 285, 247, 248, 282, + /* 20 */ 20, 0, 22, 269, 12, 13, 14, 15, 16, 269, + /* 30 */ 276, 20, 12, 13, 14, 15, 16, 277, 338, 285, + /* 40 */ 308, 309, 282, 338, 284, 308, 309, 47, 323, 324, + /* 50 */ 350, 319, 249, 239, 354, 350, 319, 297, 58, 354, + /* 60 */ 12, 13, 14, 246, 64, 242, 249, 20, 20, 309, + /* 70 */ 22, 55, 312, 313, 314, 315, 316, 317, 57, 319, + /* 80 */ 4, 81, 322, 20, 14, 338, 326, 327, 249, 286, + /* 90 */ 20, 249, 269, 249, 82, 47, 80, 350, 338, 83, + /* 100 */ 277, 354, 260, 103, 260, 282, 58, 284, 4, 267, + /* 110 */ 350, 297, 64, 2, 354, 57, 277, 117, 118, 277, + /* 120 */ 297, 277, 297, 12, 13, 14, 15, 16, 81, 81, + /* 130 */ 254, 249, 309, 269, 258, 312, 313, 314, 315, 316, + /* 140 */ 317, 277, 319, 284, 81, 322, 42, 43, 14, 326, + /* 150 */ 327, 103, 338, 294, 20, 316, 113, 81, 158, 277, + /* 160 */ 160, 338, 20, 338, 350, 117, 118, 147, 354, 330, + /* 170 */ 331, 332, 333, 350, 335, 350, 57, 354, 314, 354, + /* 180 */ 41, 145, 182, 183, 153, 185, 186, 187, 188, 189, + /* 190 */ 190, 191, 192, 193, 194, 195, 196, 197, 316, 12, + /* 200 */ 13, 14, 15, 16, 173, 174, 158, 241, 160, 243, + /* 210 */ 210, 168, 169, 331, 332, 333, 20, 335, 284, 12, + /* 220 */ 13, 12, 13, 14, 15, 16, 292, 20, 294, 22, + /* 230 */ 182, 183, 0, 185, 186, 187, 188, 189, 190, 191, + /* 240 */ 192, 193, 194, 195, 196, 197, 12, 13, 14, 15, + /* 250 */ 16, 215, 216, 21, 47, 251, 24, 25, 26, 27, + /* 260 */ 28, 29, 30, 31, 32, 58, 20, 12, 13, 82, + /* 270 */ 2, 64, 0, 210, 271, 20, 272, 22, 47, 81, + /* 280 */ 12, 13, 14, 15, 16, 282, 210, 246, 81, 182, + /* 290 */ 249, 94, 95, 96, 97, 98, 99, 100, 101, 102, + /* 300 */ 103, 104, 47, 106, 107, 108, 109, 110, 111, 14, + /* 310 */ 103, 308, 309, 117, 118, 20, 14, 15, 16, 64, + /* 320 */ 186, 41, 319, 242, 117, 118, 219, 220, 221, 222, + /* 330 */ 223, 20, 60, 61, 62, 63, 81, 65, 66, 67, + /* 340 */ 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, + /* 350 */ 78, 0, 257, 261, 259, 157, 81, 159, 103, 150, + /* 360 */ 20, 269, 82, 282, 93, 158, 242, 160, 229, 242, + /* 370 */ 278, 21, 117, 118, 24, 25, 26, 27, 28, 29, + /* 380 */ 30, 31, 32, 112, 113, 114, 115, 116, 253, 182, + /* 390 */ 183, 160, 185, 186, 187, 188, 189, 190, 191, 192, + /* 400 */ 193, 194, 195, 196, 197, 249, 282, 249, 210, 282, + /* 410 */ 275, 60, 61, 158, 269, 160, 65, 242, 260, 68, + /* 420 */ 69, 276, 242, 72, 73, 74, 12, 13, 14, 279, + /* 430 */ 285, 242, 282, 277, 20, 277, 22, 182, 183, 230, + /* 440 */ 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, + /* 450 */ 195, 196, 197, 242, 22, 60, 61, 282, 269, 242, + /* 460 */ 65, 47, 282, 68, 69, 210, 277, 72, 73, 74, + /* 470 */ 112, 282, 316, 284, 12, 13, 283, 284, 64, 47, + /* 480 */ 64, 186, 20, 242, 22, 210, 249, 331, 332, 333, + /* 490 */ 35, 335, 20, 282, 22, 81, 64, 260, 309, 282, + /* 500 */ 253, 312, 313, 314, 315, 316, 317, 242, 319, 47, + /* 510 */ 269, 322, 55, 266, 277, 326, 327, 103, 277, 47, + /* 520 */ 297, 49, 275, 282, 279, 284, 64, 282, 170, 171, + /* 530 */ 85, 117, 118, 88, 249, 103, 12, 13, 81, 84, + /* 540 */ 83, 86, 87, 81, 89, 260, 22, 282, 93, 0, + /* 550 */ 309, 4, 267, 312, 313, 314, 315, 316, 317, 47, + /* 560 */ 319, 338, 277, 322, 20, 103, 22, 326, 327, 328, + /* 570 */ 115, 47, 158, 350, 160, 297, 64, 354, 269, 117, + /* 580 */ 118, 340, 255, 256, 249, 276, 345, 346, 64, 75, + /* 590 */ 158, 37, 160, 49, 285, 260, 182, 183, 20, 185, + /* 600 */ 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, + /* 610 */ 196, 197, 277, 261, 182, 183, 338, 141, 42, 43, + /* 620 */ 158, 269, 160, 261, 268, 251, 242, 103, 350, 242, + /* 630 */ 278, 269, 354, 12, 13, 121, 122, 281, 264, 163, + /* 640 */ 278, 20, 93, 22, 182, 183, 272, 185, 186, 187, + /* 650 */ 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, + /* 660 */ 242, 112, 113, 114, 115, 116, 282, 210, 47, 282, + /* 670 */ 249, 242, 1, 2, 198, 271, 242, 249, 18, 3, + /* 680 */ 20, 260, 158, 310, 160, 64, 282, 27, 260, 242, + /* 690 */ 30, 12, 13, 14, 15, 16, 255, 256, 277, 249, + /* 700 */ 282, 242, 81, 20, 18, 277, 182, 334, 48, 23, + /* 710 */ 260, 282, 308, 309, 47, 242, 282, 193, 194, 195, + /* 720 */ 0, 35, 36, 319, 103, 39, 242, 277, 269, 282, + /* 730 */ 242, 262, 310, 310, 265, 41, 277, 58, 117, 118, + /* 740 */ 0, 282, 56, 284, 24, 25, 26, 27, 28, 29, + /* 750 */ 30, 31, 32, 82, 295, 282, 334, 334, 211, 208, + /* 760 */ 209, 82, 242, 209, 186, 92, 282, 81, 309, 90, + /* 770 */ 282, 312, 313, 314, 315, 316, 317, 64, 319, 158, + /* 780 */ 120, 160, 0, 123, 124, 125, 126, 127, 128, 129, + /* 790 */ 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, + /* 800 */ 140, 20, 282, 182, 183, 119, 185, 186, 187, 188, + /* 810 */ 189, 190, 191, 192, 193, 194, 195, 196, 197, 269, + /* 820 */ 141, 249, 12, 13, 14, 15, 16, 160, 278, 249, + /* 830 */ 147, 0, 260, 93, 249, 242, 150, 151, 152, 57, + /* 840 */ 260, 155, 163, 85, 58, 260, 88, 161, 85, 277, + /* 850 */ 93, 88, 112, 113, 114, 115, 116, 277, 172, 249, + /* 860 */ 249, 175, 277, 177, 178, 179, 180, 181, 58, 270, + /* 870 */ 260, 260, 115, 0, 270, 282, 45, 198, 199, 200, + /* 880 */ 201, 202, 203, 204, 205, 206, 207, 277, 277, 0, + /* 890 */ 242, 277, 249, 0, 85, 22, 210, 88, 44, 21, + /* 900 */ 90, 225, 288, 260, 41, 19, 41, 47, 41, 117, + /* 910 */ 118, 22, 34, 1, 2, 22, 81, 269, 41, 33, + /* 920 */ 277, 227, 270, 270, 64, 277, 91, 270, 147, 270, + /* 930 */ 282, 45, 284, 270, 33, 81, 41, 51, 52, 53, + /* 940 */ 54, 55, 301, 243, 41, 82, 45, 82, 41, 82, + /* 950 */ 41, 141, 51, 52, 53, 54, 55, 309, 357, 82, + /* 960 */ 312, 313, 314, 315, 316, 317, 80, 319, 182, 83, + /* 970 */ 322, 348, 258, 163, 326, 327, 328, 82, 41, 41, + /* 980 */ 41, 80, 41, 41, 83, 82, 41, 41, 0, 82, + /* 990 */ 305, 82, 41, 250, 346, 269, 342, 250, 248, 281, + /* 1000 */ 311, 336, 116, 339, 242, 212, 351, 351, 198, 199, + /* 1010 */ 200, 201, 202, 203, 204, 205, 206, 207, 307, 82, + /* 1020 */ 82, 82, 20, 82, 82, 351, 249, 82, 82, 143, + /* 1030 */ 242, 269, 146, 82, 46, 45, 306, 156, 47, 277, + /* 1040 */ 255, 299, 289, 142, 282, 144, 284, 146, 249, 148, + /* 1050 */ 249, 165, 40, 167, 141, 287, 242, 269, 287, 20, + /* 1060 */ 249, 244, 244, 20, 303, 277, 253, 284, 167, 253, + /* 1070 */ 282, 309, 284, 20, 312, 313, 314, 315, 316, 317, + /* 1080 */ 296, 319, 20, 269, 322, 298, 253, 296, 326, 327, + /* 1090 */ 328, 277, 253, 20, 290, 253, 282, 309, 284, 337, + /* 1100 */ 312, 313, 314, 315, 316, 317, 277, 319, 242, 253, + /* 1110 */ 322, 249, 253, 244, 326, 327, 328, 269, 269, 269, + /* 1120 */ 269, 269, 242, 309, 269, 337, 312, 313, 314, 315, + /* 1130 */ 316, 317, 244, 319, 249, 269, 322, 269, 251, 269, + /* 1140 */ 326, 327, 328, 277, 269, 269, 282, 303, 282, 269, + /* 1150 */ 284, 337, 269, 282, 249, 166, 284, 277, 20, 302, + /* 1160 */ 217, 251, 282, 297, 284, 251, 251, 277, 218, 290, + /* 1170 */ 296, 347, 311, 242, 282, 309, 282, 297, 312, 313, + /* 1180 */ 314, 315, 316, 317, 293, 319, 282, 347, 224, 309, + /* 1190 */ 4, 293, 312, 313, 314, 315, 316, 317, 149, 319, + /* 1200 */ 269, 209, 344, 307, 338, 19, 213, 277, 277, 20, + /* 1210 */ 310, 40, 228, 282, 231, 284, 350, 81, 338, 33, + /* 1220 */ 354, 293, 226, 144, 291, 343, 282, 293, 329, 282, + /* 1230 */ 350, 45, 277, 251, 354, 282, 50, 242, 341, 290, + /* 1240 */ 309, 55, 358, 312, 313, 314, 315, 316, 317, 325, + /* 1250 */ 319, 242, 265, 322, 251, 353, 277, 326, 327, 353, + /* 1260 */ 242, 353, 81, 352, 269, 259, 80, 273, 352, 83, + /* 1270 */ 352, 249, 277, 244, 251, 300, 304, 282, 269, 284, + /* 1280 */ 282, 240, 263, 263, 263, 252, 277, 269, 0, 0, + /* 1290 */ 40, 282, 0, 284, 72, 277, 0, 176, 47, 47, + /* 1300 */ 282, 47, 284, 47, 309, 176, 0, 312, 313, 314, + /* 1310 */ 315, 316, 317, 318, 319, 320, 321, 242, 309, 47, + /* 1320 */ 47, 312, 313, 314, 315, 316, 317, 309, 319, 176, + /* 1330 */ 312, 313, 314, 315, 316, 317, 0, 319, 176, 242, + /* 1340 */ 322, 0, 47, 0, 269, 327, 22, 0, 47, 0, + /* 1350 */ 81, 163, 277, 162, 160, 158, 0, 282, 0, 284, + /* 1360 */ 154, 153, 0, 0, 355, 356, 269, 44, 0, 0, + /* 1370 */ 0, 0, 0, 0, 277, 0, 0, 0, 0, 282, + /* 1380 */ 0, 284, 0, 0, 309, 0, 0, 312, 313, 314, + /* 1390 */ 315, 316, 317, 242, 319, 0, 0, 0, 0, 40, + /* 1400 */ 0, 0, 0, 0, 0, 242, 309, 0, 0, 312, + /* 1410 */ 313, 314, 315, 316, 317, 22, 319, 0, 0, 0, + /* 1420 */ 269, 0, 41, 14, 349, 37, 14, 40, 277, 0, + /* 1430 */ 0, 0, 269, 282, 44, 284, 44, 38, 0, 149, + /* 1440 */ 277, 0, 37, 0, 0, 282, 295, 284, 37, 0, + /* 1450 */ 37, 0, 37, 356, 0, 0, 242, 37, 0, 37, + /* 1460 */ 309, 47, 37, 312, 313, 314, 315, 316, 317, 59, + /* 1470 */ 319, 45, 309, 45, 47, 312, 313, 314, 315, 316, + /* 1480 */ 317, 47, 319, 269, 321, 45, 47, 45, 0, 0, + /* 1490 */ 0, 277, 0, 47, 22, 0, 282, 47, 284, 47, + /* 1500 */ 47, 41, 41, 47, 47, 0, 90, 242, 22, 295, + /* 1510 */ 47, 88, 47, 0, 48, 22, 0, 47, 22, 0, + /* 1520 */ 22, 242, 0, 309, 22, 0, 312, 313, 314, 315, + /* 1530 */ 316, 317, 20, 319, 269, 0, 147, 22, 47, 0, + /* 1540 */ 0, 0, 277, 164, 0, 81, 37, 282, 269, 284, + /* 1550 */ 82, 41, 41, 214, 81, 41, 277, 82, 81, 81, + /* 1560 */ 295, 282, 82, 284, 41, 44, 81, 41, 44, 214, + /* 1570 */ 242, 147, 44, 208, 309, 142, 144, 312, 313, 314, + /* 1580 */ 315, 316, 317, 147, 319, 82, 82, 242, 309, 41, + /* 1590 */ 44, 312, 313, 314, 315, 316, 317, 269, 319, 82, + /* 1600 */ 41, 47, 82, 47, 2, 277, 41, 47, 47, 47, + /* 1610 */ 282, 47, 284, 82, 269, 81, 44, 44, 82, 182, + /* 1620 */ 82, 22, 277, 81, 81, 0, 81, 282, 81, 284, + /* 1630 */ 37, 82, 184, 81, 81, 44, 242, 309, 44, 145, + /* 1640 */ 312, 313, 314, 315, 316, 317, 91, 319, 242, 214, + /* 1650 */ 81, 81, 22, 82, 309, 81, 81, 312, 313, 314, + /* 1660 */ 315, 316, 317, 269, 319, 82, 47, 47, 92, 142, + /* 1670 */ 81, 277, 82, 47, 82, 269, 282, 81, 284, 47, + /* 1680 */ 47, 81, 81, 277, 47, 82, 82, 81, 282, 242, + /* 1690 */ 284, 105, 105, 22, 81, 81, 47, 81, 93, 41, + /* 1700 */ 105, 22, 105, 309, 59, 58, 312, 313, 314, 315, + /* 1710 */ 316, 317, 47, 319, 64, 309, 269, 79, 312, 313, + /* 1720 */ 314, 315, 316, 317, 277, 319, 41, 47, 47, 282, + /* 1730 */ 47, 284, 22, 47, 47, 47, 47, 64, 47, 47, + /* 1740 */ 47, 242, 47, 47, 47, 47, 0, 47, 45, 37, + /* 1750 */ 242, 0, 47, 45, 0, 37, 309, 0, 47, 312, + /* 1760 */ 313, 314, 315, 316, 317, 45, 319, 37, 269, 47, + /* 1770 */ 45, 37, 0, 47, 0, 22, 277, 269, 22, 21, + /* 1780 */ 21, 282, 22, 284, 20, 277, 359, 359, 359, 359, + /* 1790 */ 282, 359, 284, 359, 359, 359, 359, 359, 359, 242, + /* 1800 */ 359, 359, 359, 359, 359, 359, 359, 359, 309, 359, + /* 1810 */ 359, 312, 313, 314, 315, 316, 317, 309, 319, 359, + /* 1820 */ 312, 313, 314, 315, 316, 317, 269, 319, 359, 359, + /* 1830 */ 359, 359, 359, 359, 277, 359, 359, 249, 359, 282, + /* 1840 */ 242, 284, 359, 359, 359, 359, 359, 359, 359, 359, + /* 1850 */ 359, 359, 359, 359, 359, 359, 359, 242, 359, 359, + /* 1860 */ 359, 359, 359, 359, 359, 277, 309, 269, 359, 312, + /* 1870 */ 313, 314, 315, 316, 317, 277, 319, 359, 359, 359, + /* 1880 */ 282, 359, 284, 359, 269, 297, 359, 359, 359, 359, + /* 1890 */ 359, 359, 277, 359, 359, 359, 359, 282, 359, 284, + /* 1900 */ 359, 359, 359, 359, 316, 359, 242, 309, 359, 359, + /* 1910 */ 312, 313, 314, 315, 316, 317, 359, 319, 242, 331, + /* 1920 */ 332, 333, 359, 335, 309, 359, 338, 312, 313, 314, + /* 1930 */ 315, 316, 317, 269, 319, 359, 359, 359, 350, 249, + /* 1940 */ 359, 277, 354, 359, 359, 269, 282, 359, 284, 359, + /* 1950 */ 359, 359, 359, 277, 359, 359, 359, 359, 282, 242, + /* 1960 */ 284, 359, 359, 359, 359, 359, 359, 277, 359, 359, + /* 1970 */ 359, 359, 359, 309, 359, 359, 312, 313, 314, 315, + /* 1980 */ 316, 317, 359, 319, 359, 309, 269, 297, 312, 313, + /* 1990 */ 314, 315, 316, 317, 277, 319, 359, 359, 359, 282, + /* 2000 */ 359, 284, 359, 359, 359, 359, 316, 359, 359, 359, + /* 2010 */ 359, 242, 359, 359, 359, 359, 359, 359, 359, 359, + /* 2020 */ 242, 331, 332, 333, 359, 335, 309, 359, 338, 312, + /* 2030 */ 313, 314, 315, 316, 317, 359, 319, 359, 269, 359, + /* 2040 */ 350, 359, 359, 359, 354, 359, 277, 269, 359, 359, + /* 2050 */ 359, 282, 359, 284, 359, 277, 359, 359, 359, 359, + /* 2060 */ 282, 359, 284, 359, 359, 359, 359, 359, 359, 359, + /* 2070 */ 359, 359, 359, 359, 359, 359, 359, 359, 309, 359, + /* 2080 */ 359, 312, 313, 314, 315, 316, 317, 309, 319, 359, + /* 2090 */ 312, 313, 314, 315, 316, 317, 359, 319, }; -#define YY_SHIFT_COUNT (618) +#define YY_SHIFT_COUNT (621) #define YY_SHIFT_MIN (0) -#define YY_SHIFT_MAX (1931) +#define YY_SHIFT_MAX (1774) static const unsigned short int yy_shift_ofst[] = { - /* 0 */ 688, 0, 0, 48, 96, 96, 96, 96, 253, 253, - /* 10 */ 96, 96, 301, 349, 506, 349, 349, 349, 349, 349, - /* 20 */ 349, 349, 349, 349, 349, 349, 349, 349, 349, 349, - /* 30 */ 349, 349, 349, 349, 349, 349, 349, 349, 144, 144, - /* 40 */ 102, 102, 102, 1931, 1931, 1931, 1931, 372, 124, 214, - /* 50 */ 58, 58, 260, 260, 172, 214, 214, 58, 58, 58, - /* 60 */ 58, 58, 58, 58, 37, 58, 58, 186, 228, 259, - /* 70 */ 186, 58, 58, 186, 58, 186, 186, 259, 186, 58, - /* 80 */ 255, 542, 724, 758, 758, 189, 236, 838, 838, 838, - /* 90 */ 838, 838, 838, 838, 838, 838, 838, 838, 838, 838, - /* 100 */ 838, 838, 838, 838, 838, 838, 291, 126, 354, 354, - /* 110 */ 397, 516, 483, 483, 483, 525, 516, 313, 259, 186, - /* 120 */ 186, 259, 377, 343, 243, 243, 243, 243, 243, 243, - /* 130 */ 243, 340, 141, 405, 76, 195, 57, 215, 500, 545, - /* 140 */ 579, 179, 694, 700, 617, 678, 617, 455, 455, 455, - /* 150 */ 657, 666, 799, 1002, 979, 980, 876, 1002, 1002, 997, - /* 160 */ 910, 910, 1002, 1037, 1037, 1042, 37, 259, 37, 1056, - /* 170 */ 1058, 37, 1056, 37, 313, 1070, 37, 37, 1002, 37, - /* 180 */ 1037, 186, 186, 186, 186, 186, 186, 186, 186, 186, - /* 190 */ 186, 186, 1002, 1037, 1045, 1045, 1042, 255, 956, 259, - /* 200 */ 255, 1002, 1056, 255, 313, 1070, 255, 1106, 922, 933, - /* 210 */ 1045, 922, 933, 1045, 1045, 186, 936, 1009, 960, 799, - /* 220 */ 966, 313, 1158, 1140, 954, 957, 949, 954, 957, 954, - /* 230 */ 957, 1104, 933, 1045, 1045, 933, 1045, 1054, 313, 1070, - /* 240 */ 255, 377, 255, 313, 1143, 343, 1002, 255, 1037, 2152, - /* 250 */ 2152, 2152, 2152, 2152, 2152, 2152, 577, 1458, 231, 353, - /* 260 */ 3, 19, 316, 256, 496, 719, 777, 112, 112, 112, - /* 270 */ 112, 112, 112, 112, 112, 135, 438, 344, 396, 355, - /* 280 */ 625, 566, 642, 642, 642, 642, 555, 808, 711, 813, - /* 290 */ 815, 819, 872, 887, 913, 809, 855, 867, 892, 914, - /* 300 */ 723, 710, 772, 902, 766, 903, 873, 905, 907, 926, - /* 310 */ 929, 931, 826, 875, 934, 952, 962, 965, 974, 976, - /* 320 */ 859, 935, 1260, 1261, 1222, 1263, 1193, 1266, 1220, 1093, - /* 330 */ 1223, 1224, 1227, 1094, 1275, 1237, 1238, 1113, 1289, 1115, - /* 340 */ 1291, 1245, 1293, 1272, 1295, 1250, 1298, 1218, 1141, 1145, - /* 350 */ 1187, 1146, 1305, 1312, 1162, 1166, 1318, 1322, 1281, 1326, - /* 360 */ 1327, 1328, 1329, 1330, 1331, 1334, 1335, 1336, 1338, 1339, - /* 370 */ 1340, 1341, 1343, 1344, 1345, 1347, 1348, 1309, 1351, 1352, - /* 380 */ 1353, 1355, 1356, 1365, 1350, 1374, 1375, 1376, 1377, 1379, - /* 390 */ 1380, 1342, 1346, 1354, 1367, 1349, 1370, 1357, 1385, 1359, - /* 400 */ 1361, 1387, 1388, 1389, 1362, 1244, 1392, 1402, 1368, 1407, - /* 410 */ 1360, 1408, 1409, 1366, 1369, 1378, 1411, 1381, 1371, 1393, - /* 420 */ 1412, 1382, 1386, 1395, 1433, 1390, 1394, 1398, 1438, 1441, - /* 430 */ 1443, 1444, 1314, 1358, 1400, 1423, 1448, 1403, 1406, 1421, - /* 440 */ 1424, 1410, 1414, 1425, 1427, 1428, 1449, 1436, 1466, 1462, - /* 450 */ 1422, 1485, 1465, 1442, 1488, 1471, 1492, 1472, 1475, 1496, - /* 460 */ 1363, 1450, 1498, 1372, 1477, 1373, 1397, 1501, 1502, 1505, - /* 470 */ 1401, 1506, 1434, 1480, 1383, 1478, 1479, 1315, 1452, 1494, - /* 480 */ 1455, 1459, 1461, 1464, 1467, 1507, 1499, 1503, 1469, 1510, - /* 490 */ 1384, 1473, 1483, 1520, 1396, 1513, 1524, 1487, 1529, 1391, - /* 500 */ 1489, 1525, 1526, 1527, 1528, 1530, 1531, 1489, 1574, 1399, - /* 510 */ 1538, 1500, 1504, 1512, 1537, 1511, 1515, 1540, 1575, 1418, - /* 520 */ 1534, 1532, 1536, 1535, 1539, 1468, 1542, 1608, 1572, 1481, - /* 530 */ 1543, 1519, 1568, 1581, 1545, 1546, 1548, 1597, 1549, 1541, - /* 540 */ 1550, 1580, 1589, 1564, 1566, 1599, 1569, 1567, 1604, 1571, - /* 550 */ 1573, 1607, 1577, 1578, 1609, 1582, 1554, 1556, 1559, 1560, - /* 560 */ 1645, 1576, 1585, 1587, 1623, 1590, 1518, 1650, 1617, 1615, - /* 570 */ 1638, 1622, 1610, 1646, 1641, 1647, 1648, 1649, 1652, 1669, - /* 580 */ 1653, 1654, 1628, 1410, 1655, 1414, 1656, 1657, 1658, 1659, - /* 590 */ 1660, 1662, 1710, 1664, 1670, 1679, 1723, 1680, 1681, 1691, - /* 600 */ 1730, 1684, 1693, 1703, 1750, 1695, 1698, 1707, 1745, 1699, - /* 610 */ 1701, 1748, 1751, 1732, 1731, 1733, 1734, 1736, 1738, + /* 0 */ 686, 0, 0, 48, 207, 207, 207, 207, 255, 255, + /* 10 */ 207, 207, 414, 462, 621, 462, 462, 462, 462, 462, + /* 20 */ 462, 462, 462, 462, 462, 462, 462, 462, 462, 462, + /* 30 */ 462, 462, 462, 462, 462, 462, 462, 462, 63, 63, + /* 40 */ 47, 47, 47, 524, 524, 524, 524, 198, 457, 275, + /* 50 */ 11, 11, 104, 104, 76, 196, 275, 275, 11, 11, + /* 60 */ 11, 11, 11, 11, 11, 58, 11, 11, 142, 246, + /* 70 */ 311, 142, 11, 11, 142, 11, 142, 142, 142, 11, + /* 80 */ 119, 660, 679, 810, 810, 350, 395, 432, 432, 432, + /* 90 */ 432, 432, 432, 432, 432, 432, 432, 432, 432, 432, + /* 100 */ 432, 432, 432, 432, 432, 432, 455, 196, 70, 70, + /* 110 */ 21, 512, 683, 683, 683, 782, 512, 340, 311, 142, + /* 120 */ 142, 416, 416, 673, 713, 197, 197, 197, 197, 197, + /* 130 */ 197, 197, 886, 232, 351, 209, 107, 472, 43, 36, + /* 140 */ 134, 295, 544, 576, 757, 781, 551, 554, 551, 676, + /* 150 */ 676, 676, 547, 578, 793, 1002, 990, 991, 881, 1002, + /* 160 */ 1002, 1012, 913, 913, 1002, 1039, 1039, 1043, 58, 311, + /* 170 */ 58, 1053, 1062, 58, 1053, 58, 340, 1073, 58, 58, + /* 180 */ 1002, 58, 1039, 142, 142, 142, 142, 142, 142, 142, + /* 190 */ 142, 142, 142, 142, 1002, 1039, 416, 416, 1043, 119, + /* 200 */ 989, 311, 119, 1002, 1053, 119, 340, 1073, 119, 1138, + /* 210 */ 950, 943, 416, 950, 943, 416, 416, 142, 964, 1049, + /* 220 */ 993, 793, 992, 340, 1189, 1171, 984, 996, 983, 984, + /* 230 */ 996, 984, 996, 1136, 943, 416, 416, 943, 416, 1079, + /* 240 */ 340, 1073, 119, 673, 119, 340, 1181, 416, 713, 1002, + /* 250 */ 119, 1039, 2098, 2098, 2098, 2098, 2098, 2098, 2098, 272, + /* 260 */ 901, 720, 1186, 549, 740, 12, 111, 268, 20, 187, + /* 270 */ 271, 234, 234, 234, 234, 234, 234, 234, 234, 31, + /* 280 */ 16, 514, 358, 671, 476, 302, 302, 302, 302, 831, + /* 290 */ 280, 445, 758, 763, 809, 873, 889, 893, 878, 863, + /* 300 */ 865, 867, 912, 792, 694, 139, 877, 786, 895, 854, + /* 310 */ 903, 907, 909, 937, 938, 231, 667, 939, 941, 942, + /* 320 */ 945, 946, 951, 835, 860, 988, 1288, 1289, 1250, 1292, + /* 330 */ 1222, 1296, 1251, 1121, 1252, 1254, 1256, 1129, 1306, 1272, + /* 340 */ 1273, 1153, 1336, 1162, 1341, 1295, 1343, 1324, 1347, 1301, + /* 350 */ 1349, 1269, 1188, 1191, 1194, 1197, 1356, 1358, 1206, 1208, + /* 360 */ 1362, 1363, 1323, 1368, 1369, 1370, 1371, 1372, 1373, 1375, + /* 370 */ 1376, 1377, 1378, 1380, 1382, 1383, 1385, 1386, 1395, 1396, + /* 380 */ 1397, 1359, 1398, 1400, 1401, 1402, 1403, 1404, 1393, 1407, + /* 390 */ 1408, 1417, 1418, 1419, 1421, 1387, 1388, 1381, 1409, 1390, + /* 400 */ 1412, 1392, 1429, 1399, 1405, 1430, 1431, 1438, 1411, 1290, + /* 410 */ 1441, 1443, 1413, 1444, 1410, 1449, 1451, 1414, 1426, 1415, + /* 420 */ 1454, 1427, 1428, 1420, 1455, 1434, 1440, 1422, 1458, 1439, + /* 430 */ 1442, 1425, 1488, 1489, 1490, 1492, 1416, 1423, 1446, 1472, + /* 440 */ 1495, 1450, 1452, 1453, 1456, 1460, 1461, 1457, 1463, 1465, + /* 450 */ 1505, 1486, 1513, 1493, 1466, 1516, 1496, 1470, 1519, 1498, + /* 460 */ 1522, 1502, 1512, 1525, 1389, 1491, 1535, 1379, 1515, 1424, + /* 470 */ 1432, 1539, 1540, 1541, 1436, 1544, 1464, 1509, 1433, 1510, + /* 480 */ 1511, 1339, 1468, 1514, 1475, 1473, 1477, 1478, 1480, 1523, + /* 490 */ 1521, 1524, 1485, 1526, 1355, 1503, 1504, 1528, 1365, 1548, + /* 500 */ 1546, 1517, 1559, 1435, 1520, 1554, 1556, 1560, 1561, 1562, + /* 510 */ 1564, 1520, 1602, 1437, 1565, 1531, 1534, 1536, 1572, 1542, + /* 520 */ 1543, 1573, 1599, 1448, 1545, 1538, 1549, 1547, 1552, 1494, + /* 530 */ 1553, 1625, 1593, 1527, 1569, 1555, 1591, 1594, 1570, 1571, + /* 540 */ 1574, 1630, 1575, 1576, 1583, 1619, 1620, 1589, 1590, 1626, + /* 550 */ 1596, 1592, 1632, 1600, 1603, 1633, 1601, 1604, 1637, 1606, + /* 560 */ 1586, 1587, 1595, 1597, 1671, 1605, 1613, 1614, 1649, 1616, + /* 570 */ 1658, 1658, 1679, 1645, 1647, 1665, 1650, 1638, 1685, 1680, + /* 580 */ 1681, 1683, 1686, 1687, 1710, 1688, 1689, 1673, 1460, 1691, + /* 590 */ 1461, 1692, 1693, 1695, 1696, 1697, 1698, 1746, 1700, 1703, + /* 600 */ 1712, 1751, 1705, 1708, 1718, 1754, 1711, 1720, 1730, 1757, + /* 610 */ 1722, 1725, 1734, 1772, 1726, 1774, 1753, 1758, 1756, 1760, + /* 620 */ 1759, 1764, }; -#define YY_REDUCE_COUNT (255) -#define YY_REDUCE_MIN (-324) -#define YY_REDUCE_MAX (1796) +#define YY_REDUCE_COUNT (258) +#define YY_REDUCE_MIN (-300) +#define YY_REDUCE_MAX (1778) static const short yy_reduce_ofst[] = { - /* 0 */ 162, -241, 278, 439, 663, 725, 757, 817, 849, 900, - /* 10 */ -178, 930, 561, 963, 993, 505, 1044, 1053, 1107, 1147, - /* 20 */ 1163, 1211, 1243, 1273, 1324, 1364, 1404, 1419, 1470, 1482, - /* 30 */ 1522, 1533, 1584, 1614, 1644, 1676, 1709, 1742, 1766, 1796, - /* 40 */ 282, -213, 578, -270, -266, -264, -176, -284, 281, 424, - /* 50 */ 306, 362, -91, -46, -255, -192, 71, -226, -199, 150, - /* 60 */ 302, 324, 463, 467, -1, 471, 485, 28, 45, -190, - /* 70 */ 56, 528, 531, 197, 481, 192, 352, 100, 196, 538, - /* 80 */ -225, -242, -324, -324, -324, -64, -48, 77, 168, 257, - /* 90 */ 311, 315, 341, 416, 441, 465, 498, 502, 518, 546, - /* 100 */ 554, 559, 560, 565, 612, 643, 97, -247, 75, 140, - /* 110 */ -125, -232, -252, -204, -37, -100, 106, 142, 394, 512, - /* 120 */ 573, 218, 284, 453, -261, 294, 325, 330, 335, 444, - /* 130 */ 557, 271, 381, 582, 503, 572, 639, 610, 684, 684, - /* 140 */ 720, 744, 689, 685, 661, 661, 661, 649, 653, 655, - /* 150 */ 669, 684, 713, 773, 718, 769, 729, 781, 782, 746, - /* 160 */ 770, 774, 801, 816, 820, 760, 811, 779, 814, 784, - /* 170 */ 780, 827, 786, 832, 810, 803, 836, 839, 847, 843, - /* 180 */ 854, 829, 830, 831, 833, 834, 837, 840, 841, 842, - /* 190 */ 844, 845, 853, 862, 821, 824, 818, 865, 822, 835, - /* 200 */ 868, 879, 851, 870, 857, 860, 886, 846, 804, 863, - /* 210 */ 856, 805, 866, 861, 877, 684, 823, 828, 848, 869, - /* 220 */ 661, 896, 871, 864, 858, 850, 852, 874, 878, 881, - /* 230 */ 880, 882, 893, 898, 899, 901, 904, 895, 920, 912, - /* 240 */ 968, 938, 969, 928, 950, 971, 977, 973, 984, 937, - /* 250 */ 925, 972, 989, 994, 987, 1008, + /* 0 */ -186, -240, -177, 241, 648, 762, 788, 814, 866, 880, + /* 10 */ 189, 931, 995, 1009, 1018, 459, 1075, 1097, 1151, 1163, + /* 20 */ 1214, 1265, 1279, 1328, 1345, 1394, 1406, 1447, 1499, 1508, + /* 30 */ 1557, 1598, 1615, 1664, 1676, 1717, 1769, 1778, 1588, 1690, + /* 40 */ -161, -118, 156, -268, -263, 3, 404, -175, 223, 278, + /* 50 */ -158, 285, -238, -230, -300, -279, -295, -253, -156, 158, + /* 60 */ 237, 335, 421, 428, 450, 247, 572, 580, -246, -136, + /* 70 */ -66, 92, 585, 610, 145, 611, 352, 309, 362, 643, + /* 80 */ 374, -197, -275, -275, -275, -34, -124, 81, 124, 127, + /* 90 */ 175, 180, 211, 217, 265, 384, 387, 418, 429, 434, + /* 100 */ 447, 473, 484, 488, 520, 593, 356, 193, -183, 41, + /* 110 */ 135, 327, 373, 422, 423, 4, 441, 614, -141, 550, + /* 120 */ -269, 150, 245, 469, 95, 599, 604, 652, 653, 657, + /* 130 */ 659, 663, 641, 700, 714, 601, 623, 743, 685, 654, + /* 140 */ 726, 726, 747, 750, 718, 689, 665, 665, 665, 655, + /* 150 */ 656, 674, 664, 726, 711, 777, 730, 785, 742, 799, + /* 160 */ 801, 753, 768, 771, 811, 817, 818, 761, 813, 783, + /* 170 */ 816, 784, 787, 833, 791, 839, 829, 804, 842, 856, + /* 180 */ 862, 859, 869, 848, 849, 850, 851, 852, 855, 868, + /* 190 */ 870, 875, 876, 883, 885, 888, 864, 871, 844, 887, + /* 200 */ 857, 872, 910, 905, 874, 914, 890, 879, 915, 861, + /* 210 */ 824, 891, 892, 840, 898, 894, 904, 726, 858, 882, + /* 220 */ 897, 896, 665, 930, 900, 899, 902, 911, 884, 906, + /* 230 */ 916, 908, 918, 924, 928, 944, 947, 934, 953, 933, + /* 240 */ 955, 949, 982, 987, 1003, 979, 994, 998, 1006, 1022, + /* 250 */ 1023, 1029, 975, 972, 1019, 1020, 1021, 1033, 1041, }; static const YYACTIONTYPE yy_default[] = { - /* 0 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 10 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 20 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 30 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 40 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 50 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 60 */ 1360, 1360, 1360, 1360, 1429, 1360, 1360, 1360, 1360, 1360, - /* 70 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 80 */ 1427, 1567, 1360, 1734, 1360, 1360, 1360, 1360, 1360, 1360, - /* 90 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 100 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 110 */ 1429, 1360, 1745, 1745, 1745, 1427, 1360, 1360, 1360, 1360, - /* 120 */ 1360, 1360, 1523, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 130 */ 1360, 1603, 1360, 1360, 1811, 1360, 1609, 1769, 1360, 1360, - /* 140 */ 1360, 1360, 1476, 1761, 1737, 1751, 1738, 1796, 1796, 1796, - /* 150 */ 1754, 1360, 1765, 1360, 1360, 1360, 1595, 1360, 1360, 1572, - /* 160 */ 1569, 1569, 1360, 1360, 1360, 1360, 1429, 1360, 1429, 1360, - /* 170 */ 1360, 1429, 1360, 1429, 1360, 1360, 1429, 1429, 1360, 1429, - /* 180 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 190 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1427, 1605, 1360, - /* 200 */ 1427, 1360, 1360, 1427, 1360, 1360, 1427, 1360, 1776, 1774, - /* 210 */ 1360, 1776, 1774, 1360, 1360, 1360, 1788, 1784, 1767, 1765, - /* 220 */ 1751, 1360, 1360, 1360, 1802, 1798, 1814, 1802, 1798, 1802, - /* 230 */ 1798, 1360, 1774, 1360, 1360, 1774, 1360, 1580, 1360, 1360, - /* 240 */ 1427, 1360, 1427, 1360, 1492, 1360, 1360, 1427, 1360, 1597, - /* 250 */ 1611, 1526, 1526, 1526, 1430, 1365, 1360, 1360, 1360, 1360, - /* 260 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1678, 1787, 1786, - /* 270 */ 1710, 1709, 1708, 1706, 1677, 1488, 1360, 1360, 1360, 1360, - /* 280 */ 1360, 1360, 1671, 1672, 1670, 1669, 1360, 1360, 1360, 1360, - /* 290 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1735, - /* 300 */ 1360, 1799, 1803, 1360, 1360, 1360, 1654, 1360, 1360, 1360, - /* 310 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 320 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 330 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 340 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 350 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 360 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 370 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 380 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 390 */ 1360, 1360, 1360, 1394, 1360, 1360, 1360, 1360, 1360, 1360, - /* 400 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 410 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 420 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 430 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 440 */ 1360, 1457, 1456, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 450 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 460 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 470 */ 1360, 1360, 1360, 1360, 1360, 1758, 1768, 1360, 1360, 1360, - /* 480 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1654, 1360, 1785, - /* 490 */ 1360, 1744, 1740, 1360, 1360, 1736, 1360, 1360, 1797, 1360, - /* 500 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1730, 1360, - /* 510 */ 1703, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1665, - /* 520 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 530 */ 1360, 1360, 1653, 1360, 1694, 1360, 1360, 1360, 1360, 1360, - /* 540 */ 1360, 1360, 1360, 1520, 1360, 1360, 1360, 1360, 1360, 1360, - /* 550 */ 1360, 1360, 1360, 1360, 1360, 1360, 1505, 1503, 1502, 1501, - /* 560 */ 1360, 1498, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 570 */ 1360, 1360, 1360, 1449, 1360, 1360, 1360, 1360, 1360, 1360, - /* 580 */ 1360, 1360, 1360, 1440, 1360, 1439, 1360, 1360, 1360, 1360, - /* 590 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 600 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, - /* 610 */ 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, 1360, + /* 0 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 10 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 20 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 30 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 40 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 50 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 60 */ 1371, 1371, 1371, 1371, 1371, 1440, 1371, 1371, 1371, 1371, + /* 70 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 80 */ 1438, 1586, 1371, 1750, 1371, 1371, 1371, 1371, 1371, 1371, + /* 90 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 100 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 110 */ 1440, 1371, 1761, 1761, 1761, 1438, 1371, 1371, 1371, 1371, + /* 120 */ 1371, 1371, 1371, 1534, 1371, 1371, 1371, 1371, 1371, 1371, + /* 130 */ 1371, 1371, 1619, 1371, 1371, 1827, 1371, 1371, 1625, 1785, + /* 140 */ 1371, 1371, 1371, 1371, 1487, 1777, 1753, 1767, 1754, 1812, + /* 150 */ 1812, 1812, 1770, 1371, 1781, 1371, 1371, 1371, 1611, 1371, + /* 160 */ 1371, 1591, 1588, 1588, 1371, 1371, 1371, 1371, 1440, 1371, + /* 170 */ 1440, 1371, 1371, 1440, 1371, 1440, 1371, 1371, 1440, 1440, + /* 180 */ 1371, 1440, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 190 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1438, + /* 200 */ 1621, 1371, 1438, 1371, 1371, 1438, 1371, 1371, 1438, 1371, + /* 210 */ 1792, 1790, 1371, 1792, 1790, 1371, 1371, 1371, 1804, 1800, + /* 220 */ 1783, 1781, 1767, 1371, 1371, 1371, 1818, 1814, 1830, 1818, + /* 230 */ 1814, 1818, 1814, 1371, 1790, 1371, 1371, 1790, 1371, 1596, + /* 240 */ 1371, 1371, 1438, 1371, 1438, 1371, 1503, 1371, 1371, 1371, + /* 250 */ 1438, 1371, 1613, 1627, 1537, 1537, 1537, 1441, 1376, 1371, + /* 260 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 270 */ 1499, 1694, 1803, 1802, 1726, 1725, 1724, 1722, 1693, 1371, + /* 280 */ 1371, 1371, 1371, 1371, 1371, 1687, 1688, 1686, 1685, 1371, + /* 290 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 300 */ 1371, 1371, 1751, 1371, 1815, 1819, 1371, 1371, 1371, 1670, + /* 310 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 320 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 330 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 340 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 350 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 360 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 370 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 380 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 390 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1405, 1371, 1371, + /* 400 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 410 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 420 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 430 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 440 */ 1371, 1371, 1371, 1371, 1371, 1468, 1467, 1371, 1371, 1371, + /* 450 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 460 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 470 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1774, + /* 480 */ 1784, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 490 */ 1371, 1670, 1371, 1801, 1371, 1760, 1756, 1371, 1371, 1752, + /* 500 */ 1371, 1371, 1813, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 510 */ 1371, 1371, 1746, 1371, 1719, 1371, 1371, 1371, 1371, 1371, + /* 520 */ 1371, 1371, 1371, 1681, 1371, 1371, 1371, 1371, 1371, 1371, + /* 530 */ 1371, 1371, 1371, 1371, 1371, 1371, 1669, 1371, 1710, 1371, + /* 540 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1531, 1371, 1371, + /* 550 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 560 */ 1516, 1514, 1513, 1512, 1371, 1509, 1371, 1371, 1371, 1371, + /* 570 */ 1540, 1539, 1371, 1371, 1371, 1371, 1371, 1371, 1460, 1371, + /* 580 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1451, 1371, + /* 590 */ 1450, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 600 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 610 */ 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, 1371, + /* 620 */ 1371, 1371, }; /********** End of lemon-generated parsing tables *****************************/ @@ -947,11 +934,13 @@ static const YYCODETYPE yyFallback[] = { 0, /* BLOB => nothing */ 0, /* VARBINARY => nothing */ 0, /* DECIMAL => nothing */ - 0, /* FILE_FACTOR => nothing */ - 0, /* NK_FLOAT => nothing */ + 0, /* MAX_DELAY => nothing */ + 0, /* WATERMARK => nothing */ 0, /* ROLLUP => nothing */ 0, /* TTL => nothing */ 0, /* SMA => nothing */ + 0, /* FIRST => nothing */ + 0, /* LAST => nothing */ 0, /* SHOW => nothing */ 0, /* DATABASES => nothing */ 0, /* TABLES => nothing */ @@ -993,6 +982,7 @@ static const YYCODETYPE yyFallback[] = { 0, /* VERBOSE => nothing */ 0, /* NK_BOOL => nothing */ 0, /* RATIO => nothing */ + 0, /* NK_FLOAT => nothing */ 0, /* COMPACT => nothing */ 0, /* VNODES => nothing */ 0, /* IN => nothing */ @@ -1004,8 +994,6 @@ static const YYCODETYPE yyFallback[] = { 0, /* TRIGGER => nothing */ 0, /* AT_ONCE => nothing */ 0, /* WINDOW_CLOSE => nothing */ - 0, /* MAX_DELAY => nothing */ - 0, /* WATERMARK => nothing */ 0, /* KILL => nothing */ 0, /* CONNECTION => nothing */ 0, /* TRANSACTION => nothing */ @@ -1031,8 +1019,6 @@ static const YYCODETYPE yyFallback[] = { 0, /* TODAY => nothing */ 0, /* TIMEZONE => nothing */ 0, /* COUNT => nothing */ - 0, /* FIRST => nothing */ - 0, /* LAST => nothing */ 0, /* LAST_ROW => nothing */ 0, /* BETWEEN => nothing */ 0, /* IS => nothing */ @@ -1069,12 +1055,12 @@ static const YYCODETYPE yyFallback[] = { 0, /* ASC => nothing */ 0, /* NULLS => nothing */ 0, /* ID => nothing */ - 233, /* NK_BITNOT => ID */ - 233, /* INSERT => ID */ - 233, /* VALUES => ID */ - 233, /* IMPORT => ID */ - 233, /* NK_SEMI => ID */ - 233, /* FILE => ID */ + 232, /* NK_BITNOT => ID */ + 232, /* INSERT => ID */ + 232, /* VALUES => ID */ + 232, /* IMPORT => ID */ + 232, /* NK_SEMI => ID */ + 232, /* FILE => ID */ }; #endif /* YYFALLBACK */ @@ -1274,254 +1260,253 @@ static const char *const yyTokenName[] = { /* 109 */ "BLOB", /* 110 */ "VARBINARY", /* 111 */ "DECIMAL", - /* 112 */ "FILE_FACTOR", - /* 113 */ "NK_FLOAT", + /* 112 */ "MAX_DELAY", + /* 113 */ "WATERMARK", /* 114 */ "ROLLUP", /* 115 */ "TTL", /* 116 */ "SMA", - /* 117 */ "SHOW", - /* 118 */ "DATABASES", - /* 119 */ "TABLES", - /* 120 */ "STABLES", - /* 121 */ "MNODES", - /* 122 */ "MODULES", - /* 123 */ "QNODES", - /* 124 */ "FUNCTIONS", - /* 125 */ "INDEXES", - /* 126 */ "ACCOUNTS", - /* 127 */ "APPS", - /* 128 */ "CONNECTIONS", - /* 129 */ "LICENCE", - /* 130 */ "GRANTS", - /* 131 */ "QUERIES", - /* 132 */ "SCORES", - /* 133 */ "TOPICS", - /* 134 */ "VARIABLES", - /* 135 */ "BNODES", - /* 136 */ "SNODES", - /* 137 */ "CLUSTER", - /* 138 */ "TRANSACTIONS", - /* 139 */ "LIKE", - /* 140 */ "INDEX", - /* 141 */ "FULLTEXT", - /* 142 */ "FUNCTION", - /* 143 */ "INTERVAL", - /* 144 */ "TOPIC", - /* 145 */ "AS", - /* 146 */ "CONSUMER", - /* 147 */ "GROUP", - /* 148 */ "DESC", - /* 149 */ "DESCRIBE", - /* 150 */ "RESET", - /* 151 */ "QUERY", - /* 152 */ "CACHE", - /* 153 */ "EXPLAIN", - /* 154 */ "ANALYZE", - /* 155 */ "VERBOSE", - /* 156 */ "NK_BOOL", - /* 157 */ "RATIO", - /* 158 */ "COMPACT", - /* 159 */ "VNODES", - /* 160 */ "IN", - /* 161 */ "OUTPUTTYPE", - /* 162 */ "AGGREGATE", - /* 163 */ "BUFSIZE", - /* 164 */ "STREAM", - /* 165 */ "INTO", - /* 166 */ "TRIGGER", - /* 167 */ "AT_ONCE", - /* 168 */ "WINDOW_CLOSE", - /* 169 */ "MAX_DELAY", - /* 170 */ "WATERMARK", - /* 171 */ "KILL", - /* 172 */ "CONNECTION", - /* 173 */ "TRANSACTION", - /* 174 */ "BALANCE", - /* 175 */ "VGROUP", - /* 176 */ "MERGE", - /* 177 */ "REDISTRIBUTE", - /* 178 */ "SPLIT", - /* 179 */ "SYNCDB", - /* 180 */ "DELETE", - /* 181 */ "NULL", - /* 182 */ "NK_QUESTION", - /* 183 */ "NK_ARROW", - /* 184 */ "ROWTS", - /* 185 */ "TBNAME", - /* 186 */ "QSTARTTS", - /* 187 */ "QENDTS", - /* 188 */ "WSTARTTS", - /* 189 */ "WENDTS", - /* 190 */ "WDURATION", - /* 191 */ "CAST", - /* 192 */ "NOW", - /* 193 */ "TODAY", - /* 194 */ "TIMEZONE", - /* 195 */ "COUNT", - /* 196 */ "FIRST", - /* 197 */ "LAST", - /* 198 */ "LAST_ROW", - /* 199 */ "BETWEEN", - /* 200 */ "IS", - /* 201 */ "NK_LT", - /* 202 */ "NK_GT", - /* 203 */ "NK_LE", - /* 204 */ "NK_GE", - /* 205 */ "NK_NE", - /* 206 */ "MATCH", - /* 207 */ "NMATCH", - /* 208 */ "CONTAINS", - /* 209 */ "JOIN", - /* 210 */ "INNER", - /* 211 */ "SELECT", - /* 212 */ "DISTINCT", - /* 213 */ "WHERE", - /* 214 */ "PARTITION", - /* 215 */ "BY", - /* 216 */ "SESSION", - /* 217 */ "STATE_WINDOW", - /* 218 */ "SLIDING", - /* 219 */ "FILL", - /* 220 */ "VALUE", - /* 221 */ "NONE", - /* 222 */ "PREV", - /* 223 */ "LINEAR", - /* 224 */ "NEXT", - /* 225 */ "HAVING", - /* 226 */ "ORDER", - /* 227 */ "SLIMIT", - /* 228 */ "SOFFSET", - /* 229 */ "LIMIT", - /* 230 */ "OFFSET", - /* 231 */ "ASC", - /* 232 */ "NULLS", - /* 233 */ "ID", - /* 234 */ "NK_BITNOT", - /* 235 */ "INSERT", - /* 236 */ "VALUES", - /* 237 */ "IMPORT", - /* 238 */ "NK_SEMI", - /* 239 */ "FILE", - /* 240 */ "cmd", - /* 241 */ "account_options", - /* 242 */ "alter_account_options", - /* 243 */ "literal", - /* 244 */ "alter_account_option", - /* 245 */ "user_name", - /* 246 */ "privileges", - /* 247 */ "priv_level", - /* 248 */ "priv_type_list", - /* 249 */ "priv_type", - /* 250 */ "db_name", - /* 251 */ "dnode_endpoint", - /* 252 */ "dnode_host_name", - /* 253 */ "not_exists_opt", - /* 254 */ "db_options", - /* 255 */ "exists_opt", - /* 256 */ "alter_db_options", - /* 257 */ "integer_list", - /* 258 */ "variable_list", - /* 259 */ "retention_list", - /* 260 */ "alter_db_option", - /* 261 */ "retention", - /* 262 */ "full_table_name", - /* 263 */ "column_def_list", - /* 264 */ "tags_def_opt", - /* 265 */ "table_options", - /* 266 */ "multi_create_clause", - /* 267 */ "tags_def", - /* 268 */ "multi_drop_clause", - /* 269 */ "alter_table_clause", - /* 270 */ "alter_table_options", - /* 271 */ "column_name", - /* 272 */ "type_name", - /* 273 */ "signed_literal", - /* 274 */ "create_subtable_clause", - /* 275 */ "specific_tags_opt", - /* 276 */ "literal_list", - /* 277 */ "drop_table_clause", - /* 278 */ "col_name_list", - /* 279 */ "table_name", - /* 280 */ "column_def", - /* 281 */ "func_name_list", - /* 282 */ "alter_table_option", - /* 283 */ "col_name", - /* 284 */ "db_name_cond_opt", - /* 285 */ "like_pattern_opt", - /* 286 */ "table_name_cond", - /* 287 */ "from_db_opt", - /* 288 */ "func_name", - /* 289 */ "function_name", + /* 117 */ "FIRST", + /* 118 */ "LAST", + /* 119 */ "SHOW", + /* 120 */ "DATABASES", + /* 121 */ "TABLES", + /* 122 */ "STABLES", + /* 123 */ "MNODES", + /* 124 */ "MODULES", + /* 125 */ "QNODES", + /* 126 */ "FUNCTIONS", + /* 127 */ "INDEXES", + /* 128 */ "ACCOUNTS", + /* 129 */ "APPS", + /* 130 */ "CONNECTIONS", + /* 131 */ "LICENCE", + /* 132 */ "GRANTS", + /* 133 */ "QUERIES", + /* 134 */ "SCORES", + /* 135 */ "TOPICS", + /* 136 */ "VARIABLES", + /* 137 */ "BNODES", + /* 138 */ "SNODES", + /* 139 */ "CLUSTER", + /* 140 */ "TRANSACTIONS", + /* 141 */ "LIKE", + /* 142 */ "INDEX", + /* 143 */ "FULLTEXT", + /* 144 */ "FUNCTION", + /* 145 */ "INTERVAL", + /* 146 */ "TOPIC", + /* 147 */ "AS", + /* 148 */ "CONSUMER", + /* 149 */ "GROUP", + /* 150 */ "DESC", + /* 151 */ "DESCRIBE", + /* 152 */ "RESET", + /* 153 */ "QUERY", + /* 154 */ "CACHE", + /* 155 */ "EXPLAIN", + /* 156 */ "ANALYZE", + /* 157 */ "VERBOSE", + /* 158 */ "NK_BOOL", + /* 159 */ "RATIO", + /* 160 */ "NK_FLOAT", + /* 161 */ "COMPACT", + /* 162 */ "VNODES", + /* 163 */ "IN", + /* 164 */ "OUTPUTTYPE", + /* 165 */ "AGGREGATE", + /* 166 */ "BUFSIZE", + /* 167 */ "STREAM", + /* 168 */ "INTO", + /* 169 */ "TRIGGER", + /* 170 */ "AT_ONCE", + /* 171 */ "WINDOW_CLOSE", + /* 172 */ "KILL", + /* 173 */ "CONNECTION", + /* 174 */ "TRANSACTION", + /* 175 */ "BALANCE", + /* 176 */ "VGROUP", + /* 177 */ "MERGE", + /* 178 */ "REDISTRIBUTE", + /* 179 */ "SPLIT", + /* 180 */ "SYNCDB", + /* 181 */ "DELETE", + /* 182 */ "NULL", + /* 183 */ "NK_QUESTION", + /* 184 */ "NK_ARROW", + /* 185 */ "ROWTS", + /* 186 */ "TBNAME", + /* 187 */ "QSTARTTS", + /* 188 */ "QENDTS", + /* 189 */ "WSTARTTS", + /* 190 */ "WENDTS", + /* 191 */ "WDURATION", + /* 192 */ "CAST", + /* 193 */ "NOW", + /* 194 */ "TODAY", + /* 195 */ "TIMEZONE", + /* 196 */ "COUNT", + /* 197 */ "LAST_ROW", + /* 198 */ "BETWEEN", + /* 199 */ "IS", + /* 200 */ "NK_LT", + /* 201 */ "NK_GT", + /* 202 */ "NK_LE", + /* 203 */ "NK_GE", + /* 204 */ "NK_NE", + /* 205 */ "MATCH", + /* 206 */ "NMATCH", + /* 207 */ "CONTAINS", + /* 208 */ "JOIN", + /* 209 */ "INNER", + /* 210 */ "SELECT", + /* 211 */ "DISTINCT", + /* 212 */ "WHERE", + /* 213 */ "PARTITION", + /* 214 */ "BY", + /* 215 */ "SESSION", + /* 216 */ "STATE_WINDOW", + /* 217 */ "SLIDING", + /* 218 */ "FILL", + /* 219 */ "VALUE", + /* 220 */ "NONE", + /* 221 */ "PREV", + /* 222 */ "LINEAR", + /* 223 */ "NEXT", + /* 224 */ "HAVING", + /* 225 */ "ORDER", + /* 226 */ "SLIMIT", + /* 227 */ "SOFFSET", + /* 228 */ "LIMIT", + /* 229 */ "OFFSET", + /* 230 */ "ASC", + /* 231 */ "NULLS", + /* 232 */ "ID", + /* 233 */ "NK_BITNOT", + /* 234 */ "INSERT", + /* 235 */ "VALUES", + /* 236 */ "IMPORT", + /* 237 */ "NK_SEMI", + /* 238 */ "FILE", + /* 239 */ "cmd", + /* 240 */ "account_options", + /* 241 */ "alter_account_options", + /* 242 */ "literal", + /* 243 */ "alter_account_option", + /* 244 */ "user_name", + /* 245 */ "privileges", + /* 246 */ "priv_level", + /* 247 */ "priv_type_list", + /* 248 */ "priv_type", + /* 249 */ "db_name", + /* 250 */ "dnode_endpoint", + /* 251 */ "not_exists_opt", + /* 252 */ "db_options", + /* 253 */ "exists_opt", + /* 254 */ "alter_db_options", + /* 255 */ "integer_list", + /* 256 */ "variable_list", + /* 257 */ "retention_list", + /* 258 */ "alter_db_option", + /* 259 */ "retention", + /* 260 */ "full_table_name", + /* 261 */ "column_def_list", + /* 262 */ "tags_def_opt", + /* 263 */ "table_options", + /* 264 */ "multi_create_clause", + /* 265 */ "tags_def", + /* 266 */ "multi_drop_clause", + /* 267 */ "alter_table_clause", + /* 268 */ "alter_table_options", + /* 269 */ "column_name", + /* 270 */ "type_name", + /* 271 */ "signed_literal", + /* 272 */ "create_subtable_clause", + /* 273 */ "specific_tags_opt", + /* 274 */ "literal_list", + /* 275 */ "drop_table_clause", + /* 276 */ "col_name_list", + /* 277 */ "table_name", + /* 278 */ "column_def", + /* 279 */ "duration_list", + /* 280 */ "rollup_func_list", + /* 281 */ "alter_table_option", + /* 282 */ "duration_literal", + /* 283 */ "rollup_func_name", + /* 284 */ "function_name", + /* 285 */ "col_name", + /* 286 */ "db_name_cond_opt", + /* 287 */ "like_pattern_opt", + /* 288 */ "table_name_cond", + /* 289 */ "from_db_opt", /* 290 */ "index_name", /* 291 */ "index_options", /* 292 */ "func_list", - /* 293 */ "duration_literal", - /* 294 */ "sliding_opt", - /* 295 */ "func", - /* 296 */ "expression_list", - /* 297 */ "topic_name", - /* 298 */ "query_expression", - /* 299 */ "cgroup_name", - /* 300 */ "analyze_opt", - /* 301 */ "explain_options", - /* 302 */ "agg_func_opt", - /* 303 */ "bufsize_opt", - /* 304 */ "stream_name", - /* 305 */ "stream_options", - /* 306 */ "into_opt", - /* 307 */ "dnode_list", - /* 308 */ "where_clause_opt", - /* 309 */ "signed", - /* 310 */ "literal_func", - /* 311 */ "table_alias", - /* 312 */ "column_alias", - /* 313 */ "expression", - /* 314 */ "pseudo_column", - /* 315 */ "column_reference", - /* 316 */ "function_expression", - /* 317 */ "subquery", - /* 318 */ "star_func", - /* 319 */ "star_func_para_list", - /* 320 */ "noarg_func", - /* 321 */ "other_para_list", - /* 322 */ "star_func_para", - /* 323 */ "predicate", - /* 324 */ "compare_op", - /* 325 */ "in_op", - /* 326 */ "in_predicate_value", - /* 327 */ "boolean_value_expression", - /* 328 */ "boolean_primary", - /* 329 */ "common_expression", - /* 330 */ "from_clause", - /* 331 */ "table_reference_list", - /* 332 */ "table_reference", - /* 333 */ "table_primary", - /* 334 */ "joined_table", - /* 335 */ "alias_opt", - /* 336 */ "parenthesized_joined_table", - /* 337 */ "join_type", - /* 338 */ "search_condition", - /* 339 */ "query_specification", - /* 340 */ "set_quantifier_opt", - /* 341 */ "select_list", - /* 342 */ "partition_by_clause_opt", - /* 343 */ "twindow_clause_opt", - /* 344 */ "group_by_clause_opt", - /* 345 */ "having_clause_opt", - /* 346 */ "select_sublist", - /* 347 */ "select_item", - /* 348 */ "fill_opt", - /* 349 */ "fill_mode", - /* 350 */ "group_by_list", - /* 351 */ "query_expression_body", - /* 352 */ "order_by_clause_opt", - /* 353 */ "slimit_clause_opt", - /* 354 */ "limit_clause_opt", - /* 355 */ "query_primary", - /* 356 */ "sort_specification_list", - /* 357 */ "sort_specification", - /* 358 */ "ordering_specification_opt", - /* 359 */ "null_ordering_opt", + /* 293 */ "sliding_opt", + /* 294 */ "func", + /* 295 */ "expression_list", + /* 296 */ "topic_name", + /* 297 */ "query_expression", + /* 298 */ "cgroup_name", + /* 299 */ "analyze_opt", + /* 300 */ "explain_options", + /* 301 */ "agg_func_opt", + /* 302 */ "bufsize_opt", + /* 303 */ "stream_name", + /* 304 */ "stream_options", + /* 305 */ "into_opt", + /* 306 */ "dnode_list", + /* 307 */ "where_clause_opt", + /* 308 */ "signed", + /* 309 */ "literal_func", + /* 310 */ "table_alias", + /* 311 */ "column_alias", + /* 312 */ "expression", + /* 313 */ "pseudo_column", + /* 314 */ "column_reference", + /* 315 */ "function_expression", + /* 316 */ "subquery", + /* 317 */ "star_func", + /* 318 */ "star_func_para_list", + /* 319 */ "noarg_func", + /* 320 */ "other_para_list", + /* 321 */ "star_func_para", + /* 322 */ "predicate", + /* 323 */ "compare_op", + /* 324 */ "in_op", + /* 325 */ "in_predicate_value", + /* 326 */ "boolean_value_expression", + /* 327 */ "boolean_primary", + /* 328 */ "common_expression", + /* 329 */ "from_clause", + /* 330 */ "table_reference_list", + /* 331 */ "table_reference", + /* 332 */ "table_primary", + /* 333 */ "joined_table", + /* 334 */ "alias_opt", + /* 335 */ "parenthesized_joined_table", + /* 336 */ "join_type", + /* 337 */ "search_condition", + /* 338 */ "query_specification", + /* 339 */ "set_quantifier_opt", + /* 340 */ "select_list", + /* 341 */ "partition_by_clause_opt", + /* 342 */ "twindow_clause_opt", + /* 343 */ "group_by_clause_opt", + /* 344 */ "having_clause_opt", + /* 345 */ "select_sublist", + /* 346 */ "select_item", + /* 347 */ "fill_opt", + /* 348 */ "fill_mode", + /* 349 */ "group_by_list", + /* 350 */ "query_expression_body", + /* 351 */ "order_by_clause_opt", + /* 352 */ "slimit_clause_opt", + /* 353 */ "limit_clause_opt", + /* 354 */ "query_primary", + /* 355 */ "sort_specification_list", + /* 356 */ "sort_specification", + /* 357 */ "ordering_specification_opt", + /* 358 */ "null_ordering_opt", }; #endif /* defined(YYCOVERAGE) || !defined(NDEBUG) */ @@ -1568,7 +1553,7 @@ static const char *const yyRuleName[] = { /* 36 */ "priv_level ::= NK_STAR NK_DOT NK_STAR", /* 37 */ "priv_level ::= db_name NK_DOT NK_STAR", /* 38 */ "cmd ::= CREATE DNODE dnode_endpoint", - /* 39 */ "cmd ::= CREATE DNODE dnode_host_name PORT NK_INTEGER", + /* 39 */ "cmd ::= CREATE DNODE dnode_endpoint PORT NK_INTEGER", /* 40 */ "cmd ::= DROP DNODE NK_INTEGER", /* 41 */ "cmd ::= DROP DNODE dnode_endpoint", /* 42 */ "cmd ::= ALTER DNODE NK_INTEGER NK_STRING", @@ -1576,8 +1561,8 @@ static const char *const yyRuleName[] = { /* 44 */ "cmd ::= ALTER ALL DNODES NK_STRING", /* 45 */ "cmd ::= ALTER ALL DNODES NK_STRING NK_STRING", /* 46 */ "dnode_endpoint ::= NK_STRING", - /* 47 */ "dnode_host_name ::= NK_ID", - /* 48 */ "dnode_host_name ::= NK_IPTOKEN", + /* 47 */ "dnode_endpoint ::= NK_ID", + /* 48 */ "dnode_endpoint ::= NK_IPTOKEN", /* 49 */ "cmd ::= ALTER LOCAL NK_STRING", /* 50 */ "cmd ::= ALTER LOCAL NK_STRING NK_STRING", /* 51 */ "cmd ::= CREATE QNODE ON DNODE NK_INTEGER", @@ -1694,295 +1679,300 @@ static const char *const yyRuleName[] = { /* 162 */ "tags_def ::= TAGS NK_LP column_def_list NK_RP", /* 163 */ "table_options ::=", /* 164 */ "table_options ::= table_options COMMENT NK_STRING", - /* 165 */ "table_options ::= table_options FILE_FACTOR NK_FLOAT", - /* 166 */ "table_options ::= table_options ROLLUP NK_LP func_name_list NK_RP", - /* 167 */ "table_options ::= table_options TTL NK_INTEGER", - /* 168 */ "table_options ::= table_options SMA NK_LP col_name_list NK_RP", - /* 169 */ "alter_table_options ::= alter_table_option", - /* 170 */ "alter_table_options ::= alter_table_options alter_table_option", - /* 171 */ "alter_table_option ::= COMMENT NK_STRING", - /* 172 */ "alter_table_option ::= TTL NK_INTEGER", - /* 173 */ "col_name_list ::= col_name", - /* 174 */ "col_name_list ::= col_name_list NK_COMMA col_name", - /* 175 */ "col_name ::= column_name", - /* 176 */ "cmd ::= SHOW DNODES", - /* 177 */ "cmd ::= SHOW USERS", - /* 178 */ "cmd ::= SHOW DATABASES", - /* 179 */ "cmd ::= SHOW db_name_cond_opt TABLES like_pattern_opt", - /* 180 */ "cmd ::= SHOW db_name_cond_opt STABLES like_pattern_opt", - /* 181 */ "cmd ::= SHOW db_name_cond_opt VGROUPS", - /* 182 */ "cmd ::= SHOW MNODES", - /* 183 */ "cmd ::= SHOW MODULES", - /* 184 */ "cmd ::= SHOW QNODES", - /* 185 */ "cmd ::= SHOW FUNCTIONS", - /* 186 */ "cmd ::= SHOW INDEXES FROM table_name_cond from_db_opt", - /* 187 */ "cmd ::= SHOW STREAMS", - /* 188 */ "cmd ::= SHOW ACCOUNTS", - /* 189 */ "cmd ::= SHOW APPS", - /* 190 */ "cmd ::= SHOW CONNECTIONS", - /* 191 */ "cmd ::= SHOW LICENCE", - /* 192 */ "cmd ::= SHOW GRANTS", - /* 193 */ "cmd ::= SHOW CREATE DATABASE db_name", - /* 194 */ "cmd ::= SHOW CREATE TABLE full_table_name", - /* 195 */ "cmd ::= SHOW CREATE STABLE full_table_name", - /* 196 */ "cmd ::= SHOW QUERIES", - /* 197 */ "cmd ::= SHOW SCORES", - /* 198 */ "cmd ::= SHOW TOPICS", - /* 199 */ "cmd ::= SHOW VARIABLES", - /* 200 */ "cmd ::= SHOW BNODES", - /* 201 */ "cmd ::= SHOW SNODES", - /* 202 */ "cmd ::= SHOW CLUSTER", - /* 203 */ "cmd ::= SHOW TRANSACTIONS", - /* 204 */ "db_name_cond_opt ::=", - /* 205 */ "db_name_cond_opt ::= db_name NK_DOT", - /* 206 */ "like_pattern_opt ::=", - /* 207 */ "like_pattern_opt ::= LIKE NK_STRING", - /* 208 */ "table_name_cond ::= table_name", - /* 209 */ "from_db_opt ::=", - /* 210 */ "from_db_opt ::= FROM db_name", - /* 211 */ "func_name_list ::= func_name", - /* 212 */ "func_name_list ::= func_name_list NK_COMMA func_name", - /* 213 */ "func_name ::= function_name", - /* 214 */ "cmd ::= CREATE SMA INDEX not_exists_opt index_name ON table_name index_options", - /* 215 */ "cmd ::= CREATE FULLTEXT INDEX not_exists_opt index_name ON table_name NK_LP col_name_list NK_RP", - /* 216 */ "cmd ::= DROP INDEX exists_opt index_name ON table_name", - /* 217 */ "index_options ::=", - /* 218 */ "index_options ::= FUNCTION NK_LP func_list NK_RP INTERVAL NK_LP duration_literal NK_RP sliding_opt", - /* 219 */ "index_options ::= FUNCTION NK_LP func_list NK_RP INTERVAL NK_LP duration_literal NK_COMMA duration_literal NK_RP sliding_opt", - /* 220 */ "func_list ::= func", - /* 221 */ "func_list ::= func_list NK_COMMA func", - /* 222 */ "func ::= function_name NK_LP expression_list NK_RP", - /* 223 */ "cmd ::= CREATE TOPIC not_exists_opt topic_name AS query_expression", - /* 224 */ "cmd ::= CREATE TOPIC not_exists_opt topic_name AS DATABASE db_name", - /* 225 */ "cmd ::= CREATE TOPIC not_exists_opt topic_name AS STABLE full_table_name", - /* 226 */ "cmd ::= DROP TOPIC exists_opt topic_name", - /* 227 */ "cmd ::= DROP CONSUMER GROUP exists_opt cgroup_name ON topic_name", - /* 228 */ "cmd ::= DESC full_table_name", - /* 229 */ "cmd ::= DESCRIBE full_table_name", - /* 230 */ "cmd ::= RESET QUERY CACHE", - /* 231 */ "cmd ::= EXPLAIN analyze_opt explain_options query_expression", - /* 232 */ "analyze_opt ::=", - /* 233 */ "analyze_opt ::= ANALYZE", - /* 234 */ "explain_options ::=", - /* 235 */ "explain_options ::= explain_options VERBOSE NK_BOOL", - /* 236 */ "explain_options ::= explain_options RATIO NK_FLOAT", - /* 237 */ "cmd ::= COMPACT VNODES IN NK_LP integer_list NK_RP", - /* 238 */ "cmd ::= CREATE agg_func_opt FUNCTION not_exists_opt function_name AS NK_STRING OUTPUTTYPE type_name bufsize_opt", - /* 239 */ "cmd ::= DROP FUNCTION exists_opt function_name", - /* 240 */ "agg_func_opt ::=", - /* 241 */ "agg_func_opt ::= AGGREGATE", - /* 242 */ "bufsize_opt ::=", - /* 243 */ "bufsize_opt ::= BUFSIZE NK_INTEGER", - /* 244 */ "cmd ::= CREATE STREAM not_exists_opt stream_name stream_options into_opt AS query_expression", - /* 245 */ "cmd ::= DROP STREAM exists_opt stream_name", - /* 246 */ "into_opt ::=", - /* 247 */ "into_opt ::= INTO full_table_name", - /* 248 */ "stream_options ::=", - /* 249 */ "stream_options ::= stream_options TRIGGER AT_ONCE", - /* 250 */ "stream_options ::= stream_options TRIGGER WINDOW_CLOSE", - /* 251 */ "stream_options ::= stream_options TRIGGER MAX_DELAY duration_literal", - /* 252 */ "stream_options ::= stream_options WATERMARK duration_literal", - /* 253 */ "cmd ::= KILL CONNECTION NK_INTEGER", - /* 254 */ "cmd ::= KILL QUERY NK_STRING", - /* 255 */ "cmd ::= KILL TRANSACTION NK_INTEGER", - /* 256 */ "cmd ::= BALANCE VGROUP", - /* 257 */ "cmd ::= MERGE VGROUP NK_INTEGER NK_INTEGER", - /* 258 */ "cmd ::= REDISTRIBUTE VGROUP NK_INTEGER dnode_list", - /* 259 */ "cmd ::= SPLIT VGROUP NK_INTEGER", - /* 260 */ "dnode_list ::= DNODE NK_INTEGER", - /* 261 */ "dnode_list ::= dnode_list DNODE NK_INTEGER", - /* 262 */ "cmd ::= SYNCDB db_name REPLICA", - /* 263 */ "cmd ::= DELETE FROM full_table_name where_clause_opt", - /* 264 */ "cmd ::= query_expression", - /* 265 */ "literal ::= NK_INTEGER", - /* 266 */ "literal ::= NK_FLOAT", - /* 267 */ "literal ::= NK_STRING", - /* 268 */ "literal ::= NK_BOOL", - /* 269 */ "literal ::= TIMESTAMP NK_STRING", - /* 270 */ "literal ::= duration_literal", - /* 271 */ "literal ::= NULL", - /* 272 */ "literal ::= NK_QUESTION", - /* 273 */ "duration_literal ::= NK_VARIABLE", - /* 274 */ "signed ::= NK_INTEGER", - /* 275 */ "signed ::= NK_PLUS NK_INTEGER", - /* 276 */ "signed ::= NK_MINUS NK_INTEGER", - /* 277 */ "signed ::= NK_FLOAT", - /* 278 */ "signed ::= NK_PLUS NK_FLOAT", - /* 279 */ "signed ::= NK_MINUS NK_FLOAT", - /* 280 */ "signed_literal ::= signed", - /* 281 */ "signed_literal ::= NK_STRING", - /* 282 */ "signed_literal ::= NK_BOOL", - /* 283 */ "signed_literal ::= TIMESTAMP NK_STRING", - /* 284 */ "signed_literal ::= duration_literal", - /* 285 */ "signed_literal ::= NULL", - /* 286 */ "signed_literal ::= literal_func", - /* 287 */ "literal_list ::= signed_literal", - /* 288 */ "literal_list ::= literal_list NK_COMMA signed_literal", - /* 289 */ "db_name ::= NK_ID", - /* 290 */ "table_name ::= NK_ID", - /* 291 */ "column_name ::= NK_ID", - /* 292 */ "function_name ::= NK_ID", - /* 293 */ "table_alias ::= NK_ID", - /* 294 */ "column_alias ::= NK_ID", - /* 295 */ "user_name ::= NK_ID", - /* 296 */ "index_name ::= NK_ID", - /* 297 */ "topic_name ::= NK_ID", - /* 298 */ "stream_name ::= NK_ID", - /* 299 */ "cgroup_name ::= NK_ID", - /* 300 */ "expression ::= literal", - /* 301 */ "expression ::= pseudo_column", - /* 302 */ "expression ::= column_reference", - /* 303 */ "expression ::= function_expression", - /* 304 */ "expression ::= subquery", - /* 305 */ "expression ::= NK_LP expression NK_RP", - /* 306 */ "expression ::= NK_PLUS expression", - /* 307 */ "expression ::= NK_MINUS expression", - /* 308 */ "expression ::= expression NK_PLUS expression", - /* 309 */ "expression ::= expression NK_MINUS expression", - /* 310 */ "expression ::= expression NK_STAR expression", - /* 311 */ "expression ::= expression NK_SLASH expression", - /* 312 */ "expression ::= expression NK_REM expression", - /* 313 */ "expression ::= column_reference NK_ARROW NK_STRING", - /* 314 */ "expression_list ::= expression", - /* 315 */ "expression_list ::= expression_list NK_COMMA expression", - /* 316 */ "column_reference ::= column_name", - /* 317 */ "column_reference ::= table_name NK_DOT column_name", - /* 318 */ "pseudo_column ::= ROWTS", - /* 319 */ "pseudo_column ::= TBNAME", - /* 320 */ "pseudo_column ::= table_name NK_DOT TBNAME", - /* 321 */ "pseudo_column ::= QSTARTTS", - /* 322 */ "pseudo_column ::= QENDTS", - /* 323 */ "pseudo_column ::= WSTARTTS", - /* 324 */ "pseudo_column ::= WENDTS", - /* 325 */ "pseudo_column ::= WDURATION", - /* 326 */ "function_expression ::= function_name NK_LP expression_list NK_RP", - /* 327 */ "function_expression ::= star_func NK_LP star_func_para_list NK_RP", - /* 328 */ "function_expression ::= CAST NK_LP expression AS type_name NK_RP", - /* 329 */ "function_expression ::= literal_func", - /* 330 */ "literal_func ::= noarg_func NK_LP NK_RP", - /* 331 */ "literal_func ::= NOW", - /* 332 */ "noarg_func ::= NOW", - /* 333 */ "noarg_func ::= TODAY", - /* 334 */ "noarg_func ::= TIMEZONE", - /* 335 */ "star_func ::= COUNT", - /* 336 */ "star_func ::= FIRST", - /* 337 */ "star_func ::= LAST", - /* 338 */ "star_func ::= LAST_ROW", - /* 339 */ "star_func_para_list ::= NK_STAR", - /* 340 */ "star_func_para_list ::= other_para_list", - /* 341 */ "other_para_list ::= star_func_para", - /* 342 */ "other_para_list ::= other_para_list NK_COMMA star_func_para", - /* 343 */ "star_func_para ::= expression", - /* 344 */ "star_func_para ::= table_name NK_DOT NK_STAR", - /* 345 */ "predicate ::= expression compare_op expression", - /* 346 */ "predicate ::= expression BETWEEN expression AND expression", - /* 347 */ "predicate ::= expression NOT BETWEEN expression AND expression", - /* 348 */ "predicate ::= expression IS NULL", - /* 349 */ "predicate ::= expression IS NOT NULL", - /* 350 */ "predicate ::= expression in_op in_predicate_value", - /* 351 */ "compare_op ::= NK_LT", - /* 352 */ "compare_op ::= NK_GT", - /* 353 */ "compare_op ::= NK_LE", - /* 354 */ "compare_op ::= NK_GE", - /* 355 */ "compare_op ::= NK_NE", - /* 356 */ "compare_op ::= NK_EQ", - /* 357 */ "compare_op ::= LIKE", - /* 358 */ "compare_op ::= NOT LIKE", - /* 359 */ "compare_op ::= MATCH", - /* 360 */ "compare_op ::= NMATCH", - /* 361 */ "compare_op ::= CONTAINS", - /* 362 */ "in_op ::= IN", - /* 363 */ "in_op ::= NOT IN", - /* 364 */ "in_predicate_value ::= NK_LP expression_list NK_RP", - /* 365 */ "boolean_value_expression ::= boolean_primary", - /* 366 */ "boolean_value_expression ::= NOT boolean_primary", - /* 367 */ "boolean_value_expression ::= boolean_value_expression OR boolean_value_expression", - /* 368 */ "boolean_value_expression ::= boolean_value_expression AND boolean_value_expression", - /* 369 */ "boolean_primary ::= predicate", - /* 370 */ "boolean_primary ::= NK_LP boolean_value_expression NK_RP", - /* 371 */ "common_expression ::= expression", - /* 372 */ "common_expression ::= boolean_value_expression", - /* 373 */ "from_clause ::= FROM table_reference_list", - /* 374 */ "table_reference_list ::= table_reference", - /* 375 */ "table_reference_list ::= table_reference_list NK_COMMA table_reference", - /* 376 */ "table_reference ::= table_primary", - /* 377 */ "table_reference ::= joined_table", - /* 378 */ "table_primary ::= table_name alias_opt", - /* 379 */ "table_primary ::= db_name NK_DOT table_name alias_opt", - /* 380 */ "table_primary ::= subquery alias_opt", - /* 381 */ "table_primary ::= parenthesized_joined_table", - /* 382 */ "alias_opt ::=", - /* 383 */ "alias_opt ::= table_alias", - /* 384 */ "alias_opt ::= AS table_alias", - /* 385 */ "parenthesized_joined_table ::= NK_LP joined_table NK_RP", - /* 386 */ "parenthesized_joined_table ::= NK_LP parenthesized_joined_table NK_RP", - /* 387 */ "joined_table ::= table_reference join_type JOIN table_reference ON search_condition", - /* 388 */ "join_type ::=", - /* 389 */ "join_type ::= INNER", - /* 390 */ "query_specification ::= SELECT set_quantifier_opt select_list from_clause where_clause_opt partition_by_clause_opt twindow_clause_opt group_by_clause_opt having_clause_opt", - /* 391 */ "set_quantifier_opt ::=", - /* 392 */ "set_quantifier_opt ::= DISTINCT", - /* 393 */ "set_quantifier_opt ::= ALL", - /* 394 */ "select_list ::= NK_STAR", - /* 395 */ "select_list ::= select_sublist", - /* 396 */ "select_sublist ::= select_item", - /* 397 */ "select_sublist ::= select_sublist NK_COMMA select_item", - /* 398 */ "select_item ::= common_expression", - /* 399 */ "select_item ::= common_expression column_alias", - /* 400 */ "select_item ::= common_expression AS column_alias", - /* 401 */ "select_item ::= table_name NK_DOT NK_STAR", - /* 402 */ "where_clause_opt ::=", - /* 403 */ "where_clause_opt ::= WHERE search_condition", - /* 404 */ "partition_by_clause_opt ::=", - /* 405 */ "partition_by_clause_opt ::= PARTITION BY expression_list", - /* 406 */ "twindow_clause_opt ::=", - /* 407 */ "twindow_clause_opt ::= SESSION NK_LP column_reference NK_COMMA duration_literal NK_RP", - /* 408 */ "twindow_clause_opt ::= STATE_WINDOW NK_LP expression NK_RP", - /* 409 */ "twindow_clause_opt ::= INTERVAL NK_LP duration_literal NK_RP sliding_opt fill_opt", - /* 410 */ "twindow_clause_opt ::= INTERVAL NK_LP duration_literal NK_COMMA duration_literal NK_RP sliding_opt fill_opt", - /* 411 */ "sliding_opt ::=", - /* 412 */ "sliding_opt ::= SLIDING NK_LP duration_literal NK_RP", - /* 413 */ "fill_opt ::=", - /* 414 */ "fill_opt ::= FILL NK_LP fill_mode NK_RP", - /* 415 */ "fill_opt ::= FILL NK_LP VALUE NK_COMMA literal_list NK_RP", - /* 416 */ "fill_mode ::= NONE", - /* 417 */ "fill_mode ::= PREV", - /* 418 */ "fill_mode ::= NULL", - /* 419 */ "fill_mode ::= LINEAR", - /* 420 */ "fill_mode ::= NEXT", - /* 421 */ "group_by_clause_opt ::=", - /* 422 */ "group_by_clause_opt ::= GROUP BY group_by_list", - /* 423 */ "group_by_list ::= expression", - /* 424 */ "group_by_list ::= group_by_list NK_COMMA expression", - /* 425 */ "having_clause_opt ::=", - /* 426 */ "having_clause_opt ::= HAVING search_condition", - /* 427 */ "query_expression ::= query_expression_body order_by_clause_opt slimit_clause_opt limit_clause_opt", - /* 428 */ "query_expression_body ::= query_primary", - /* 429 */ "query_expression_body ::= query_expression_body UNION ALL query_expression_body", - /* 430 */ "query_expression_body ::= query_expression_body UNION query_expression_body", - /* 431 */ "query_primary ::= query_specification", - /* 432 */ "query_primary ::= NK_LP query_expression_body order_by_clause_opt slimit_clause_opt limit_clause_opt NK_RP", - /* 433 */ "order_by_clause_opt ::=", - /* 434 */ "order_by_clause_opt ::= ORDER BY sort_specification_list", - /* 435 */ "slimit_clause_opt ::=", - /* 436 */ "slimit_clause_opt ::= SLIMIT NK_INTEGER", - /* 437 */ "slimit_clause_opt ::= SLIMIT NK_INTEGER SOFFSET NK_INTEGER", - /* 438 */ "slimit_clause_opt ::= SLIMIT NK_INTEGER NK_COMMA NK_INTEGER", - /* 439 */ "limit_clause_opt ::=", - /* 440 */ "limit_clause_opt ::= LIMIT NK_INTEGER", - /* 441 */ "limit_clause_opt ::= LIMIT NK_INTEGER OFFSET NK_INTEGER", - /* 442 */ "limit_clause_opt ::= LIMIT NK_INTEGER NK_COMMA NK_INTEGER", - /* 443 */ "subquery ::= NK_LP query_expression NK_RP", - /* 444 */ "search_condition ::= common_expression", - /* 445 */ "sort_specification_list ::= sort_specification", - /* 446 */ "sort_specification_list ::= sort_specification_list NK_COMMA sort_specification", - /* 447 */ "sort_specification ::= expression ordering_specification_opt null_ordering_opt", - /* 448 */ "ordering_specification_opt ::=", - /* 449 */ "ordering_specification_opt ::= ASC", - /* 450 */ "ordering_specification_opt ::= DESC", - /* 451 */ "null_ordering_opt ::=", - /* 452 */ "null_ordering_opt ::= NULLS FIRST", - /* 453 */ "null_ordering_opt ::= NULLS LAST", + /* 165 */ "table_options ::= table_options MAX_DELAY duration_list", + /* 166 */ "table_options ::= table_options WATERMARK duration_list", + /* 167 */ "table_options ::= table_options ROLLUP NK_LP rollup_func_list NK_RP", + /* 168 */ "table_options ::= table_options TTL NK_INTEGER", + /* 169 */ "table_options ::= table_options SMA NK_LP col_name_list NK_RP", + /* 170 */ "alter_table_options ::= alter_table_option", + /* 171 */ "alter_table_options ::= alter_table_options alter_table_option", + /* 172 */ "alter_table_option ::= COMMENT NK_STRING", + /* 173 */ "alter_table_option ::= TTL NK_INTEGER", + /* 174 */ "duration_list ::= duration_literal", + /* 175 */ "duration_list ::= duration_list NK_COMMA duration_literal", + /* 176 */ "rollup_func_list ::= rollup_func_name", + /* 177 */ "rollup_func_list ::= rollup_func_list NK_COMMA rollup_func_name", + /* 178 */ "rollup_func_name ::= function_name", + /* 179 */ "rollup_func_name ::= FIRST", + /* 180 */ "rollup_func_name ::= LAST", + /* 181 */ "col_name_list ::= col_name", + /* 182 */ "col_name_list ::= col_name_list NK_COMMA col_name", + /* 183 */ "col_name ::= column_name", + /* 184 */ "cmd ::= SHOW DNODES", + /* 185 */ "cmd ::= SHOW USERS", + /* 186 */ "cmd ::= SHOW DATABASES", + /* 187 */ "cmd ::= SHOW db_name_cond_opt TABLES like_pattern_opt", + /* 188 */ "cmd ::= SHOW db_name_cond_opt STABLES like_pattern_opt", + /* 189 */ "cmd ::= SHOW db_name_cond_opt VGROUPS", + /* 190 */ "cmd ::= SHOW MNODES", + /* 191 */ "cmd ::= SHOW MODULES", + /* 192 */ "cmd ::= SHOW QNODES", + /* 193 */ "cmd ::= SHOW FUNCTIONS", + /* 194 */ "cmd ::= SHOW INDEXES FROM table_name_cond from_db_opt", + /* 195 */ "cmd ::= SHOW STREAMS", + /* 196 */ "cmd ::= SHOW ACCOUNTS", + /* 197 */ "cmd ::= SHOW APPS", + /* 198 */ "cmd ::= SHOW CONNECTIONS", + /* 199 */ "cmd ::= SHOW LICENCE", + /* 200 */ "cmd ::= SHOW GRANTS", + /* 201 */ "cmd ::= SHOW CREATE DATABASE db_name", + /* 202 */ "cmd ::= SHOW CREATE TABLE full_table_name", + /* 203 */ "cmd ::= SHOW CREATE STABLE full_table_name", + /* 204 */ "cmd ::= SHOW QUERIES", + /* 205 */ "cmd ::= SHOW SCORES", + /* 206 */ "cmd ::= SHOW TOPICS", + /* 207 */ "cmd ::= SHOW VARIABLES", + /* 208 */ "cmd ::= SHOW BNODES", + /* 209 */ "cmd ::= SHOW SNODES", + /* 210 */ "cmd ::= SHOW CLUSTER", + /* 211 */ "cmd ::= SHOW TRANSACTIONS", + /* 212 */ "db_name_cond_opt ::=", + /* 213 */ "db_name_cond_opt ::= db_name NK_DOT", + /* 214 */ "like_pattern_opt ::=", + /* 215 */ "like_pattern_opt ::= LIKE NK_STRING", + /* 216 */ "table_name_cond ::= table_name", + /* 217 */ "from_db_opt ::=", + /* 218 */ "from_db_opt ::= FROM db_name", + /* 219 */ "cmd ::= CREATE SMA INDEX not_exists_opt index_name ON table_name index_options", + /* 220 */ "cmd ::= CREATE FULLTEXT INDEX not_exists_opt index_name ON table_name NK_LP col_name_list NK_RP", + /* 221 */ "cmd ::= DROP INDEX exists_opt index_name ON table_name", + /* 222 */ "index_options ::=", + /* 223 */ "index_options ::= FUNCTION NK_LP func_list NK_RP INTERVAL NK_LP duration_literal NK_RP sliding_opt", + /* 224 */ "index_options ::= FUNCTION NK_LP func_list NK_RP INTERVAL NK_LP duration_literal NK_COMMA duration_literal NK_RP sliding_opt", + /* 225 */ "func_list ::= func", + /* 226 */ "func_list ::= func_list NK_COMMA func", + /* 227 */ "func ::= function_name NK_LP expression_list NK_RP", + /* 228 */ "cmd ::= CREATE TOPIC not_exists_opt topic_name AS query_expression", + /* 229 */ "cmd ::= CREATE TOPIC not_exists_opt topic_name AS DATABASE db_name", + /* 230 */ "cmd ::= CREATE TOPIC not_exists_opt topic_name AS STABLE full_table_name", + /* 231 */ "cmd ::= DROP TOPIC exists_opt topic_name", + /* 232 */ "cmd ::= DROP CONSUMER GROUP exists_opt cgroup_name ON topic_name", + /* 233 */ "cmd ::= DESC full_table_name", + /* 234 */ "cmd ::= DESCRIBE full_table_name", + /* 235 */ "cmd ::= RESET QUERY CACHE", + /* 236 */ "cmd ::= EXPLAIN analyze_opt explain_options query_expression", + /* 237 */ "analyze_opt ::=", + /* 238 */ "analyze_opt ::= ANALYZE", + /* 239 */ "explain_options ::=", + /* 240 */ "explain_options ::= explain_options VERBOSE NK_BOOL", + /* 241 */ "explain_options ::= explain_options RATIO NK_FLOAT", + /* 242 */ "cmd ::= COMPACT VNODES IN NK_LP integer_list NK_RP", + /* 243 */ "cmd ::= CREATE agg_func_opt FUNCTION not_exists_opt function_name AS NK_STRING OUTPUTTYPE type_name bufsize_opt", + /* 244 */ "cmd ::= DROP FUNCTION exists_opt function_name", + /* 245 */ "agg_func_opt ::=", + /* 246 */ "agg_func_opt ::= AGGREGATE", + /* 247 */ "bufsize_opt ::=", + /* 248 */ "bufsize_opt ::= BUFSIZE NK_INTEGER", + /* 249 */ "cmd ::= CREATE STREAM not_exists_opt stream_name stream_options into_opt AS query_expression", + /* 250 */ "cmd ::= DROP STREAM exists_opt stream_name", + /* 251 */ "into_opt ::=", + /* 252 */ "into_opt ::= INTO full_table_name", + /* 253 */ "stream_options ::=", + /* 254 */ "stream_options ::= stream_options TRIGGER AT_ONCE", + /* 255 */ "stream_options ::= stream_options TRIGGER WINDOW_CLOSE", + /* 256 */ "stream_options ::= stream_options TRIGGER MAX_DELAY duration_literal", + /* 257 */ "stream_options ::= stream_options WATERMARK duration_literal", + /* 258 */ "cmd ::= KILL CONNECTION NK_INTEGER", + /* 259 */ "cmd ::= KILL QUERY NK_STRING", + /* 260 */ "cmd ::= KILL TRANSACTION NK_INTEGER", + /* 261 */ "cmd ::= BALANCE VGROUP", + /* 262 */ "cmd ::= MERGE VGROUP NK_INTEGER NK_INTEGER", + /* 263 */ "cmd ::= REDISTRIBUTE VGROUP NK_INTEGER dnode_list", + /* 264 */ "cmd ::= SPLIT VGROUP NK_INTEGER", + /* 265 */ "dnode_list ::= DNODE NK_INTEGER", + /* 266 */ "dnode_list ::= dnode_list DNODE NK_INTEGER", + /* 267 */ "cmd ::= SYNCDB db_name REPLICA", + /* 268 */ "cmd ::= DELETE FROM full_table_name where_clause_opt", + /* 269 */ "cmd ::= query_expression", + /* 270 */ "literal ::= NK_INTEGER", + /* 271 */ "literal ::= NK_FLOAT", + /* 272 */ "literal ::= NK_STRING", + /* 273 */ "literal ::= NK_BOOL", + /* 274 */ "literal ::= TIMESTAMP NK_STRING", + /* 275 */ "literal ::= duration_literal", + /* 276 */ "literal ::= NULL", + /* 277 */ "literal ::= NK_QUESTION", + /* 278 */ "duration_literal ::= NK_VARIABLE", + /* 279 */ "signed ::= NK_INTEGER", + /* 280 */ "signed ::= NK_PLUS NK_INTEGER", + /* 281 */ "signed ::= NK_MINUS NK_INTEGER", + /* 282 */ "signed ::= NK_FLOAT", + /* 283 */ "signed ::= NK_PLUS NK_FLOAT", + /* 284 */ "signed ::= NK_MINUS NK_FLOAT", + /* 285 */ "signed_literal ::= signed", + /* 286 */ "signed_literal ::= NK_STRING", + /* 287 */ "signed_literal ::= NK_BOOL", + /* 288 */ "signed_literal ::= TIMESTAMP NK_STRING", + /* 289 */ "signed_literal ::= duration_literal", + /* 290 */ "signed_literal ::= NULL", + /* 291 */ "signed_literal ::= literal_func", + /* 292 */ "literal_list ::= signed_literal", + /* 293 */ "literal_list ::= literal_list NK_COMMA signed_literal", + /* 294 */ "db_name ::= NK_ID", + /* 295 */ "table_name ::= NK_ID", + /* 296 */ "column_name ::= NK_ID", + /* 297 */ "function_name ::= NK_ID", + /* 298 */ "table_alias ::= NK_ID", + /* 299 */ "column_alias ::= NK_ID", + /* 300 */ "user_name ::= NK_ID", + /* 301 */ "index_name ::= NK_ID", + /* 302 */ "topic_name ::= NK_ID", + /* 303 */ "stream_name ::= NK_ID", + /* 304 */ "cgroup_name ::= NK_ID", + /* 305 */ "expression ::= literal", + /* 306 */ "expression ::= pseudo_column", + /* 307 */ "expression ::= column_reference", + /* 308 */ "expression ::= function_expression", + /* 309 */ "expression ::= subquery", + /* 310 */ "expression ::= NK_LP expression NK_RP", + /* 311 */ "expression ::= NK_PLUS expression", + /* 312 */ "expression ::= NK_MINUS expression", + /* 313 */ "expression ::= expression NK_PLUS expression", + /* 314 */ "expression ::= expression NK_MINUS expression", + /* 315 */ "expression ::= expression NK_STAR expression", + /* 316 */ "expression ::= expression NK_SLASH expression", + /* 317 */ "expression ::= expression NK_REM expression", + /* 318 */ "expression ::= column_reference NK_ARROW NK_STRING", + /* 319 */ "expression_list ::= expression", + /* 320 */ "expression_list ::= expression_list NK_COMMA expression", + /* 321 */ "column_reference ::= column_name", + /* 322 */ "column_reference ::= table_name NK_DOT column_name", + /* 323 */ "pseudo_column ::= ROWTS", + /* 324 */ "pseudo_column ::= TBNAME", + /* 325 */ "pseudo_column ::= table_name NK_DOT TBNAME", + /* 326 */ "pseudo_column ::= QSTARTTS", + /* 327 */ "pseudo_column ::= QENDTS", + /* 328 */ "pseudo_column ::= WSTARTTS", + /* 329 */ "pseudo_column ::= WENDTS", + /* 330 */ "pseudo_column ::= WDURATION", + /* 331 */ "function_expression ::= function_name NK_LP expression_list NK_RP", + /* 332 */ "function_expression ::= star_func NK_LP star_func_para_list NK_RP", + /* 333 */ "function_expression ::= CAST NK_LP expression AS type_name NK_RP", + /* 334 */ "function_expression ::= literal_func", + /* 335 */ "literal_func ::= noarg_func NK_LP NK_RP", + /* 336 */ "literal_func ::= NOW", + /* 337 */ "noarg_func ::= NOW", + /* 338 */ "noarg_func ::= TODAY", + /* 339 */ "noarg_func ::= TIMEZONE", + /* 340 */ "star_func ::= COUNT", + /* 341 */ "star_func ::= FIRST", + /* 342 */ "star_func ::= LAST", + /* 343 */ "star_func ::= LAST_ROW", + /* 344 */ "star_func_para_list ::= NK_STAR", + /* 345 */ "star_func_para_list ::= other_para_list", + /* 346 */ "other_para_list ::= star_func_para", + /* 347 */ "other_para_list ::= other_para_list NK_COMMA star_func_para", + /* 348 */ "star_func_para ::= expression", + /* 349 */ "star_func_para ::= table_name NK_DOT NK_STAR", + /* 350 */ "predicate ::= expression compare_op expression", + /* 351 */ "predicate ::= expression BETWEEN expression AND expression", + /* 352 */ "predicate ::= expression NOT BETWEEN expression AND expression", + /* 353 */ "predicate ::= expression IS NULL", + /* 354 */ "predicate ::= expression IS NOT NULL", + /* 355 */ "predicate ::= expression in_op in_predicate_value", + /* 356 */ "compare_op ::= NK_LT", + /* 357 */ "compare_op ::= NK_GT", + /* 358 */ "compare_op ::= NK_LE", + /* 359 */ "compare_op ::= NK_GE", + /* 360 */ "compare_op ::= NK_NE", + /* 361 */ "compare_op ::= NK_EQ", + /* 362 */ "compare_op ::= LIKE", + /* 363 */ "compare_op ::= NOT LIKE", + /* 364 */ "compare_op ::= MATCH", + /* 365 */ "compare_op ::= NMATCH", + /* 366 */ "compare_op ::= CONTAINS", + /* 367 */ "in_op ::= IN", + /* 368 */ "in_op ::= NOT IN", + /* 369 */ "in_predicate_value ::= NK_LP expression_list NK_RP", + /* 370 */ "boolean_value_expression ::= boolean_primary", + /* 371 */ "boolean_value_expression ::= NOT boolean_primary", + /* 372 */ "boolean_value_expression ::= boolean_value_expression OR boolean_value_expression", + /* 373 */ "boolean_value_expression ::= boolean_value_expression AND boolean_value_expression", + /* 374 */ "boolean_primary ::= predicate", + /* 375 */ "boolean_primary ::= NK_LP boolean_value_expression NK_RP", + /* 376 */ "common_expression ::= expression", + /* 377 */ "common_expression ::= boolean_value_expression", + /* 378 */ "from_clause ::= FROM table_reference_list", + /* 379 */ "table_reference_list ::= table_reference", + /* 380 */ "table_reference_list ::= table_reference_list NK_COMMA table_reference", + /* 381 */ "table_reference ::= table_primary", + /* 382 */ "table_reference ::= joined_table", + /* 383 */ "table_primary ::= table_name alias_opt", + /* 384 */ "table_primary ::= db_name NK_DOT table_name alias_opt", + /* 385 */ "table_primary ::= subquery alias_opt", + /* 386 */ "table_primary ::= parenthesized_joined_table", + /* 387 */ "alias_opt ::=", + /* 388 */ "alias_opt ::= table_alias", + /* 389 */ "alias_opt ::= AS table_alias", + /* 390 */ "parenthesized_joined_table ::= NK_LP joined_table NK_RP", + /* 391 */ "parenthesized_joined_table ::= NK_LP parenthesized_joined_table NK_RP", + /* 392 */ "joined_table ::= table_reference join_type JOIN table_reference ON search_condition", + /* 393 */ "join_type ::=", + /* 394 */ "join_type ::= INNER", + /* 395 */ "query_specification ::= SELECT set_quantifier_opt select_list from_clause where_clause_opt partition_by_clause_opt twindow_clause_opt group_by_clause_opt having_clause_opt", + /* 396 */ "set_quantifier_opt ::=", + /* 397 */ "set_quantifier_opt ::= DISTINCT", + /* 398 */ "set_quantifier_opt ::= ALL", + /* 399 */ "select_list ::= NK_STAR", + /* 400 */ "select_list ::= select_sublist", + /* 401 */ "select_sublist ::= select_item", + /* 402 */ "select_sublist ::= select_sublist NK_COMMA select_item", + /* 403 */ "select_item ::= common_expression", + /* 404 */ "select_item ::= common_expression column_alias", + /* 405 */ "select_item ::= common_expression AS column_alias", + /* 406 */ "select_item ::= table_name NK_DOT NK_STAR", + /* 407 */ "where_clause_opt ::=", + /* 408 */ "where_clause_opt ::= WHERE search_condition", + /* 409 */ "partition_by_clause_opt ::=", + /* 410 */ "partition_by_clause_opt ::= PARTITION BY expression_list", + /* 411 */ "twindow_clause_opt ::=", + /* 412 */ "twindow_clause_opt ::= SESSION NK_LP column_reference NK_COMMA duration_literal NK_RP", + /* 413 */ "twindow_clause_opt ::= STATE_WINDOW NK_LP expression NK_RP", + /* 414 */ "twindow_clause_opt ::= INTERVAL NK_LP duration_literal NK_RP sliding_opt fill_opt", + /* 415 */ "twindow_clause_opt ::= INTERVAL NK_LP duration_literal NK_COMMA duration_literal NK_RP sliding_opt fill_opt", + /* 416 */ "sliding_opt ::=", + /* 417 */ "sliding_opt ::= SLIDING NK_LP duration_literal NK_RP", + /* 418 */ "fill_opt ::=", + /* 419 */ "fill_opt ::= FILL NK_LP fill_mode NK_RP", + /* 420 */ "fill_opt ::= FILL NK_LP VALUE NK_COMMA literal_list NK_RP", + /* 421 */ "fill_mode ::= NONE", + /* 422 */ "fill_mode ::= PREV", + /* 423 */ "fill_mode ::= NULL", + /* 424 */ "fill_mode ::= LINEAR", + /* 425 */ "fill_mode ::= NEXT", + /* 426 */ "group_by_clause_opt ::=", + /* 427 */ "group_by_clause_opt ::= GROUP BY group_by_list", + /* 428 */ "group_by_list ::= expression", + /* 429 */ "group_by_list ::= group_by_list NK_COMMA expression", + /* 430 */ "having_clause_opt ::=", + /* 431 */ "having_clause_opt ::= HAVING search_condition", + /* 432 */ "query_expression ::= query_expression_body order_by_clause_opt slimit_clause_opt limit_clause_opt", + /* 433 */ "query_expression_body ::= query_primary", + /* 434 */ "query_expression_body ::= query_expression_body UNION ALL query_expression_body", + /* 435 */ "query_expression_body ::= query_expression_body UNION query_expression_body", + /* 436 */ "query_primary ::= query_specification", + /* 437 */ "query_primary ::= NK_LP query_expression_body order_by_clause_opt slimit_clause_opt limit_clause_opt NK_RP", + /* 438 */ "order_by_clause_opt ::=", + /* 439 */ "order_by_clause_opt ::= ORDER BY sort_specification_list", + /* 440 */ "slimit_clause_opt ::=", + /* 441 */ "slimit_clause_opt ::= SLIMIT NK_INTEGER", + /* 442 */ "slimit_clause_opt ::= SLIMIT NK_INTEGER SOFFSET NK_INTEGER", + /* 443 */ "slimit_clause_opt ::= SLIMIT NK_INTEGER NK_COMMA NK_INTEGER", + /* 444 */ "limit_clause_opt ::=", + /* 445 */ "limit_clause_opt ::= LIMIT NK_INTEGER", + /* 446 */ "limit_clause_opt ::= LIMIT NK_INTEGER OFFSET NK_INTEGER", + /* 447 */ "limit_clause_opt ::= LIMIT NK_INTEGER NK_COMMA NK_INTEGER", + /* 448 */ "subquery ::= NK_LP query_expression NK_RP", + /* 449 */ "search_condition ::= common_expression", + /* 450 */ "sort_specification_list ::= sort_specification", + /* 451 */ "sort_specification_list ::= sort_specification_list NK_COMMA sort_specification", + /* 452 */ "sort_specification ::= expression ordering_specification_opt null_ordering_opt", + /* 453 */ "ordering_specification_opt ::=", + /* 454 */ "ordering_specification_opt ::= ASC", + /* 455 */ "ordering_specification_opt ::= DESC", + /* 456 */ "null_ordering_opt ::=", + /* 457 */ "null_ordering_opt ::= NULLS FIRST", + /* 458 */ "null_ordering_opt ::= NULLS LAST", }; #endif /* NDEBUG */ @@ -2109,174 +2099,174 @@ static void yy_destructor( */ /********* Begin destructor definitions ***************************************/ /* Default NON-TERMINAL Destructor */ - case 240: /* cmd */ - case 243: /* literal */ - case 254: /* db_options */ - case 256: /* alter_db_options */ - case 261: /* retention */ - case 262: /* full_table_name */ - case 265: /* table_options */ - case 269: /* alter_table_clause */ - case 270: /* alter_table_options */ - case 273: /* signed_literal */ - case 274: /* create_subtable_clause */ - case 277: /* drop_table_clause */ - case 280: /* column_def */ - case 283: /* col_name */ - case 284: /* db_name_cond_opt */ - case 285: /* like_pattern_opt */ - case 286: /* table_name_cond */ - case 287: /* from_db_opt */ - case 288: /* func_name */ + case 239: /* cmd */ + case 242: /* literal */ + case 252: /* db_options */ + case 254: /* alter_db_options */ + case 259: /* retention */ + case 260: /* full_table_name */ + case 263: /* table_options */ + case 267: /* alter_table_clause */ + case 268: /* alter_table_options */ + case 271: /* signed_literal */ + case 272: /* create_subtable_clause */ + case 275: /* drop_table_clause */ + case 278: /* column_def */ + case 282: /* duration_literal */ + case 283: /* rollup_func_name */ + case 285: /* col_name */ + case 286: /* db_name_cond_opt */ + case 287: /* like_pattern_opt */ + case 288: /* table_name_cond */ + case 289: /* from_db_opt */ case 291: /* index_options */ - case 293: /* duration_literal */ - case 294: /* sliding_opt */ - case 295: /* func */ - case 298: /* query_expression */ - case 301: /* explain_options */ - case 305: /* stream_options */ - case 306: /* into_opt */ - case 308: /* where_clause_opt */ - case 309: /* signed */ - case 310: /* literal_func */ - case 313: /* expression */ - case 314: /* pseudo_column */ - case 315: /* column_reference */ - case 316: /* function_expression */ - case 317: /* subquery */ - case 322: /* star_func_para */ - case 323: /* predicate */ - case 326: /* in_predicate_value */ - case 327: /* boolean_value_expression */ - case 328: /* boolean_primary */ - case 329: /* common_expression */ - case 330: /* from_clause */ - case 331: /* table_reference_list */ - case 332: /* table_reference */ - case 333: /* table_primary */ - case 334: /* joined_table */ - case 336: /* parenthesized_joined_table */ - case 338: /* search_condition */ - case 339: /* query_specification */ - case 343: /* twindow_clause_opt */ - case 345: /* having_clause_opt */ - case 347: /* select_item */ - case 348: /* fill_opt */ - case 351: /* query_expression_body */ - case 353: /* slimit_clause_opt */ - case 354: /* limit_clause_opt */ - case 355: /* query_primary */ - case 357: /* sort_specification */ + case 293: /* sliding_opt */ + case 294: /* func */ + case 297: /* query_expression */ + case 300: /* explain_options */ + case 304: /* stream_options */ + case 305: /* into_opt */ + case 307: /* where_clause_opt */ + case 308: /* signed */ + case 309: /* literal_func */ + case 312: /* expression */ + case 313: /* pseudo_column */ + case 314: /* column_reference */ + case 315: /* function_expression */ + case 316: /* subquery */ + case 321: /* star_func_para */ + case 322: /* predicate */ + case 325: /* in_predicate_value */ + case 326: /* boolean_value_expression */ + case 327: /* boolean_primary */ + case 328: /* common_expression */ + case 329: /* from_clause */ + case 330: /* table_reference_list */ + case 331: /* table_reference */ + case 332: /* table_primary */ + case 333: /* joined_table */ + case 335: /* parenthesized_joined_table */ + case 337: /* search_condition */ + case 338: /* query_specification */ + case 342: /* twindow_clause_opt */ + case 344: /* having_clause_opt */ + case 346: /* select_item */ + case 347: /* fill_opt */ + case 350: /* query_expression_body */ + case 352: /* slimit_clause_opt */ + case 353: /* limit_clause_opt */ + case 354: /* query_primary */ + case 356: /* sort_specification */ { - nodesDestroyNode((yypminor->yy632)); + nodesDestroyNode((yypminor->yy674)); } break; - case 241: /* account_options */ - case 242: /* alter_account_options */ - case 244: /* alter_account_option */ - case 303: /* bufsize_opt */ + case 240: /* account_options */ + case 241: /* alter_account_options */ + case 243: /* alter_account_option */ + case 302: /* bufsize_opt */ { } break; - case 245: /* user_name */ - case 247: /* priv_level */ - case 250: /* db_name */ - case 251: /* dnode_endpoint */ - case 252: /* dnode_host_name */ - case 271: /* column_name */ - case 279: /* table_name */ - case 289: /* function_name */ + case 244: /* user_name */ + case 246: /* priv_level */ + case 249: /* db_name */ + case 250: /* dnode_endpoint */ + case 269: /* column_name */ + case 277: /* table_name */ + case 284: /* function_name */ case 290: /* index_name */ - case 297: /* topic_name */ - case 299: /* cgroup_name */ - case 304: /* stream_name */ - case 311: /* table_alias */ - case 312: /* column_alias */ - case 318: /* star_func */ - case 320: /* noarg_func */ - case 335: /* alias_opt */ + case 296: /* topic_name */ + case 298: /* cgroup_name */ + case 303: /* stream_name */ + case 310: /* table_alias */ + case 311: /* column_alias */ + case 317: /* star_func */ + case 319: /* noarg_func */ + case 334: /* alias_opt */ { } break; - case 246: /* privileges */ - case 248: /* priv_type_list */ - case 249: /* priv_type */ + case 245: /* privileges */ + case 247: /* priv_type_list */ + case 248: /* priv_type */ { } break; - case 253: /* not_exists_opt */ - case 255: /* exists_opt */ - case 300: /* analyze_opt */ - case 302: /* agg_func_opt */ - case 340: /* set_quantifier_opt */ + case 251: /* not_exists_opt */ + case 253: /* exists_opt */ + case 299: /* analyze_opt */ + case 301: /* agg_func_opt */ + case 339: /* set_quantifier_opt */ { } break; - case 257: /* integer_list */ - case 258: /* variable_list */ - case 259: /* retention_list */ - case 263: /* column_def_list */ - case 264: /* tags_def_opt */ - case 266: /* multi_create_clause */ - case 267: /* tags_def */ - case 268: /* multi_drop_clause */ - case 275: /* specific_tags_opt */ - case 276: /* literal_list */ - case 278: /* col_name_list */ - case 281: /* func_name_list */ + case 255: /* integer_list */ + case 256: /* variable_list */ + case 257: /* retention_list */ + case 261: /* column_def_list */ + case 262: /* tags_def_opt */ + case 264: /* multi_create_clause */ + case 265: /* tags_def */ + case 266: /* multi_drop_clause */ + case 273: /* specific_tags_opt */ + case 274: /* literal_list */ + case 276: /* col_name_list */ + case 279: /* duration_list */ + case 280: /* rollup_func_list */ case 292: /* func_list */ - case 296: /* expression_list */ - case 307: /* dnode_list */ - case 319: /* star_func_para_list */ - case 321: /* other_para_list */ - case 341: /* select_list */ - case 342: /* partition_by_clause_opt */ - case 344: /* group_by_clause_opt */ - case 346: /* select_sublist */ - case 350: /* group_by_list */ - case 352: /* order_by_clause_opt */ - case 356: /* sort_specification_list */ + case 295: /* expression_list */ + case 306: /* dnode_list */ + case 318: /* star_func_para_list */ + case 320: /* other_para_list */ + case 340: /* select_list */ + case 341: /* partition_by_clause_opt */ + case 343: /* group_by_clause_opt */ + case 345: /* select_sublist */ + case 349: /* group_by_list */ + case 351: /* order_by_clause_opt */ + case 355: /* sort_specification_list */ { - nodesDestroyList((yypminor->yy424)); + nodesDestroyList((yypminor->yy530)); } break; - case 260: /* alter_db_option */ - case 282: /* alter_table_option */ + case 258: /* alter_db_option */ + case 281: /* alter_table_option */ { } break; - case 272: /* type_name */ + case 270: /* type_name */ { } break; - case 324: /* compare_op */ - case 325: /* in_op */ + case 323: /* compare_op */ + case 324: /* in_op */ { } break; - case 337: /* join_type */ + case 336: /* join_type */ { } break; - case 349: /* fill_mode */ + case 348: /* fill_mode */ { } break; - case 358: /* ordering_specification_opt */ + case 357: /* ordering_specification_opt */ { } break; - case 359: /* null_ordering_opt */ + case 358: /* null_ordering_opt */ { } @@ -2575,460 +2565,465 @@ static const struct { YYCODETYPE lhs; /* Symbol on the left-hand side of the rule */ signed char nrhs; /* Negative of the number of RHS symbols in the rule */ } yyRuleInfo[] = { - { 240, -6 }, /* (0) cmd ::= CREATE ACCOUNT NK_ID PASS NK_STRING account_options */ - { 240, -4 }, /* (1) cmd ::= ALTER ACCOUNT NK_ID alter_account_options */ - { 241, 0 }, /* (2) account_options ::= */ - { 241, -3 }, /* (3) account_options ::= account_options PPS literal */ - { 241, -3 }, /* (4) account_options ::= account_options TSERIES literal */ - { 241, -3 }, /* (5) account_options ::= account_options STORAGE literal */ - { 241, -3 }, /* (6) account_options ::= account_options STREAMS literal */ - { 241, -3 }, /* (7) account_options ::= account_options QTIME literal */ - { 241, -3 }, /* (8) account_options ::= account_options DBS literal */ - { 241, -3 }, /* (9) account_options ::= account_options USERS literal */ - { 241, -3 }, /* (10) account_options ::= account_options CONNS literal */ - { 241, -3 }, /* (11) account_options ::= account_options STATE literal */ - { 242, -1 }, /* (12) alter_account_options ::= alter_account_option */ - { 242, -2 }, /* (13) alter_account_options ::= alter_account_options alter_account_option */ - { 244, -2 }, /* (14) alter_account_option ::= PASS literal */ - { 244, -2 }, /* (15) alter_account_option ::= PPS literal */ - { 244, -2 }, /* (16) alter_account_option ::= TSERIES literal */ - { 244, -2 }, /* (17) alter_account_option ::= STORAGE literal */ - { 244, -2 }, /* (18) alter_account_option ::= STREAMS literal */ - { 244, -2 }, /* (19) alter_account_option ::= QTIME literal */ - { 244, -2 }, /* (20) alter_account_option ::= DBS literal */ - { 244, -2 }, /* (21) alter_account_option ::= USERS literal */ - { 244, -2 }, /* (22) alter_account_option ::= CONNS literal */ - { 244, -2 }, /* (23) alter_account_option ::= STATE literal */ - { 240, -5 }, /* (24) cmd ::= CREATE USER user_name PASS NK_STRING */ - { 240, -5 }, /* (25) cmd ::= ALTER USER user_name PASS NK_STRING */ - { 240, -5 }, /* (26) cmd ::= ALTER USER user_name PRIVILEGE NK_STRING */ - { 240, -3 }, /* (27) cmd ::= DROP USER user_name */ - { 240, -6 }, /* (28) cmd ::= GRANT privileges ON priv_level TO user_name */ - { 240, -6 }, /* (29) cmd ::= REVOKE privileges ON priv_level FROM user_name */ - { 246, -1 }, /* (30) privileges ::= ALL */ - { 246, -1 }, /* (31) privileges ::= priv_type_list */ - { 248, -1 }, /* (32) priv_type_list ::= priv_type */ - { 248, -3 }, /* (33) priv_type_list ::= priv_type_list NK_COMMA priv_type */ - { 249, -1 }, /* (34) priv_type ::= READ */ - { 249, -1 }, /* (35) priv_type ::= WRITE */ - { 247, -3 }, /* (36) priv_level ::= NK_STAR NK_DOT NK_STAR */ - { 247, -3 }, /* (37) priv_level ::= db_name NK_DOT NK_STAR */ - { 240, -3 }, /* (38) cmd ::= CREATE DNODE dnode_endpoint */ - { 240, -5 }, /* (39) cmd ::= CREATE DNODE dnode_host_name PORT NK_INTEGER */ - { 240, -3 }, /* (40) cmd ::= DROP DNODE NK_INTEGER */ - { 240, -3 }, /* (41) cmd ::= DROP DNODE dnode_endpoint */ - { 240, -4 }, /* (42) cmd ::= ALTER DNODE NK_INTEGER NK_STRING */ - { 240, -5 }, /* (43) cmd ::= ALTER DNODE NK_INTEGER NK_STRING NK_STRING */ - { 240, -4 }, /* (44) cmd ::= ALTER ALL DNODES NK_STRING */ - { 240, -5 }, /* (45) cmd ::= ALTER ALL DNODES NK_STRING NK_STRING */ - { 251, -1 }, /* (46) dnode_endpoint ::= NK_STRING */ - { 252, -1 }, /* (47) dnode_host_name ::= NK_ID */ - { 252, -1 }, /* (48) dnode_host_name ::= NK_IPTOKEN */ - { 240, -3 }, /* (49) cmd ::= ALTER LOCAL NK_STRING */ - { 240, -4 }, /* (50) cmd ::= ALTER LOCAL NK_STRING NK_STRING */ - { 240, -5 }, /* (51) cmd ::= CREATE QNODE ON DNODE NK_INTEGER */ - { 240, -5 }, /* (52) cmd ::= DROP QNODE ON DNODE NK_INTEGER */ - { 240, -5 }, /* (53) cmd ::= CREATE BNODE ON DNODE NK_INTEGER */ - { 240, -5 }, /* (54) cmd ::= DROP BNODE ON DNODE NK_INTEGER */ - { 240, -5 }, /* (55) cmd ::= CREATE SNODE ON DNODE NK_INTEGER */ - { 240, -5 }, /* (56) cmd ::= DROP SNODE ON DNODE NK_INTEGER */ - { 240, -5 }, /* (57) cmd ::= CREATE MNODE ON DNODE NK_INTEGER */ - { 240, -5 }, /* (58) cmd ::= DROP MNODE ON DNODE NK_INTEGER */ - { 240, -5 }, /* (59) cmd ::= CREATE DATABASE not_exists_opt db_name db_options */ - { 240, -4 }, /* (60) cmd ::= DROP DATABASE exists_opt db_name */ - { 240, -2 }, /* (61) cmd ::= USE db_name */ - { 240, -4 }, /* (62) cmd ::= ALTER DATABASE db_name alter_db_options */ - { 253, -3 }, /* (63) not_exists_opt ::= IF NOT EXISTS */ - { 253, 0 }, /* (64) not_exists_opt ::= */ - { 255, -2 }, /* (65) exists_opt ::= IF EXISTS */ - { 255, 0 }, /* (66) exists_opt ::= */ - { 254, 0 }, /* (67) db_options ::= */ - { 254, -3 }, /* (68) db_options ::= db_options BUFFER NK_INTEGER */ - { 254, -3 }, /* (69) db_options ::= db_options CACHELAST NK_INTEGER */ - { 254, -3 }, /* (70) db_options ::= db_options COMP NK_INTEGER */ - { 254, -3 }, /* (71) db_options ::= db_options DURATION NK_INTEGER */ - { 254, -3 }, /* (72) db_options ::= db_options DURATION NK_VARIABLE */ - { 254, -3 }, /* (73) db_options ::= db_options FSYNC NK_INTEGER */ - { 254, -3 }, /* (74) db_options ::= db_options MAXROWS NK_INTEGER */ - { 254, -3 }, /* (75) db_options ::= db_options MINROWS NK_INTEGER */ - { 254, -3 }, /* (76) db_options ::= db_options KEEP integer_list */ - { 254, -3 }, /* (77) db_options ::= db_options KEEP variable_list */ - { 254, -3 }, /* (78) db_options ::= db_options PAGES NK_INTEGER */ - { 254, -3 }, /* (79) db_options ::= db_options PAGESIZE NK_INTEGER */ - { 254, -3 }, /* (80) db_options ::= db_options PRECISION NK_STRING */ - { 254, -3 }, /* (81) db_options ::= db_options REPLICA NK_INTEGER */ - { 254, -3 }, /* (82) db_options ::= db_options STRICT NK_INTEGER */ - { 254, -3 }, /* (83) db_options ::= db_options WAL NK_INTEGER */ - { 254, -3 }, /* (84) db_options ::= db_options VGROUPS NK_INTEGER */ - { 254, -3 }, /* (85) db_options ::= db_options SINGLE_STABLE NK_INTEGER */ - { 254, -3 }, /* (86) db_options ::= db_options RETENTIONS retention_list */ - { 254, -3 }, /* (87) db_options ::= db_options SCHEMALESS NK_INTEGER */ - { 256, -1 }, /* (88) alter_db_options ::= alter_db_option */ - { 256, -2 }, /* (89) alter_db_options ::= alter_db_options alter_db_option */ - { 260, -2 }, /* (90) alter_db_option ::= BUFFER NK_INTEGER */ - { 260, -2 }, /* (91) alter_db_option ::= CACHELAST NK_INTEGER */ - { 260, -2 }, /* (92) alter_db_option ::= FSYNC NK_INTEGER */ - { 260, -2 }, /* (93) alter_db_option ::= KEEP integer_list */ - { 260, -2 }, /* (94) alter_db_option ::= KEEP variable_list */ - { 260, -2 }, /* (95) alter_db_option ::= PAGES NK_INTEGER */ - { 260, -2 }, /* (96) alter_db_option ::= REPLICA NK_INTEGER */ - { 260, -2 }, /* (97) alter_db_option ::= STRICT NK_INTEGER */ - { 260, -2 }, /* (98) alter_db_option ::= WAL NK_INTEGER */ - { 257, -1 }, /* (99) integer_list ::= NK_INTEGER */ - { 257, -3 }, /* (100) integer_list ::= integer_list NK_COMMA NK_INTEGER */ - { 258, -1 }, /* (101) variable_list ::= NK_VARIABLE */ - { 258, -3 }, /* (102) variable_list ::= variable_list NK_COMMA NK_VARIABLE */ - { 259, -1 }, /* (103) retention_list ::= retention */ - { 259, -3 }, /* (104) retention_list ::= retention_list NK_COMMA retention */ - { 261, -3 }, /* (105) retention ::= NK_VARIABLE NK_COLON NK_VARIABLE */ - { 240, -9 }, /* (106) cmd ::= CREATE TABLE not_exists_opt full_table_name NK_LP column_def_list NK_RP tags_def_opt table_options */ - { 240, -3 }, /* (107) cmd ::= CREATE TABLE multi_create_clause */ - { 240, -9 }, /* (108) cmd ::= CREATE STABLE not_exists_opt full_table_name NK_LP column_def_list NK_RP tags_def table_options */ - { 240, -3 }, /* (109) cmd ::= DROP TABLE multi_drop_clause */ - { 240, -4 }, /* (110) cmd ::= DROP STABLE exists_opt full_table_name */ - { 240, -3 }, /* (111) cmd ::= ALTER TABLE alter_table_clause */ - { 240, -3 }, /* (112) cmd ::= ALTER STABLE alter_table_clause */ - { 269, -2 }, /* (113) alter_table_clause ::= full_table_name alter_table_options */ - { 269, -5 }, /* (114) alter_table_clause ::= full_table_name ADD COLUMN column_name type_name */ - { 269, -4 }, /* (115) alter_table_clause ::= full_table_name DROP COLUMN column_name */ - { 269, -5 }, /* (116) alter_table_clause ::= full_table_name MODIFY COLUMN column_name type_name */ - { 269, -5 }, /* (117) alter_table_clause ::= full_table_name RENAME COLUMN column_name column_name */ - { 269, -5 }, /* (118) alter_table_clause ::= full_table_name ADD TAG column_name type_name */ - { 269, -4 }, /* (119) alter_table_clause ::= full_table_name DROP TAG column_name */ - { 269, -5 }, /* (120) alter_table_clause ::= full_table_name MODIFY TAG column_name type_name */ - { 269, -5 }, /* (121) alter_table_clause ::= full_table_name RENAME TAG column_name column_name */ - { 269, -6 }, /* (122) alter_table_clause ::= full_table_name SET TAG column_name NK_EQ signed_literal */ - { 266, -1 }, /* (123) multi_create_clause ::= create_subtable_clause */ - { 266, -2 }, /* (124) multi_create_clause ::= multi_create_clause create_subtable_clause */ - { 274, -10 }, /* (125) create_subtable_clause ::= not_exists_opt full_table_name USING full_table_name specific_tags_opt TAGS NK_LP literal_list NK_RP table_options */ - { 268, -1 }, /* (126) multi_drop_clause ::= drop_table_clause */ - { 268, -2 }, /* (127) multi_drop_clause ::= multi_drop_clause drop_table_clause */ - { 277, -2 }, /* (128) drop_table_clause ::= exists_opt full_table_name */ - { 275, 0 }, /* (129) specific_tags_opt ::= */ - { 275, -3 }, /* (130) specific_tags_opt ::= NK_LP col_name_list NK_RP */ - { 262, -1 }, /* (131) full_table_name ::= table_name */ - { 262, -3 }, /* (132) full_table_name ::= db_name NK_DOT table_name */ - { 263, -1 }, /* (133) column_def_list ::= column_def */ - { 263, -3 }, /* (134) column_def_list ::= column_def_list NK_COMMA column_def */ - { 280, -2 }, /* (135) column_def ::= column_name type_name */ - { 280, -4 }, /* (136) column_def ::= column_name type_name COMMENT NK_STRING */ - { 272, -1 }, /* (137) type_name ::= BOOL */ - { 272, -1 }, /* (138) type_name ::= TINYINT */ - { 272, -1 }, /* (139) type_name ::= SMALLINT */ - { 272, -1 }, /* (140) type_name ::= INT */ - { 272, -1 }, /* (141) type_name ::= INTEGER */ - { 272, -1 }, /* (142) type_name ::= BIGINT */ - { 272, -1 }, /* (143) type_name ::= FLOAT */ - { 272, -1 }, /* (144) type_name ::= DOUBLE */ - { 272, -4 }, /* (145) type_name ::= BINARY NK_LP NK_INTEGER NK_RP */ - { 272, -1 }, /* (146) type_name ::= TIMESTAMP */ - { 272, -4 }, /* (147) type_name ::= NCHAR NK_LP NK_INTEGER NK_RP */ - { 272, -2 }, /* (148) type_name ::= TINYINT UNSIGNED */ - { 272, -2 }, /* (149) type_name ::= SMALLINT UNSIGNED */ - { 272, -2 }, /* (150) type_name ::= INT UNSIGNED */ - { 272, -2 }, /* (151) type_name ::= BIGINT UNSIGNED */ - { 272, -1 }, /* (152) type_name ::= JSON */ - { 272, -4 }, /* (153) type_name ::= VARCHAR NK_LP NK_INTEGER NK_RP */ - { 272, -1 }, /* (154) type_name ::= MEDIUMBLOB */ - { 272, -1 }, /* (155) type_name ::= BLOB */ - { 272, -4 }, /* (156) type_name ::= VARBINARY NK_LP NK_INTEGER NK_RP */ - { 272, -1 }, /* (157) type_name ::= DECIMAL */ - { 272, -4 }, /* (158) type_name ::= DECIMAL NK_LP NK_INTEGER NK_RP */ - { 272, -6 }, /* (159) type_name ::= DECIMAL NK_LP NK_INTEGER NK_COMMA NK_INTEGER NK_RP */ - { 264, 0 }, /* (160) tags_def_opt ::= */ - { 264, -1 }, /* (161) tags_def_opt ::= tags_def */ - { 267, -4 }, /* (162) tags_def ::= TAGS NK_LP column_def_list NK_RP */ - { 265, 0 }, /* (163) table_options ::= */ - { 265, -3 }, /* (164) table_options ::= table_options COMMENT NK_STRING */ - { 265, -3 }, /* (165) table_options ::= table_options FILE_FACTOR NK_FLOAT */ - { 265, -5 }, /* (166) table_options ::= table_options ROLLUP NK_LP func_name_list NK_RP */ - { 265, -3 }, /* (167) table_options ::= table_options TTL NK_INTEGER */ - { 265, -5 }, /* (168) table_options ::= table_options SMA NK_LP col_name_list NK_RP */ - { 270, -1 }, /* (169) alter_table_options ::= alter_table_option */ - { 270, -2 }, /* (170) alter_table_options ::= alter_table_options alter_table_option */ - { 282, -2 }, /* (171) alter_table_option ::= COMMENT NK_STRING */ - { 282, -2 }, /* (172) alter_table_option ::= TTL NK_INTEGER */ - { 278, -1 }, /* (173) col_name_list ::= col_name */ - { 278, -3 }, /* (174) col_name_list ::= col_name_list NK_COMMA col_name */ - { 283, -1 }, /* (175) col_name ::= column_name */ - { 240, -2 }, /* (176) cmd ::= SHOW DNODES */ - { 240, -2 }, /* (177) cmd ::= SHOW USERS */ - { 240, -2 }, /* (178) cmd ::= SHOW DATABASES */ - { 240, -4 }, /* (179) cmd ::= SHOW db_name_cond_opt TABLES like_pattern_opt */ - { 240, -4 }, /* (180) cmd ::= SHOW db_name_cond_opt STABLES like_pattern_opt */ - { 240, -3 }, /* (181) cmd ::= SHOW db_name_cond_opt VGROUPS */ - { 240, -2 }, /* (182) cmd ::= SHOW MNODES */ - { 240, -2 }, /* (183) cmd ::= SHOW MODULES */ - { 240, -2 }, /* (184) cmd ::= SHOW QNODES */ - { 240, -2 }, /* (185) cmd ::= SHOW FUNCTIONS */ - { 240, -5 }, /* (186) cmd ::= SHOW INDEXES FROM table_name_cond from_db_opt */ - { 240, -2 }, /* (187) cmd ::= SHOW STREAMS */ - { 240, -2 }, /* (188) cmd ::= SHOW ACCOUNTS */ - { 240, -2 }, /* (189) cmd ::= SHOW APPS */ - { 240, -2 }, /* (190) cmd ::= SHOW CONNECTIONS */ - { 240, -2 }, /* (191) cmd ::= SHOW LICENCE */ - { 240, -2 }, /* (192) cmd ::= SHOW GRANTS */ - { 240, -4 }, /* (193) cmd ::= SHOW CREATE DATABASE db_name */ - { 240, -4 }, /* (194) cmd ::= SHOW CREATE TABLE full_table_name */ - { 240, -4 }, /* (195) cmd ::= SHOW CREATE STABLE full_table_name */ - { 240, -2 }, /* (196) cmd ::= SHOW QUERIES */ - { 240, -2 }, /* (197) cmd ::= SHOW SCORES */ - { 240, -2 }, /* (198) cmd ::= SHOW TOPICS */ - { 240, -2 }, /* (199) cmd ::= SHOW VARIABLES */ - { 240, -2 }, /* (200) cmd ::= SHOW BNODES */ - { 240, -2 }, /* (201) cmd ::= SHOW SNODES */ - { 240, -2 }, /* (202) cmd ::= SHOW CLUSTER */ - { 240, -2 }, /* (203) cmd ::= SHOW TRANSACTIONS */ - { 284, 0 }, /* (204) db_name_cond_opt ::= */ - { 284, -2 }, /* (205) db_name_cond_opt ::= db_name NK_DOT */ - { 285, 0 }, /* (206) like_pattern_opt ::= */ - { 285, -2 }, /* (207) like_pattern_opt ::= LIKE NK_STRING */ - { 286, -1 }, /* (208) table_name_cond ::= table_name */ - { 287, 0 }, /* (209) from_db_opt ::= */ - { 287, -2 }, /* (210) from_db_opt ::= FROM db_name */ - { 281, -1 }, /* (211) func_name_list ::= func_name */ - { 281, -3 }, /* (212) func_name_list ::= func_name_list NK_COMMA func_name */ - { 288, -1 }, /* (213) func_name ::= function_name */ - { 240, -8 }, /* (214) cmd ::= CREATE SMA INDEX not_exists_opt index_name ON table_name index_options */ - { 240, -10 }, /* (215) cmd ::= CREATE FULLTEXT INDEX not_exists_opt index_name ON table_name NK_LP col_name_list NK_RP */ - { 240, -6 }, /* (216) cmd ::= DROP INDEX exists_opt index_name ON table_name */ - { 291, 0 }, /* (217) index_options ::= */ - { 291, -9 }, /* (218) index_options ::= FUNCTION NK_LP func_list NK_RP INTERVAL NK_LP duration_literal NK_RP sliding_opt */ - { 291, -11 }, /* (219) index_options ::= FUNCTION NK_LP func_list NK_RP INTERVAL NK_LP duration_literal NK_COMMA duration_literal NK_RP sliding_opt */ - { 292, -1 }, /* (220) func_list ::= func */ - { 292, -3 }, /* (221) func_list ::= func_list NK_COMMA func */ - { 295, -4 }, /* (222) func ::= function_name NK_LP expression_list NK_RP */ - { 240, -6 }, /* (223) cmd ::= CREATE TOPIC not_exists_opt topic_name AS query_expression */ - { 240, -7 }, /* (224) cmd ::= CREATE TOPIC not_exists_opt topic_name AS DATABASE db_name */ - { 240, -7 }, /* (225) cmd ::= CREATE TOPIC not_exists_opt topic_name AS STABLE full_table_name */ - { 240, -4 }, /* (226) cmd ::= DROP TOPIC exists_opt topic_name */ - { 240, -7 }, /* (227) cmd ::= DROP CONSUMER GROUP exists_opt cgroup_name ON topic_name */ - { 240, -2 }, /* (228) cmd ::= DESC full_table_name */ - { 240, -2 }, /* (229) cmd ::= DESCRIBE full_table_name */ - { 240, -3 }, /* (230) cmd ::= RESET QUERY CACHE */ - { 240, -4 }, /* (231) cmd ::= EXPLAIN analyze_opt explain_options query_expression */ - { 300, 0 }, /* (232) analyze_opt ::= */ - { 300, -1 }, /* (233) analyze_opt ::= ANALYZE */ - { 301, 0 }, /* (234) explain_options ::= */ - { 301, -3 }, /* (235) explain_options ::= explain_options VERBOSE NK_BOOL */ - { 301, -3 }, /* (236) explain_options ::= explain_options RATIO NK_FLOAT */ - { 240, -6 }, /* (237) cmd ::= COMPACT VNODES IN NK_LP integer_list NK_RP */ - { 240, -10 }, /* (238) cmd ::= CREATE agg_func_opt FUNCTION not_exists_opt function_name AS NK_STRING OUTPUTTYPE type_name bufsize_opt */ - { 240, -4 }, /* (239) cmd ::= DROP FUNCTION exists_opt function_name */ - { 302, 0 }, /* (240) agg_func_opt ::= */ - { 302, -1 }, /* (241) agg_func_opt ::= AGGREGATE */ - { 303, 0 }, /* (242) bufsize_opt ::= */ - { 303, -2 }, /* (243) bufsize_opt ::= BUFSIZE NK_INTEGER */ - { 240, -8 }, /* (244) cmd ::= CREATE STREAM not_exists_opt stream_name stream_options into_opt AS query_expression */ - { 240, -4 }, /* (245) cmd ::= DROP STREAM exists_opt stream_name */ - { 306, 0 }, /* (246) into_opt ::= */ - { 306, -2 }, /* (247) into_opt ::= INTO full_table_name */ - { 305, 0 }, /* (248) stream_options ::= */ - { 305, -3 }, /* (249) stream_options ::= stream_options TRIGGER AT_ONCE */ - { 305, -3 }, /* (250) stream_options ::= stream_options TRIGGER WINDOW_CLOSE */ - { 305, -4 }, /* (251) stream_options ::= stream_options TRIGGER MAX_DELAY duration_literal */ - { 305, -3 }, /* (252) stream_options ::= stream_options WATERMARK duration_literal */ - { 240, -3 }, /* (253) cmd ::= KILL CONNECTION NK_INTEGER */ - { 240, -3 }, /* (254) cmd ::= KILL QUERY NK_STRING */ - { 240, -3 }, /* (255) cmd ::= KILL TRANSACTION NK_INTEGER */ - { 240, -2 }, /* (256) cmd ::= BALANCE VGROUP */ - { 240, -4 }, /* (257) cmd ::= MERGE VGROUP NK_INTEGER NK_INTEGER */ - { 240, -4 }, /* (258) cmd ::= REDISTRIBUTE VGROUP NK_INTEGER dnode_list */ - { 240, -3 }, /* (259) cmd ::= SPLIT VGROUP NK_INTEGER */ - { 307, -2 }, /* (260) dnode_list ::= DNODE NK_INTEGER */ - { 307, -3 }, /* (261) dnode_list ::= dnode_list DNODE NK_INTEGER */ - { 240, -3 }, /* (262) cmd ::= SYNCDB db_name REPLICA */ - { 240, -4 }, /* (263) cmd ::= DELETE FROM full_table_name where_clause_opt */ - { 240, -1 }, /* (264) cmd ::= query_expression */ - { 243, -1 }, /* (265) literal ::= NK_INTEGER */ - { 243, -1 }, /* (266) literal ::= NK_FLOAT */ - { 243, -1 }, /* (267) literal ::= NK_STRING */ - { 243, -1 }, /* (268) literal ::= NK_BOOL */ - { 243, -2 }, /* (269) literal ::= TIMESTAMP NK_STRING */ - { 243, -1 }, /* (270) literal ::= duration_literal */ - { 243, -1 }, /* (271) literal ::= NULL */ - { 243, -1 }, /* (272) literal ::= NK_QUESTION */ - { 293, -1 }, /* (273) duration_literal ::= NK_VARIABLE */ - { 309, -1 }, /* (274) signed ::= NK_INTEGER */ - { 309, -2 }, /* (275) signed ::= NK_PLUS NK_INTEGER */ - { 309, -2 }, /* (276) signed ::= NK_MINUS NK_INTEGER */ - { 309, -1 }, /* (277) signed ::= NK_FLOAT */ - { 309, -2 }, /* (278) signed ::= NK_PLUS NK_FLOAT */ - { 309, -2 }, /* (279) signed ::= NK_MINUS NK_FLOAT */ - { 273, -1 }, /* (280) signed_literal ::= signed */ - { 273, -1 }, /* (281) signed_literal ::= NK_STRING */ - { 273, -1 }, /* (282) signed_literal ::= NK_BOOL */ - { 273, -2 }, /* (283) signed_literal ::= TIMESTAMP NK_STRING */ - { 273, -1 }, /* (284) signed_literal ::= duration_literal */ - { 273, -1 }, /* (285) signed_literal ::= NULL */ - { 273, -1 }, /* (286) signed_literal ::= literal_func */ - { 276, -1 }, /* (287) literal_list ::= signed_literal */ - { 276, -3 }, /* (288) literal_list ::= literal_list NK_COMMA signed_literal */ - { 250, -1 }, /* (289) db_name ::= NK_ID */ - { 279, -1 }, /* (290) table_name ::= NK_ID */ - { 271, -1 }, /* (291) column_name ::= NK_ID */ - { 289, -1 }, /* (292) function_name ::= NK_ID */ - { 311, -1 }, /* (293) table_alias ::= NK_ID */ - { 312, -1 }, /* (294) column_alias ::= NK_ID */ - { 245, -1 }, /* (295) user_name ::= NK_ID */ - { 290, -1 }, /* (296) index_name ::= NK_ID */ - { 297, -1 }, /* (297) topic_name ::= NK_ID */ - { 304, -1 }, /* (298) stream_name ::= NK_ID */ - { 299, -1 }, /* (299) cgroup_name ::= NK_ID */ - { 313, -1 }, /* (300) expression ::= literal */ - { 313, -1 }, /* (301) expression ::= pseudo_column */ - { 313, -1 }, /* (302) expression ::= column_reference */ - { 313, -1 }, /* (303) expression ::= function_expression */ - { 313, -1 }, /* (304) expression ::= subquery */ - { 313, -3 }, /* (305) expression ::= NK_LP expression NK_RP */ - { 313, -2 }, /* (306) expression ::= NK_PLUS expression */ - { 313, -2 }, /* (307) expression ::= NK_MINUS expression */ - { 313, -3 }, /* (308) expression ::= expression NK_PLUS expression */ - { 313, -3 }, /* (309) expression ::= expression NK_MINUS expression */ - { 313, -3 }, /* (310) expression ::= expression NK_STAR expression */ - { 313, -3 }, /* (311) expression ::= expression NK_SLASH expression */ - { 313, -3 }, /* (312) expression ::= expression NK_REM expression */ - { 313, -3 }, /* (313) expression ::= column_reference NK_ARROW NK_STRING */ - { 296, -1 }, /* (314) expression_list ::= expression */ - { 296, -3 }, /* (315) expression_list ::= expression_list NK_COMMA expression */ - { 315, -1 }, /* (316) column_reference ::= column_name */ - { 315, -3 }, /* (317) column_reference ::= table_name NK_DOT column_name */ - { 314, -1 }, /* (318) pseudo_column ::= ROWTS */ - { 314, -1 }, /* (319) pseudo_column ::= TBNAME */ - { 314, -3 }, /* (320) pseudo_column ::= table_name NK_DOT TBNAME */ - { 314, -1 }, /* (321) pseudo_column ::= QSTARTTS */ - { 314, -1 }, /* (322) pseudo_column ::= QENDTS */ - { 314, -1 }, /* (323) pseudo_column ::= WSTARTTS */ - { 314, -1 }, /* (324) pseudo_column ::= WENDTS */ - { 314, -1 }, /* (325) pseudo_column ::= WDURATION */ - { 316, -4 }, /* (326) function_expression ::= function_name NK_LP expression_list NK_RP */ - { 316, -4 }, /* (327) function_expression ::= star_func NK_LP star_func_para_list NK_RP */ - { 316, -6 }, /* (328) function_expression ::= CAST NK_LP expression AS type_name NK_RP */ - { 316, -1 }, /* (329) function_expression ::= literal_func */ - { 310, -3 }, /* (330) literal_func ::= noarg_func NK_LP NK_RP */ - { 310, -1 }, /* (331) literal_func ::= NOW */ - { 320, -1 }, /* (332) noarg_func ::= NOW */ - { 320, -1 }, /* (333) noarg_func ::= TODAY */ - { 320, -1 }, /* (334) noarg_func ::= TIMEZONE */ - { 318, -1 }, /* (335) star_func ::= COUNT */ - { 318, -1 }, /* (336) star_func ::= FIRST */ - { 318, -1 }, /* (337) star_func ::= LAST */ - { 318, -1 }, /* (338) star_func ::= LAST_ROW */ - { 319, -1 }, /* (339) star_func_para_list ::= NK_STAR */ - { 319, -1 }, /* (340) star_func_para_list ::= other_para_list */ - { 321, -1 }, /* (341) other_para_list ::= star_func_para */ - { 321, -3 }, /* (342) other_para_list ::= other_para_list NK_COMMA star_func_para */ - { 322, -1 }, /* (343) star_func_para ::= expression */ - { 322, -3 }, /* (344) star_func_para ::= table_name NK_DOT NK_STAR */ - { 323, -3 }, /* (345) predicate ::= expression compare_op expression */ - { 323, -5 }, /* (346) predicate ::= expression BETWEEN expression AND expression */ - { 323, -6 }, /* (347) predicate ::= expression NOT BETWEEN expression AND expression */ - { 323, -3 }, /* (348) predicate ::= expression IS NULL */ - { 323, -4 }, /* (349) predicate ::= expression IS NOT NULL */ - { 323, -3 }, /* (350) predicate ::= expression in_op in_predicate_value */ - { 324, -1 }, /* (351) compare_op ::= NK_LT */ - { 324, -1 }, /* (352) compare_op ::= NK_GT */ - { 324, -1 }, /* (353) compare_op ::= NK_LE */ - { 324, -1 }, /* (354) compare_op ::= NK_GE */ - { 324, -1 }, /* (355) compare_op ::= NK_NE */ - { 324, -1 }, /* (356) compare_op ::= NK_EQ */ - { 324, -1 }, /* (357) compare_op ::= LIKE */ - { 324, -2 }, /* (358) compare_op ::= NOT LIKE */ - { 324, -1 }, /* (359) compare_op ::= MATCH */ - { 324, -1 }, /* (360) compare_op ::= NMATCH */ - { 324, -1 }, /* (361) compare_op ::= CONTAINS */ - { 325, -1 }, /* (362) in_op ::= IN */ - { 325, -2 }, /* (363) in_op ::= NOT IN */ - { 326, -3 }, /* (364) in_predicate_value ::= NK_LP expression_list NK_RP */ - { 327, -1 }, /* (365) boolean_value_expression ::= boolean_primary */ - { 327, -2 }, /* (366) boolean_value_expression ::= NOT boolean_primary */ - { 327, -3 }, /* (367) boolean_value_expression ::= boolean_value_expression OR boolean_value_expression */ - { 327, -3 }, /* (368) boolean_value_expression ::= boolean_value_expression AND boolean_value_expression */ - { 328, -1 }, /* (369) boolean_primary ::= predicate */ - { 328, -3 }, /* (370) boolean_primary ::= NK_LP boolean_value_expression NK_RP */ - { 329, -1 }, /* (371) common_expression ::= expression */ - { 329, -1 }, /* (372) common_expression ::= boolean_value_expression */ - { 330, -2 }, /* (373) from_clause ::= FROM table_reference_list */ - { 331, -1 }, /* (374) table_reference_list ::= table_reference */ - { 331, -3 }, /* (375) table_reference_list ::= table_reference_list NK_COMMA table_reference */ - { 332, -1 }, /* (376) table_reference ::= table_primary */ - { 332, -1 }, /* (377) table_reference ::= joined_table */ - { 333, -2 }, /* (378) table_primary ::= table_name alias_opt */ - { 333, -4 }, /* (379) table_primary ::= db_name NK_DOT table_name alias_opt */ - { 333, -2 }, /* (380) table_primary ::= subquery alias_opt */ - { 333, -1 }, /* (381) table_primary ::= parenthesized_joined_table */ - { 335, 0 }, /* (382) alias_opt ::= */ - { 335, -1 }, /* (383) alias_opt ::= table_alias */ - { 335, -2 }, /* (384) alias_opt ::= AS table_alias */ - { 336, -3 }, /* (385) parenthesized_joined_table ::= NK_LP joined_table NK_RP */ - { 336, -3 }, /* (386) parenthesized_joined_table ::= NK_LP parenthesized_joined_table NK_RP */ - { 334, -6 }, /* (387) joined_table ::= table_reference join_type JOIN table_reference ON search_condition */ - { 337, 0 }, /* (388) join_type ::= */ - { 337, -1 }, /* (389) join_type ::= INNER */ - { 339, -9 }, /* (390) query_specification ::= SELECT set_quantifier_opt select_list from_clause where_clause_opt partition_by_clause_opt twindow_clause_opt group_by_clause_opt having_clause_opt */ - { 340, 0 }, /* (391) set_quantifier_opt ::= */ - { 340, -1 }, /* (392) set_quantifier_opt ::= DISTINCT */ - { 340, -1 }, /* (393) set_quantifier_opt ::= ALL */ - { 341, -1 }, /* (394) select_list ::= NK_STAR */ - { 341, -1 }, /* (395) select_list ::= select_sublist */ - { 346, -1 }, /* (396) select_sublist ::= select_item */ - { 346, -3 }, /* (397) select_sublist ::= select_sublist NK_COMMA select_item */ - { 347, -1 }, /* (398) select_item ::= common_expression */ - { 347, -2 }, /* (399) select_item ::= common_expression column_alias */ - { 347, -3 }, /* (400) select_item ::= common_expression AS column_alias */ - { 347, -3 }, /* (401) select_item ::= table_name NK_DOT NK_STAR */ - { 308, 0 }, /* (402) where_clause_opt ::= */ - { 308, -2 }, /* (403) where_clause_opt ::= WHERE search_condition */ - { 342, 0 }, /* (404) partition_by_clause_opt ::= */ - { 342, -3 }, /* (405) partition_by_clause_opt ::= PARTITION BY expression_list */ - { 343, 0 }, /* (406) twindow_clause_opt ::= */ - { 343, -6 }, /* (407) twindow_clause_opt ::= SESSION NK_LP column_reference NK_COMMA duration_literal NK_RP */ - { 343, -4 }, /* (408) twindow_clause_opt ::= STATE_WINDOW NK_LP expression NK_RP */ - { 343, -6 }, /* (409) twindow_clause_opt ::= INTERVAL NK_LP duration_literal NK_RP sliding_opt fill_opt */ - { 343, -8 }, /* (410) twindow_clause_opt ::= INTERVAL NK_LP duration_literal NK_COMMA duration_literal NK_RP sliding_opt fill_opt */ - { 294, 0 }, /* (411) sliding_opt ::= */ - { 294, -4 }, /* (412) sliding_opt ::= SLIDING NK_LP duration_literal NK_RP */ - { 348, 0 }, /* (413) fill_opt ::= */ - { 348, -4 }, /* (414) fill_opt ::= FILL NK_LP fill_mode NK_RP */ - { 348, -6 }, /* (415) fill_opt ::= FILL NK_LP VALUE NK_COMMA literal_list NK_RP */ - { 349, -1 }, /* (416) fill_mode ::= NONE */ - { 349, -1 }, /* (417) fill_mode ::= PREV */ - { 349, -1 }, /* (418) fill_mode ::= NULL */ - { 349, -1 }, /* (419) fill_mode ::= LINEAR */ - { 349, -1 }, /* (420) fill_mode ::= NEXT */ - { 344, 0 }, /* (421) group_by_clause_opt ::= */ - { 344, -3 }, /* (422) group_by_clause_opt ::= GROUP BY group_by_list */ - { 350, -1 }, /* (423) group_by_list ::= expression */ - { 350, -3 }, /* (424) group_by_list ::= group_by_list NK_COMMA expression */ - { 345, 0 }, /* (425) having_clause_opt ::= */ - { 345, -2 }, /* (426) having_clause_opt ::= HAVING search_condition */ - { 298, -4 }, /* (427) query_expression ::= query_expression_body order_by_clause_opt slimit_clause_opt limit_clause_opt */ - { 351, -1 }, /* (428) query_expression_body ::= query_primary */ - { 351, -4 }, /* (429) query_expression_body ::= query_expression_body UNION ALL query_expression_body */ - { 351, -3 }, /* (430) query_expression_body ::= query_expression_body UNION query_expression_body */ - { 355, -1 }, /* (431) query_primary ::= query_specification */ - { 355, -6 }, /* (432) query_primary ::= NK_LP query_expression_body order_by_clause_opt slimit_clause_opt limit_clause_opt NK_RP */ - { 352, 0 }, /* (433) order_by_clause_opt ::= */ - { 352, -3 }, /* (434) order_by_clause_opt ::= ORDER BY sort_specification_list */ - { 353, 0 }, /* (435) slimit_clause_opt ::= */ - { 353, -2 }, /* (436) slimit_clause_opt ::= SLIMIT NK_INTEGER */ - { 353, -4 }, /* (437) slimit_clause_opt ::= SLIMIT NK_INTEGER SOFFSET NK_INTEGER */ - { 353, -4 }, /* (438) slimit_clause_opt ::= SLIMIT NK_INTEGER NK_COMMA NK_INTEGER */ - { 354, 0 }, /* (439) limit_clause_opt ::= */ - { 354, -2 }, /* (440) limit_clause_opt ::= LIMIT NK_INTEGER */ - { 354, -4 }, /* (441) limit_clause_opt ::= LIMIT NK_INTEGER OFFSET NK_INTEGER */ - { 354, -4 }, /* (442) limit_clause_opt ::= LIMIT NK_INTEGER NK_COMMA NK_INTEGER */ - { 317, -3 }, /* (443) subquery ::= NK_LP query_expression NK_RP */ - { 338, -1 }, /* (444) search_condition ::= common_expression */ - { 356, -1 }, /* (445) sort_specification_list ::= sort_specification */ - { 356, -3 }, /* (446) sort_specification_list ::= sort_specification_list NK_COMMA sort_specification */ - { 357, -3 }, /* (447) sort_specification ::= expression ordering_specification_opt null_ordering_opt */ - { 358, 0 }, /* (448) ordering_specification_opt ::= */ - { 358, -1 }, /* (449) ordering_specification_opt ::= ASC */ - { 358, -1 }, /* (450) ordering_specification_opt ::= DESC */ - { 359, 0 }, /* (451) null_ordering_opt ::= */ - { 359, -2 }, /* (452) null_ordering_opt ::= NULLS FIRST */ - { 359, -2 }, /* (453) null_ordering_opt ::= NULLS LAST */ + { 239, -6 }, /* (0) cmd ::= CREATE ACCOUNT NK_ID PASS NK_STRING account_options */ + { 239, -4 }, /* (1) cmd ::= ALTER ACCOUNT NK_ID alter_account_options */ + { 240, 0 }, /* (2) account_options ::= */ + { 240, -3 }, /* (3) account_options ::= account_options PPS literal */ + { 240, -3 }, /* (4) account_options ::= account_options TSERIES literal */ + { 240, -3 }, /* (5) account_options ::= account_options STORAGE literal */ + { 240, -3 }, /* (6) account_options ::= account_options STREAMS literal */ + { 240, -3 }, /* (7) account_options ::= account_options QTIME literal */ + { 240, -3 }, /* (8) account_options ::= account_options DBS literal */ + { 240, -3 }, /* (9) account_options ::= account_options USERS literal */ + { 240, -3 }, /* (10) account_options ::= account_options CONNS literal */ + { 240, -3 }, /* (11) account_options ::= account_options STATE literal */ + { 241, -1 }, /* (12) alter_account_options ::= alter_account_option */ + { 241, -2 }, /* (13) alter_account_options ::= alter_account_options alter_account_option */ + { 243, -2 }, /* (14) alter_account_option ::= PASS literal */ + { 243, -2 }, /* (15) alter_account_option ::= PPS literal */ + { 243, -2 }, /* (16) alter_account_option ::= TSERIES literal */ + { 243, -2 }, /* (17) alter_account_option ::= STORAGE literal */ + { 243, -2 }, /* (18) alter_account_option ::= STREAMS literal */ + { 243, -2 }, /* (19) alter_account_option ::= QTIME literal */ + { 243, -2 }, /* (20) alter_account_option ::= DBS literal */ + { 243, -2 }, /* (21) alter_account_option ::= USERS literal */ + { 243, -2 }, /* (22) alter_account_option ::= CONNS literal */ + { 243, -2 }, /* (23) alter_account_option ::= STATE literal */ + { 239, -5 }, /* (24) cmd ::= CREATE USER user_name PASS NK_STRING */ + { 239, -5 }, /* (25) cmd ::= ALTER USER user_name PASS NK_STRING */ + { 239, -5 }, /* (26) cmd ::= ALTER USER user_name PRIVILEGE NK_STRING */ + { 239, -3 }, /* (27) cmd ::= DROP USER user_name */ + { 239, -6 }, /* (28) cmd ::= GRANT privileges ON priv_level TO user_name */ + { 239, -6 }, /* (29) cmd ::= REVOKE privileges ON priv_level FROM user_name */ + { 245, -1 }, /* (30) privileges ::= ALL */ + { 245, -1 }, /* (31) privileges ::= priv_type_list */ + { 247, -1 }, /* (32) priv_type_list ::= priv_type */ + { 247, -3 }, /* (33) priv_type_list ::= priv_type_list NK_COMMA priv_type */ + { 248, -1 }, /* (34) priv_type ::= READ */ + { 248, -1 }, /* (35) priv_type ::= WRITE */ + { 246, -3 }, /* (36) priv_level ::= NK_STAR NK_DOT NK_STAR */ + { 246, -3 }, /* (37) priv_level ::= db_name NK_DOT NK_STAR */ + { 239, -3 }, /* (38) cmd ::= CREATE DNODE dnode_endpoint */ + { 239, -5 }, /* (39) cmd ::= CREATE DNODE dnode_endpoint PORT NK_INTEGER */ + { 239, -3 }, /* (40) cmd ::= DROP DNODE NK_INTEGER */ + { 239, -3 }, /* (41) cmd ::= DROP DNODE dnode_endpoint */ + { 239, -4 }, /* (42) cmd ::= ALTER DNODE NK_INTEGER NK_STRING */ + { 239, -5 }, /* (43) cmd ::= ALTER DNODE NK_INTEGER NK_STRING NK_STRING */ + { 239, -4 }, /* (44) cmd ::= ALTER ALL DNODES NK_STRING */ + { 239, -5 }, /* (45) cmd ::= ALTER ALL DNODES NK_STRING NK_STRING */ + { 250, -1 }, /* (46) dnode_endpoint ::= NK_STRING */ + { 250, -1 }, /* (47) dnode_endpoint ::= NK_ID */ + { 250, -1 }, /* (48) dnode_endpoint ::= NK_IPTOKEN */ + { 239, -3 }, /* (49) cmd ::= ALTER LOCAL NK_STRING */ + { 239, -4 }, /* (50) cmd ::= ALTER LOCAL NK_STRING NK_STRING */ + { 239, -5 }, /* (51) cmd ::= CREATE QNODE ON DNODE NK_INTEGER */ + { 239, -5 }, /* (52) cmd ::= DROP QNODE ON DNODE NK_INTEGER */ + { 239, -5 }, /* (53) cmd ::= CREATE BNODE ON DNODE NK_INTEGER */ + { 239, -5 }, /* (54) cmd ::= DROP BNODE ON DNODE NK_INTEGER */ + { 239, -5 }, /* (55) cmd ::= CREATE SNODE ON DNODE NK_INTEGER */ + { 239, -5 }, /* (56) cmd ::= DROP SNODE ON DNODE NK_INTEGER */ + { 239, -5 }, /* (57) cmd ::= CREATE MNODE ON DNODE NK_INTEGER */ + { 239, -5 }, /* (58) cmd ::= DROP MNODE ON DNODE NK_INTEGER */ + { 239, -5 }, /* (59) cmd ::= CREATE DATABASE not_exists_opt db_name db_options */ + { 239, -4 }, /* (60) cmd ::= DROP DATABASE exists_opt db_name */ + { 239, -2 }, /* (61) cmd ::= USE db_name */ + { 239, -4 }, /* (62) cmd ::= ALTER DATABASE db_name alter_db_options */ + { 251, -3 }, /* (63) not_exists_opt ::= IF NOT EXISTS */ + { 251, 0 }, /* (64) not_exists_opt ::= */ + { 253, -2 }, /* (65) exists_opt ::= IF EXISTS */ + { 253, 0 }, /* (66) exists_opt ::= */ + { 252, 0 }, /* (67) db_options ::= */ + { 252, -3 }, /* (68) db_options ::= db_options BUFFER NK_INTEGER */ + { 252, -3 }, /* (69) db_options ::= db_options CACHELAST NK_INTEGER */ + { 252, -3 }, /* (70) db_options ::= db_options COMP NK_INTEGER */ + { 252, -3 }, /* (71) db_options ::= db_options DURATION NK_INTEGER */ + { 252, -3 }, /* (72) db_options ::= db_options DURATION NK_VARIABLE */ + { 252, -3 }, /* (73) db_options ::= db_options FSYNC NK_INTEGER */ + { 252, -3 }, /* (74) db_options ::= db_options MAXROWS NK_INTEGER */ + { 252, -3 }, /* (75) db_options ::= db_options MINROWS NK_INTEGER */ + { 252, -3 }, /* (76) db_options ::= db_options KEEP integer_list */ + { 252, -3 }, /* (77) db_options ::= db_options KEEP variable_list */ + { 252, -3 }, /* (78) db_options ::= db_options PAGES NK_INTEGER */ + { 252, -3 }, /* (79) db_options ::= db_options PAGESIZE NK_INTEGER */ + { 252, -3 }, /* (80) db_options ::= db_options PRECISION NK_STRING */ + { 252, -3 }, /* (81) db_options ::= db_options REPLICA NK_INTEGER */ + { 252, -3 }, /* (82) db_options ::= db_options STRICT NK_INTEGER */ + { 252, -3 }, /* (83) db_options ::= db_options WAL NK_INTEGER */ + { 252, -3 }, /* (84) db_options ::= db_options VGROUPS NK_INTEGER */ + { 252, -3 }, /* (85) db_options ::= db_options SINGLE_STABLE NK_INTEGER */ + { 252, -3 }, /* (86) db_options ::= db_options RETENTIONS retention_list */ + { 252, -3 }, /* (87) db_options ::= db_options SCHEMALESS NK_INTEGER */ + { 254, -1 }, /* (88) alter_db_options ::= alter_db_option */ + { 254, -2 }, /* (89) alter_db_options ::= alter_db_options alter_db_option */ + { 258, -2 }, /* (90) alter_db_option ::= BUFFER NK_INTEGER */ + { 258, -2 }, /* (91) alter_db_option ::= CACHELAST NK_INTEGER */ + { 258, -2 }, /* (92) alter_db_option ::= FSYNC NK_INTEGER */ + { 258, -2 }, /* (93) alter_db_option ::= KEEP integer_list */ + { 258, -2 }, /* (94) alter_db_option ::= KEEP variable_list */ + { 258, -2 }, /* (95) alter_db_option ::= PAGES NK_INTEGER */ + { 258, -2 }, /* (96) alter_db_option ::= REPLICA NK_INTEGER */ + { 258, -2 }, /* (97) alter_db_option ::= STRICT NK_INTEGER */ + { 258, -2 }, /* (98) alter_db_option ::= WAL NK_INTEGER */ + { 255, -1 }, /* (99) integer_list ::= NK_INTEGER */ + { 255, -3 }, /* (100) integer_list ::= integer_list NK_COMMA NK_INTEGER */ + { 256, -1 }, /* (101) variable_list ::= NK_VARIABLE */ + { 256, -3 }, /* (102) variable_list ::= variable_list NK_COMMA NK_VARIABLE */ + { 257, -1 }, /* (103) retention_list ::= retention */ + { 257, -3 }, /* (104) retention_list ::= retention_list NK_COMMA retention */ + { 259, -3 }, /* (105) retention ::= NK_VARIABLE NK_COLON NK_VARIABLE */ + { 239, -9 }, /* (106) cmd ::= CREATE TABLE not_exists_opt full_table_name NK_LP column_def_list NK_RP tags_def_opt table_options */ + { 239, -3 }, /* (107) cmd ::= CREATE TABLE multi_create_clause */ + { 239, -9 }, /* (108) cmd ::= CREATE STABLE not_exists_opt full_table_name NK_LP column_def_list NK_RP tags_def table_options */ + { 239, -3 }, /* (109) cmd ::= DROP TABLE multi_drop_clause */ + { 239, -4 }, /* (110) cmd ::= DROP STABLE exists_opt full_table_name */ + { 239, -3 }, /* (111) cmd ::= ALTER TABLE alter_table_clause */ + { 239, -3 }, /* (112) cmd ::= ALTER STABLE alter_table_clause */ + { 267, -2 }, /* (113) alter_table_clause ::= full_table_name alter_table_options */ + { 267, -5 }, /* (114) alter_table_clause ::= full_table_name ADD COLUMN column_name type_name */ + { 267, -4 }, /* (115) alter_table_clause ::= full_table_name DROP COLUMN column_name */ + { 267, -5 }, /* (116) alter_table_clause ::= full_table_name MODIFY COLUMN column_name type_name */ + { 267, -5 }, /* (117) alter_table_clause ::= full_table_name RENAME COLUMN column_name column_name */ + { 267, -5 }, /* (118) alter_table_clause ::= full_table_name ADD TAG column_name type_name */ + { 267, -4 }, /* (119) alter_table_clause ::= full_table_name DROP TAG column_name */ + { 267, -5 }, /* (120) alter_table_clause ::= full_table_name MODIFY TAG column_name type_name */ + { 267, -5 }, /* (121) alter_table_clause ::= full_table_name RENAME TAG column_name column_name */ + { 267, -6 }, /* (122) alter_table_clause ::= full_table_name SET TAG column_name NK_EQ signed_literal */ + { 264, -1 }, /* (123) multi_create_clause ::= create_subtable_clause */ + { 264, -2 }, /* (124) multi_create_clause ::= multi_create_clause create_subtable_clause */ + { 272, -10 }, /* (125) create_subtable_clause ::= not_exists_opt full_table_name USING full_table_name specific_tags_opt TAGS NK_LP literal_list NK_RP table_options */ + { 266, -1 }, /* (126) multi_drop_clause ::= drop_table_clause */ + { 266, -2 }, /* (127) multi_drop_clause ::= multi_drop_clause drop_table_clause */ + { 275, -2 }, /* (128) drop_table_clause ::= exists_opt full_table_name */ + { 273, 0 }, /* (129) specific_tags_opt ::= */ + { 273, -3 }, /* (130) specific_tags_opt ::= NK_LP col_name_list NK_RP */ + { 260, -1 }, /* (131) full_table_name ::= table_name */ + { 260, -3 }, /* (132) full_table_name ::= db_name NK_DOT table_name */ + { 261, -1 }, /* (133) column_def_list ::= column_def */ + { 261, -3 }, /* (134) column_def_list ::= column_def_list NK_COMMA column_def */ + { 278, -2 }, /* (135) column_def ::= column_name type_name */ + { 278, -4 }, /* (136) column_def ::= column_name type_name COMMENT NK_STRING */ + { 270, -1 }, /* (137) type_name ::= BOOL */ + { 270, -1 }, /* (138) type_name ::= TINYINT */ + { 270, -1 }, /* (139) type_name ::= SMALLINT */ + { 270, -1 }, /* (140) type_name ::= INT */ + { 270, -1 }, /* (141) type_name ::= INTEGER */ + { 270, -1 }, /* (142) type_name ::= BIGINT */ + { 270, -1 }, /* (143) type_name ::= FLOAT */ + { 270, -1 }, /* (144) type_name ::= DOUBLE */ + { 270, -4 }, /* (145) type_name ::= BINARY NK_LP NK_INTEGER NK_RP */ + { 270, -1 }, /* (146) type_name ::= TIMESTAMP */ + { 270, -4 }, /* (147) type_name ::= NCHAR NK_LP NK_INTEGER NK_RP */ + { 270, -2 }, /* (148) type_name ::= TINYINT UNSIGNED */ + { 270, -2 }, /* (149) type_name ::= SMALLINT UNSIGNED */ + { 270, -2 }, /* (150) type_name ::= INT UNSIGNED */ + { 270, -2 }, /* (151) type_name ::= BIGINT UNSIGNED */ + { 270, -1 }, /* (152) type_name ::= JSON */ + { 270, -4 }, /* (153) type_name ::= VARCHAR NK_LP NK_INTEGER NK_RP */ + { 270, -1 }, /* (154) type_name ::= MEDIUMBLOB */ + { 270, -1 }, /* (155) type_name ::= BLOB */ + { 270, -4 }, /* (156) type_name ::= VARBINARY NK_LP NK_INTEGER NK_RP */ + { 270, -1 }, /* (157) type_name ::= DECIMAL */ + { 270, -4 }, /* (158) type_name ::= DECIMAL NK_LP NK_INTEGER NK_RP */ + { 270, -6 }, /* (159) type_name ::= DECIMAL NK_LP NK_INTEGER NK_COMMA NK_INTEGER NK_RP */ + { 262, 0 }, /* (160) tags_def_opt ::= */ + { 262, -1 }, /* (161) tags_def_opt ::= tags_def */ + { 265, -4 }, /* (162) tags_def ::= TAGS NK_LP column_def_list NK_RP */ + { 263, 0 }, /* (163) table_options ::= */ + { 263, -3 }, /* (164) table_options ::= table_options COMMENT NK_STRING */ + { 263, -3 }, /* (165) table_options ::= table_options MAX_DELAY duration_list */ + { 263, -3 }, /* (166) table_options ::= table_options WATERMARK duration_list */ + { 263, -5 }, /* (167) table_options ::= table_options ROLLUP NK_LP rollup_func_list NK_RP */ + { 263, -3 }, /* (168) table_options ::= table_options TTL NK_INTEGER */ + { 263, -5 }, /* (169) table_options ::= table_options SMA NK_LP col_name_list NK_RP */ + { 268, -1 }, /* (170) alter_table_options ::= alter_table_option */ + { 268, -2 }, /* (171) alter_table_options ::= alter_table_options alter_table_option */ + { 281, -2 }, /* (172) alter_table_option ::= COMMENT NK_STRING */ + { 281, -2 }, /* (173) alter_table_option ::= TTL NK_INTEGER */ + { 279, -1 }, /* (174) duration_list ::= duration_literal */ + { 279, -3 }, /* (175) duration_list ::= duration_list NK_COMMA duration_literal */ + { 280, -1 }, /* (176) rollup_func_list ::= rollup_func_name */ + { 280, -3 }, /* (177) rollup_func_list ::= rollup_func_list NK_COMMA rollup_func_name */ + { 283, -1 }, /* (178) rollup_func_name ::= function_name */ + { 283, -1 }, /* (179) rollup_func_name ::= FIRST */ + { 283, -1 }, /* (180) rollup_func_name ::= LAST */ + { 276, -1 }, /* (181) col_name_list ::= col_name */ + { 276, -3 }, /* (182) col_name_list ::= col_name_list NK_COMMA col_name */ + { 285, -1 }, /* (183) col_name ::= column_name */ + { 239, -2 }, /* (184) cmd ::= SHOW DNODES */ + { 239, -2 }, /* (185) cmd ::= SHOW USERS */ + { 239, -2 }, /* (186) cmd ::= SHOW DATABASES */ + { 239, -4 }, /* (187) cmd ::= SHOW db_name_cond_opt TABLES like_pattern_opt */ + { 239, -4 }, /* (188) cmd ::= SHOW db_name_cond_opt STABLES like_pattern_opt */ + { 239, -3 }, /* (189) cmd ::= SHOW db_name_cond_opt VGROUPS */ + { 239, -2 }, /* (190) cmd ::= SHOW MNODES */ + { 239, -2 }, /* (191) cmd ::= SHOW MODULES */ + { 239, -2 }, /* (192) cmd ::= SHOW QNODES */ + { 239, -2 }, /* (193) cmd ::= SHOW FUNCTIONS */ + { 239, -5 }, /* (194) cmd ::= SHOW INDEXES FROM table_name_cond from_db_opt */ + { 239, -2 }, /* (195) cmd ::= SHOW STREAMS */ + { 239, -2 }, /* (196) cmd ::= SHOW ACCOUNTS */ + { 239, -2 }, /* (197) cmd ::= SHOW APPS */ + { 239, -2 }, /* (198) cmd ::= SHOW CONNECTIONS */ + { 239, -2 }, /* (199) cmd ::= SHOW LICENCE */ + { 239, -2 }, /* (200) cmd ::= SHOW GRANTS */ + { 239, -4 }, /* (201) cmd ::= SHOW CREATE DATABASE db_name */ + { 239, -4 }, /* (202) cmd ::= SHOW CREATE TABLE full_table_name */ + { 239, -4 }, /* (203) cmd ::= SHOW CREATE STABLE full_table_name */ + { 239, -2 }, /* (204) cmd ::= SHOW QUERIES */ + { 239, -2 }, /* (205) cmd ::= SHOW SCORES */ + { 239, -2 }, /* (206) cmd ::= SHOW TOPICS */ + { 239, -2 }, /* (207) cmd ::= SHOW VARIABLES */ + { 239, -2 }, /* (208) cmd ::= SHOW BNODES */ + { 239, -2 }, /* (209) cmd ::= SHOW SNODES */ + { 239, -2 }, /* (210) cmd ::= SHOW CLUSTER */ + { 239, -2 }, /* (211) cmd ::= SHOW TRANSACTIONS */ + { 286, 0 }, /* (212) db_name_cond_opt ::= */ + { 286, -2 }, /* (213) db_name_cond_opt ::= db_name NK_DOT */ + { 287, 0 }, /* (214) like_pattern_opt ::= */ + { 287, -2 }, /* (215) like_pattern_opt ::= LIKE NK_STRING */ + { 288, -1 }, /* (216) table_name_cond ::= table_name */ + { 289, 0 }, /* (217) from_db_opt ::= */ + { 289, -2 }, /* (218) from_db_opt ::= FROM db_name */ + { 239, -8 }, /* (219) cmd ::= CREATE SMA INDEX not_exists_opt index_name ON table_name index_options */ + { 239, -10 }, /* (220) cmd ::= CREATE FULLTEXT INDEX not_exists_opt index_name ON table_name NK_LP col_name_list NK_RP */ + { 239, -6 }, /* (221) cmd ::= DROP INDEX exists_opt index_name ON table_name */ + { 291, 0 }, /* (222) index_options ::= */ + { 291, -9 }, /* (223) index_options ::= FUNCTION NK_LP func_list NK_RP INTERVAL NK_LP duration_literal NK_RP sliding_opt */ + { 291, -11 }, /* (224) index_options ::= FUNCTION NK_LP func_list NK_RP INTERVAL NK_LP duration_literal NK_COMMA duration_literal NK_RP sliding_opt */ + { 292, -1 }, /* (225) func_list ::= func */ + { 292, -3 }, /* (226) func_list ::= func_list NK_COMMA func */ + { 294, -4 }, /* (227) func ::= function_name NK_LP expression_list NK_RP */ + { 239, -6 }, /* (228) cmd ::= CREATE TOPIC not_exists_opt topic_name AS query_expression */ + { 239, -7 }, /* (229) cmd ::= CREATE TOPIC not_exists_opt topic_name AS DATABASE db_name */ + { 239, -7 }, /* (230) cmd ::= CREATE TOPIC not_exists_opt topic_name AS STABLE full_table_name */ + { 239, -4 }, /* (231) cmd ::= DROP TOPIC exists_opt topic_name */ + { 239, -7 }, /* (232) cmd ::= DROP CONSUMER GROUP exists_opt cgroup_name ON topic_name */ + { 239, -2 }, /* (233) cmd ::= DESC full_table_name */ + { 239, -2 }, /* (234) cmd ::= DESCRIBE full_table_name */ + { 239, -3 }, /* (235) cmd ::= RESET QUERY CACHE */ + { 239, -4 }, /* (236) cmd ::= EXPLAIN analyze_opt explain_options query_expression */ + { 299, 0 }, /* (237) analyze_opt ::= */ + { 299, -1 }, /* (238) analyze_opt ::= ANALYZE */ + { 300, 0 }, /* (239) explain_options ::= */ + { 300, -3 }, /* (240) explain_options ::= explain_options VERBOSE NK_BOOL */ + { 300, -3 }, /* (241) explain_options ::= explain_options RATIO NK_FLOAT */ + { 239, -6 }, /* (242) cmd ::= COMPACT VNODES IN NK_LP integer_list NK_RP */ + { 239, -10 }, /* (243) cmd ::= CREATE agg_func_opt FUNCTION not_exists_opt function_name AS NK_STRING OUTPUTTYPE type_name bufsize_opt */ + { 239, -4 }, /* (244) cmd ::= DROP FUNCTION exists_opt function_name */ + { 301, 0 }, /* (245) agg_func_opt ::= */ + { 301, -1 }, /* (246) agg_func_opt ::= AGGREGATE */ + { 302, 0 }, /* (247) bufsize_opt ::= */ + { 302, -2 }, /* (248) bufsize_opt ::= BUFSIZE NK_INTEGER */ + { 239, -8 }, /* (249) cmd ::= CREATE STREAM not_exists_opt stream_name stream_options into_opt AS query_expression */ + { 239, -4 }, /* (250) cmd ::= DROP STREAM exists_opt stream_name */ + { 305, 0 }, /* (251) into_opt ::= */ + { 305, -2 }, /* (252) into_opt ::= INTO full_table_name */ + { 304, 0 }, /* (253) stream_options ::= */ + { 304, -3 }, /* (254) stream_options ::= stream_options TRIGGER AT_ONCE */ + { 304, -3 }, /* (255) stream_options ::= stream_options TRIGGER WINDOW_CLOSE */ + { 304, -4 }, /* (256) stream_options ::= stream_options TRIGGER MAX_DELAY duration_literal */ + { 304, -3 }, /* (257) stream_options ::= stream_options WATERMARK duration_literal */ + { 239, -3 }, /* (258) cmd ::= KILL CONNECTION NK_INTEGER */ + { 239, -3 }, /* (259) cmd ::= KILL QUERY NK_STRING */ + { 239, -3 }, /* (260) cmd ::= KILL TRANSACTION NK_INTEGER */ + { 239, -2 }, /* (261) cmd ::= BALANCE VGROUP */ + { 239, -4 }, /* (262) cmd ::= MERGE VGROUP NK_INTEGER NK_INTEGER */ + { 239, -4 }, /* (263) cmd ::= REDISTRIBUTE VGROUP NK_INTEGER dnode_list */ + { 239, -3 }, /* (264) cmd ::= SPLIT VGROUP NK_INTEGER */ + { 306, -2 }, /* (265) dnode_list ::= DNODE NK_INTEGER */ + { 306, -3 }, /* (266) dnode_list ::= dnode_list DNODE NK_INTEGER */ + { 239, -3 }, /* (267) cmd ::= SYNCDB db_name REPLICA */ + { 239, -4 }, /* (268) cmd ::= DELETE FROM full_table_name where_clause_opt */ + { 239, -1 }, /* (269) cmd ::= query_expression */ + { 242, -1 }, /* (270) literal ::= NK_INTEGER */ + { 242, -1 }, /* (271) literal ::= NK_FLOAT */ + { 242, -1 }, /* (272) literal ::= NK_STRING */ + { 242, -1 }, /* (273) literal ::= NK_BOOL */ + { 242, -2 }, /* (274) literal ::= TIMESTAMP NK_STRING */ + { 242, -1 }, /* (275) literal ::= duration_literal */ + { 242, -1 }, /* (276) literal ::= NULL */ + { 242, -1 }, /* (277) literal ::= NK_QUESTION */ + { 282, -1 }, /* (278) duration_literal ::= NK_VARIABLE */ + { 308, -1 }, /* (279) signed ::= NK_INTEGER */ + { 308, -2 }, /* (280) signed ::= NK_PLUS NK_INTEGER */ + { 308, -2 }, /* (281) signed ::= NK_MINUS NK_INTEGER */ + { 308, -1 }, /* (282) signed ::= NK_FLOAT */ + { 308, -2 }, /* (283) signed ::= NK_PLUS NK_FLOAT */ + { 308, -2 }, /* (284) signed ::= NK_MINUS NK_FLOAT */ + { 271, -1 }, /* (285) signed_literal ::= signed */ + { 271, -1 }, /* (286) signed_literal ::= NK_STRING */ + { 271, -1 }, /* (287) signed_literal ::= NK_BOOL */ + { 271, -2 }, /* (288) signed_literal ::= TIMESTAMP NK_STRING */ + { 271, -1 }, /* (289) signed_literal ::= duration_literal */ + { 271, -1 }, /* (290) signed_literal ::= NULL */ + { 271, -1 }, /* (291) signed_literal ::= literal_func */ + { 274, -1 }, /* (292) literal_list ::= signed_literal */ + { 274, -3 }, /* (293) literal_list ::= literal_list NK_COMMA signed_literal */ + { 249, -1 }, /* (294) db_name ::= NK_ID */ + { 277, -1 }, /* (295) table_name ::= NK_ID */ + { 269, -1 }, /* (296) column_name ::= NK_ID */ + { 284, -1 }, /* (297) function_name ::= NK_ID */ + { 310, -1 }, /* (298) table_alias ::= NK_ID */ + { 311, -1 }, /* (299) column_alias ::= NK_ID */ + { 244, -1 }, /* (300) user_name ::= NK_ID */ + { 290, -1 }, /* (301) index_name ::= NK_ID */ + { 296, -1 }, /* (302) topic_name ::= NK_ID */ + { 303, -1 }, /* (303) stream_name ::= NK_ID */ + { 298, -1 }, /* (304) cgroup_name ::= NK_ID */ + { 312, -1 }, /* (305) expression ::= literal */ + { 312, -1 }, /* (306) expression ::= pseudo_column */ + { 312, -1 }, /* (307) expression ::= column_reference */ + { 312, -1 }, /* (308) expression ::= function_expression */ + { 312, -1 }, /* (309) expression ::= subquery */ + { 312, -3 }, /* (310) expression ::= NK_LP expression NK_RP */ + { 312, -2 }, /* (311) expression ::= NK_PLUS expression */ + { 312, -2 }, /* (312) expression ::= NK_MINUS expression */ + { 312, -3 }, /* (313) expression ::= expression NK_PLUS expression */ + { 312, -3 }, /* (314) expression ::= expression NK_MINUS expression */ + { 312, -3 }, /* (315) expression ::= expression NK_STAR expression */ + { 312, -3 }, /* (316) expression ::= expression NK_SLASH expression */ + { 312, -3 }, /* (317) expression ::= expression NK_REM expression */ + { 312, -3 }, /* (318) expression ::= column_reference NK_ARROW NK_STRING */ + { 295, -1 }, /* (319) expression_list ::= expression */ + { 295, -3 }, /* (320) expression_list ::= expression_list NK_COMMA expression */ + { 314, -1 }, /* (321) column_reference ::= column_name */ + { 314, -3 }, /* (322) column_reference ::= table_name NK_DOT column_name */ + { 313, -1 }, /* (323) pseudo_column ::= ROWTS */ + { 313, -1 }, /* (324) pseudo_column ::= TBNAME */ + { 313, -3 }, /* (325) pseudo_column ::= table_name NK_DOT TBNAME */ + { 313, -1 }, /* (326) pseudo_column ::= QSTARTTS */ + { 313, -1 }, /* (327) pseudo_column ::= QENDTS */ + { 313, -1 }, /* (328) pseudo_column ::= WSTARTTS */ + { 313, -1 }, /* (329) pseudo_column ::= WENDTS */ + { 313, -1 }, /* (330) pseudo_column ::= WDURATION */ + { 315, -4 }, /* (331) function_expression ::= function_name NK_LP expression_list NK_RP */ + { 315, -4 }, /* (332) function_expression ::= star_func NK_LP star_func_para_list NK_RP */ + { 315, -6 }, /* (333) function_expression ::= CAST NK_LP expression AS type_name NK_RP */ + { 315, -1 }, /* (334) function_expression ::= literal_func */ + { 309, -3 }, /* (335) literal_func ::= noarg_func NK_LP NK_RP */ + { 309, -1 }, /* (336) literal_func ::= NOW */ + { 319, -1 }, /* (337) noarg_func ::= NOW */ + { 319, -1 }, /* (338) noarg_func ::= TODAY */ + { 319, -1 }, /* (339) noarg_func ::= TIMEZONE */ + { 317, -1 }, /* (340) star_func ::= COUNT */ + { 317, -1 }, /* (341) star_func ::= FIRST */ + { 317, -1 }, /* (342) star_func ::= LAST */ + { 317, -1 }, /* (343) star_func ::= LAST_ROW */ + { 318, -1 }, /* (344) star_func_para_list ::= NK_STAR */ + { 318, -1 }, /* (345) star_func_para_list ::= other_para_list */ + { 320, -1 }, /* (346) other_para_list ::= star_func_para */ + { 320, -3 }, /* (347) other_para_list ::= other_para_list NK_COMMA star_func_para */ + { 321, -1 }, /* (348) star_func_para ::= expression */ + { 321, -3 }, /* (349) star_func_para ::= table_name NK_DOT NK_STAR */ + { 322, -3 }, /* (350) predicate ::= expression compare_op expression */ + { 322, -5 }, /* (351) predicate ::= expression BETWEEN expression AND expression */ + { 322, -6 }, /* (352) predicate ::= expression NOT BETWEEN expression AND expression */ + { 322, -3 }, /* (353) predicate ::= expression IS NULL */ + { 322, -4 }, /* (354) predicate ::= expression IS NOT NULL */ + { 322, -3 }, /* (355) predicate ::= expression in_op in_predicate_value */ + { 323, -1 }, /* (356) compare_op ::= NK_LT */ + { 323, -1 }, /* (357) compare_op ::= NK_GT */ + { 323, -1 }, /* (358) compare_op ::= NK_LE */ + { 323, -1 }, /* (359) compare_op ::= NK_GE */ + { 323, -1 }, /* (360) compare_op ::= NK_NE */ + { 323, -1 }, /* (361) compare_op ::= NK_EQ */ + { 323, -1 }, /* (362) compare_op ::= LIKE */ + { 323, -2 }, /* (363) compare_op ::= NOT LIKE */ + { 323, -1 }, /* (364) compare_op ::= MATCH */ + { 323, -1 }, /* (365) compare_op ::= NMATCH */ + { 323, -1 }, /* (366) compare_op ::= CONTAINS */ + { 324, -1 }, /* (367) in_op ::= IN */ + { 324, -2 }, /* (368) in_op ::= NOT IN */ + { 325, -3 }, /* (369) in_predicate_value ::= NK_LP expression_list NK_RP */ + { 326, -1 }, /* (370) boolean_value_expression ::= boolean_primary */ + { 326, -2 }, /* (371) boolean_value_expression ::= NOT boolean_primary */ + { 326, -3 }, /* (372) boolean_value_expression ::= boolean_value_expression OR boolean_value_expression */ + { 326, -3 }, /* (373) boolean_value_expression ::= boolean_value_expression AND boolean_value_expression */ + { 327, -1 }, /* (374) boolean_primary ::= predicate */ + { 327, -3 }, /* (375) boolean_primary ::= NK_LP boolean_value_expression NK_RP */ + { 328, -1 }, /* (376) common_expression ::= expression */ + { 328, -1 }, /* (377) common_expression ::= boolean_value_expression */ + { 329, -2 }, /* (378) from_clause ::= FROM table_reference_list */ + { 330, -1 }, /* (379) table_reference_list ::= table_reference */ + { 330, -3 }, /* (380) table_reference_list ::= table_reference_list NK_COMMA table_reference */ + { 331, -1 }, /* (381) table_reference ::= table_primary */ + { 331, -1 }, /* (382) table_reference ::= joined_table */ + { 332, -2 }, /* (383) table_primary ::= table_name alias_opt */ + { 332, -4 }, /* (384) table_primary ::= db_name NK_DOT table_name alias_opt */ + { 332, -2 }, /* (385) table_primary ::= subquery alias_opt */ + { 332, -1 }, /* (386) table_primary ::= parenthesized_joined_table */ + { 334, 0 }, /* (387) alias_opt ::= */ + { 334, -1 }, /* (388) alias_opt ::= table_alias */ + { 334, -2 }, /* (389) alias_opt ::= AS table_alias */ + { 335, -3 }, /* (390) parenthesized_joined_table ::= NK_LP joined_table NK_RP */ + { 335, -3 }, /* (391) parenthesized_joined_table ::= NK_LP parenthesized_joined_table NK_RP */ + { 333, -6 }, /* (392) joined_table ::= table_reference join_type JOIN table_reference ON search_condition */ + { 336, 0 }, /* (393) join_type ::= */ + { 336, -1 }, /* (394) join_type ::= INNER */ + { 338, -9 }, /* (395) query_specification ::= SELECT set_quantifier_opt select_list from_clause where_clause_opt partition_by_clause_opt twindow_clause_opt group_by_clause_opt having_clause_opt */ + { 339, 0 }, /* (396) set_quantifier_opt ::= */ + { 339, -1 }, /* (397) set_quantifier_opt ::= DISTINCT */ + { 339, -1 }, /* (398) set_quantifier_opt ::= ALL */ + { 340, -1 }, /* (399) select_list ::= NK_STAR */ + { 340, -1 }, /* (400) select_list ::= select_sublist */ + { 345, -1 }, /* (401) select_sublist ::= select_item */ + { 345, -3 }, /* (402) select_sublist ::= select_sublist NK_COMMA select_item */ + { 346, -1 }, /* (403) select_item ::= common_expression */ + { 346, -2 }, /* (404) select_item ::= common_expression column_alias */ + { 346, -3 }, /* (405) select_item ::= common_expression AS column_alias */ + { 346, -3 }, /* (406) select_item ::= table_name NK_DOT NK_STAR */ + { 307, 0 }, /* (407) where_clause_opt ::= */ + { 307, -2 }, /* (408) where_clause_opt ::= WHERE search_condition */ + { 341, 0 }, /* (409) partition_by_clause_opt ::= */ + { 341, -3 }, /* (410) partition_by_clause_opt ::= PARTITION BY expression_list */ + { 342, 0 }, /* (411) twindow_clause_opt ::= */ + { 342, -6 }, /* (412) twindow_clause_opt ::= SESSION NK_LP column_reference NK_COMMA duration_literal NK_RP */ + { 342, -4 }, /* (413) twindow_clause_opt ::= STATE_WINDOW NK_LP expression NK_RP */ + { 342, -6 }, /* (414) twindow_clause_opt ::= INTERVAL NK_LP duration_literal NK_RP sliding_opt fill_opt */ + { 342, -8 }, /* (415) twindow_clause_opt ::= INTERVAL NK_LP duration_literal NK_COMMA duration_literal NK_RP sliding_opt fill_opt */ + { 293, 0 }, /* (416) sliding_opt ::= */ + { 293, -4 }, /* (417) sliding_opt ::= SLIDING NK_LP duration_literal NK_RP */ + { 347, 0 }, /* (418) fill_opt ::= */ + { 347, -4 }, /* (419) fill_opt ::= FILL NK_LP fill_mode NK_RP */ + { 347, -6 }, /* (420) fill_opt ::= FILL NK_LP VALUE NK_COMMA literal_list NK_RP */ + { 348, -1 }, /* (421) fill_mode ::= NONE */ + { 348, -1 }, /* (422) fill_mode ::= PREV */ + { 348, -1 }, /* (423) fill_mode ::= NULL */ + { 348, -1 }, /* (424) fill_mode ::= LINEAR */ + { 348, -1 }, /* (425) fill_mode ::= NEXT */ + { 343, 0 }, /* (426) group_by_clause_opt ::= */ + { 343, -3 }, /* (427) group_by_clause_opt ::= GROUP BY group_by_list */ + { 349, -1 }, /* (428) group_by_list ::= expression */ + { 349, -3 }, /* (429) group_by_list ::= group_by_list NK_COMMA expression */ + { 344, 0 }, /* (430) having_clause_opt ::= */ + { 344, -2 }, /* (431) having_clause_opt ::= HAVING search_condition */ + { 297, -4 }, /* (432) query_expression ::= query_expression_body order_by_clause_opt slimit_clause_opt limit_clause_opt */ + { 350, -1 }, /* (433) query_expression_body ::= query_primary */ + { 350, -4 }, /* (434) query_expression_body ::= query_expression_body UNION ALL query_expression_body */ + { 350, -3 }, /* (435) query_expression_body ::= query_expression_body UNION query_expression_body */ + { 354, -1 }, /* (436) query_primary ::= query_specification */ + { 354, -6 }, /* (437) query_primary ::= NK_LP query_expression_body order_by_clause_opt slimit_clause_opt limit_clause_opt NK_RP */ + { 351, 0 }, /* (438) order_by_clause_opt ::= */ + { 351, -3 }, /* (439) order_by_clause_opt ::= ORDER BY sort_specification_list */ + { 352, 0 }, /* (440) slimit_clause_opt ::= */ + { 352, -2 }, /* (441) slimit_clause_opt ::= SLIMIT NK_INTEGER */ + { 352, -4 }, /* (442) slimit_clause_opt ::= SLIMIT NK_INTEGER SOFFSET NK_INTEGER */ + { 352, -4 }, /* (443) slimit_clause_opt ::= SLIMIT NK_INTEGER NK_COMMA NK_INTEGER */ + { 353, 0 }, /* (444) limit_clause_opt ::= */ + { 353, -2 }, /* (445) limit_clause_opt ::= LIMIT NK_INTEGER */ + { 353, -4 }, /* (446) limit_clause_opt ::= LIMIT NK_INTEGER OFFSET NK_INTEGER */ + { 353, -4 }, /* (447) limit_clause_opt ::= LIMIT NK_INTEGER NK_COMMA NK_INTEGER */ + { 316, -3 }, /* (448) subquery ::= NK_LP query_expression NK_RP */ + { 337, -1 }, /* (449) search_condition ::= common_expression */ + { 355, -1 }, /* (450) sort_specification_list ::= sort_specification */ + { 355, -3 }, /* (451) sort_specification_list ::= sort_specification_list NK_COMMA sort_specification */ + { 356, -3 }, /* (452) sort_specification ::= expression ordering_specification_opt null_ordering_opt */ + { 357, 0 }, /* (453) ordering_specification_opt ::= */ + { 357, -1 }, /* (454) ordering_specification_opt ::= ASC */ + { 357, -1 }, /* (455) ordering_specification_opt ::= DESC */ + { 358, 0 }, /* (456) null_ordering_opt ::= */ + { 358, -2 }, /* (457) null_ordering_opt ::= NULLS FIRST */ + { 358, -2 }, /* (458) null_ordering_opt ::= NULLS LAST */ }; static void yy_accept(yyParser*); /* Forward Declaration */ @@ -3117,11 +3112,11 @@ static YYACTIONTYPE yy_reduce( YYMINORTYPE yylhsminor; case 0: /* cmd ::= CREATE ACCOUNT NK_ID PASS NK_STRING account_options */ { pCxt->errCode = generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_EXPRIE_STATEMENT); } - yy_destructor(yypParser,241,&yymsp[0].minor); + yy_destructor(yypParser,240,&yymsp[0].minor); break; case 1: /* cmd ::= ALTER ACCOUNT NK_ID alter_account_options */ { pCxt->errCode = generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_EXPRIE_STATEMENT); } - yy_destructor(yypParser,242,&yymsp[0].minor); + yy_destructor(yypParser,241,&yymsp[0].minor); break; case 2: /* account_options ::= */ { } @@ -3135,20 +3130,20 @@ static YYACTIONTYPE yy_reduce( case 9: /* account_options ::= account_options USERS literal */ yytestcase(yyruleno==9); case 10: /* account_options ::= account_options CONNS literal */ yytestcase(yyruleno==10); case 11: /* account_options ::= account_options STATE literal */ yytestcase(yyruleno==11); -{ yy_destructor(yypParser,241,&yymsp[-2].minor); +{ yy_destructor(yypParser,240,&yymsp[-2].minor); { } - yy_destructor(yypParser,243,&yymsp[0].minor); + yy_destructor(yypParser,242,&yymsp[0].minor); } break; case 12: /* alter_account_options ::= alter_account_option */ -{ yy_destructor(yypParser,244,&yymsp[0].minor); +{ yy_destructor(yypParser,243,&yymsp[0].minor); { } } break; case 13: /* alter_account_options ::= alter_account_options alter_account_option */ -{ yy_destructor(yypParser,242,&yymsp[-1].minor); +{ yy_destructor(yypParser,241,&yymsp[-1].minor); { } - yy_destructor(yypParser,244,&yymsp[0].minor); + yy_destructor(yypParser,243,&yymsp[0].minor); } break; case 14: /* alter_account_option ::= PASS literal */ @@ -3162,63 +3157,63 @@ static YYACTIONTYPE yy_reduce( case 22: /* alter_account_option ::= CONNS literal */ yytestcase(yyruleno==22); case 23: /* alter_account_option ::= STATE literal */ yytestcase(yyruleno==23); { } - yy_destructor(yypParser,243,&yymsp[0].minor); + yy_destructor(yypParser,242,&yymsp[0].minor); break; case 24: /* cmd ::= CREATE USER user_name PASS NK_STRING */ -{ pCxt->pRootNode = createCreateUserStmt(pCxt, &yymsp[-2].minor.yy209, &yymsp[0].minor.yy0); } +{ pCxt->pRootNode = createCreateUserStmt(pCxt, &yymsp[-2].minor.yy421, &yymsp[0].minor.yy0); } break; case 25: /* cmd ::= ALTER USER user_name PASS NK_STRING */ -{ pCxt->pRootNode = createAlterUserStmt(pCxt, &yymsp[-2].minor.yy209, TSDB_ALTER_USER_PASSWD, &yymsp[0].minor.yy0); } +{ pCxt->pRootNode = createAlterUserStmt(pCxt, &yymsp[-2].minor.yy421, TSDB_ALTER_USER_PASSWD, &yymsp[0].minor.yy0); } break; case 26: /* cmd ::= ALTER USER user_name PRIVILEGE NK_STRING */ -{ pCxt->pRootNode = createAlterUserStmt(pCxt, &yymsp[-2].minor.yy209, TSDB_ALTER_USER_PRIVILEGES, &yymsp[0].minor.yy0); } +{ pCxt->pRootNode = createAlterUserStmt(pCxt, &yymsp[-2].minor.yy421, TSDB_ALTER_USER_PRIVILEGES, &yymsp[0].minor.yy0); } break; case 27: /* cmd ::= DROP USER user_name */ -{ pCxt->pRootNode = createDropUserStmt(pCxt, &yymsp[0].minor.yy209); } +{ pCxt->pRootNode = createDropUserStmt(pCxt, &yymsp[0].minor.yy421); } break; case 28: /* cmd ::= GRANT privileges ON priv_level TO user_name */ -{ pCxt->pRootNode = createGrantStmt(pCxt, yymsp[-4].minor.yy189, &yymsp[-2].minor.yy209, &yymsp[0].minor.yy209); } +{ pCxt->pRootNode = createGrantStmt(pCxt, yymsp[-4].minor.yy669, &yymsp[-2].minor.yy421, &yymsp[0].minor.yy421); } break; case 29: /* cmd ::= REVOKE privileges ON priv_level FROM user_name */ -{ pCxt->pRootNode = createRevokeStmt(pCxt, yymsp[-4].minor.yy189, &yymsp[-2].minor.yy209, &yymsp[0].minor.yy209); } +{ pCxt->pRootNode = createRevokeStmt(pCxt, yymsp[-4].minor.yy669, &yymsp[-2].minor.yy421, &yymsp[0].minor.yy421); } break; case 30: /* privileges ::= ALL */ -{ yymsp[0].minor.yy189 = PRIVILEGE_TYPE_ALL; } +{ yymsp[0].minor.yy669 = PRIVILEGE_TYPE_ALL; } break; case 31: /* privileges ::= priv_type_list */ case 32: /* priv_type_list ::= priv_type */ yytestcase(yyruleno==32); -{ yylhsminor.yy189 = yymsp[0].minor.yy189; } - yymsp[0].minor.yy189 = yylhsminor.yy189; +{ yylhsminor.yy669 = yymsp[0].minor.yy669; } + yymsp[0].minor.yy669 = yylhsminor.yy669; break; case 33: /* priv_type_list ::= priv_type_list NK_COMMA priv_type */ -{ yylhsminor.yy189 = yymsp[-2].minor.yy189 | yymsp[0].minor.yy189; } - yymsp[-2].minor.yy189 = yylhsminor.yy189; +{ yylhsminor.yy669 = yymsp[-2].minor.yy669 | yymsp[0].minor.yy669; } + yymsp[-2].minor.yy669 = yylhsminor.yy669; break; case 34: /* priv_type ::= READ */ -{ yymsp[0].minor.yy189 = PRIVILEGE_TYPE_READ; } +{ yymsp[0].minor.yy669 = PRIVILEGE_TYPE_READ; } break; case 35: /* priv_type ::= WRITE */ -{ yymsp[0].minor.yy189 = PRIVILEGE_TYPE_WRITE; } +{ yymsp[0].minor.yy669 = PRIVILEGE_TYPE_WRITE; } break; case 36: /* priv_level ::= NK_STAR NK_DOT NK_STAR */ -{ yylhsminor.yy209 = yymsp[-2].minor.yy0; } - yymsp[-2].minor.yy209 = yylhsminor.yy209; +{ yylhsminor.yy421 = yymsp[-2].minor.yy0; } + yymsp[-2].minor.yy421 = yylhsminor.yy421; break; case 37: /* priv_level ::= db_name NK_DOT NK_STAR */ -{ yylhsminor.yy209 = yymsp[-2].minor.yy209; } - yymsp[-2].minor.yy209 = yylhsminor.yy209; +{ yylhsminor.yy421 = yymsp[-2].minor.yy421; } + yymsp[-2].minor.yy421 = yylhsminor.yy421; break; case 38: /* cmd ::= CREATE DNODE dnode_endpoint */ -{ pCxt->pRootNode = createCreateDnodeStmt(pCxt, &yymsp[0].minor.yy209, NULL); } +{ pCxt->pRootNode = createCreateDnodeStmt(pCxt, &yymsp[0].minor.yy421, NULL); } break; - case 39: /* cmd ::= CREATE DNODE dnode_host_name PORT NK_INTEGER */ -{ pCxt->pRootNode = createCreateDnodeStmt(pCxt, &yymsp[-2].minor.yy209, &yymsp[0].minor.yy0); } + case 39: /* cmd ::= CREATE DNODE dnode_endpoint PORT NK_INTEGER */ +{ pCxt->pRootNode = createCreateDnodeStmt(pCxt, &yymsp[-2].minor.yy421, &yymsp[0].minor.yy0); } break; case 40: /* cmd ::= DROP DNODE NK_INTEGER */ { pCxt->pRootNode = createDropDnodeStmt(pCxt, &yymsp[0].minor.yy0); } break; case 41: /* cmd ::= DROP DNODE dnode_endpoint */ -{ pCxt->pRootNode = createDropDnodeStmt(pCxt, &yymsp[0].minor.yy209); } +{ pCxt->pRootNode = createDropDnodeStmt(pCxt, &yymsp[0].minor.yy421); } break; case 42: /* cmd ::= ALTER DNODE NK_INTEGER NK_STRING */ { pCxt->pRootNode = createAlterDnodeStmt(pCxt, &yymsp[-1].minor.yy0, &yymsp[0].minor.yy0, NULL); } @@ -3233,28 +3228,28 @@ static YYACTIONTYPE yy_reduce( { pCxt->pRootNode = createAlterDnodeStmt(pCxt, NULL, &yymsp[-1].minor.yy0, &yymsp[0].minor.yy0); } break; case 46: /* dnode_endpoint ::= NK_STRING */ - case 47: /* dnode_host_name ::= NK_ID */ yytestcase(yyruleno==47); - case 48: /* dnode_host_name ::= NK_IPTOKEN */ yytestcase(yyruleno==48); - case 289: /* db_name ::= NK_ID */ yytestcase(yyruleno==289); - case 290: /* table_name ::= NK_ID */ yytestcase(yyruleno==290); - case 291: /* column_name ::= NK_ID */ yytestcase(yyruleno==291); - case 292: /* function_name ::= NK_ID */ yytestcase(yyruleno==292); - case 293: /* table_alias ::= NK_ID */ yytestcase(yyruleno==293); - case 294: /* column_alias ::= NK_ID */ yytestcase(yyruleno==294); - case 295: /* user_name ::= NK_ID */ yytestcase(yyruleno==295); - case 296: /* index_name ::= NK_ID */ yytestcase(yyruleno==296); - case 297: /* topic_name ::= NK_ID */ yytestcase(yyruleno==297); - case 298: /* stream_name ::= NK_ID */ yytestcase(yyruleno==298); - case 299: /* cgroup_name ::= NK_ID */ yytestcase(yyruleno==299); - case 332: /* noarg_func ::= NOW */ yytestcase(yyruleno==332); - case 333: /* noarg_func ::= TODAY */ yytestcase(yyruleno==333); - case 334: /* noarg_func ::= TIMEZONE */ yytestcase(yyruleno==334); - case 335: /* star_func ::= COUNT */ yytestcase(yyruleno==335); - case 336: /* star_func ::= FIRST */ yytestcase(yyruleno==336); - case 337: /* star_func ::= LAST */ yytestcase(yyruleno==337); - case 338: /* star_func ::= LAST_ROW */ yytestcase(yyruleno==338); -{ yylhsminor.yy209 = yymsp[0].minor.yy0; } - yymsp[0].minor.yy209 = yylhsminor.yy209; + case 47: /* dnode_endpoint ::= NK_ID */ yytestcase(yyruleno==47); + case 48: /* dnode_endpoint ::= NK_IPTOKEN */ yytestcase(yyruleno==48); + case 294: /* db_name ::= NK_ID */ yytestcase(yyruleno==294); + case 295: /* table_name ::= NK_ID */ yytestcase(yyruleno==295); + case 296: /* column_name ::= NK_ID */ yytestcase(yyruleno==296); + case 297: /* function_name ::= NK_ID */ yytestcase(yyruleno==297); + case 298: /* table_alias ::= NK_ID */ yytestcase(yyruleno==298); + case 299: /* column_alias ::= NK_ID */ yytestcase(yyruleno==299); + case 300: /* user_name ::= NK_ID */ yytestcase(yyruleno==300); + case 301: /* index_name ::= NK_ID */ yytestcase(yyruleno==301); + case 302: /* topic_name ::= NK_ID */ yytestcase(yyruleno==302); + case 303: /* stream_name ::= NK_ID */ yytestcase(yyruleno==303); + case 304: /* cgroup_name ::= NK_ID */ yytestcase(yyruleno==304); + case 337: /* noarg_func ::= NOW */ yytestcase(yyruleno==337); + case 338: /* noarg_func ::= TODAY */ yytestcase(yyruleno==338); + case 339: /* noarg_func ::= TIMEZONE */ yytestcase(yyruleno==339); + case 340: /* star_func ::= COUNT */ yytestcase(yyruleno==340); + case 341: /* star_func ::= FIRST */ yytestcase(yyruleno==341); + case 342: /* star_func ::= LAST */ yytestcase(yyruleno==342); + case 343: /* star_func ::= LAST_ROW */ yytestcase(yyruleno==343); +{ yylhsminor.yy421 = yymsp[0].minor.yy0; } + yymsp[0].minor.yy421 = yylhsminor.yy421; break; case 49: /* cmd ::= ALTER LOCAL NK_STRING */ { pCxt->pRootNode = createAlterLocalStmt(pCxt, &yymsp[0].minor.yy0, NULL); } @@ -3287,1155 +3282,1166 @@ static YYACTIONTYPE yy_reduce( { pCxt->pRootNode = createDropComponentNodeStmt(pCxt, QUERY_NODE_DROP_MNODE_STMT, &yymsp[0].minor.yy0); } break; case 59: /* cmd ::= CREATE DATABASE not_exists_opt db_name db_options */ -{ pCxt->pRootNode = createCreateDatabaseStmt(pCxt, yymsp[-2].minor.yy137, &yymsp[-1].minor.yy209, yymsp[0].minor.yy632); } +{ pCxt->pRootNode = createCreateDatabaseStmt(pCxt, yymsp[-2].minor.yy621, &yymsp[-1].minor.yy421, yymsp[0].minor.yy674); } break; case 60: /* cmd ::= DROP DATABASE exists_opt db_name */ -{ pCxt->pRootNode = createDropDatabaseStmt(pCxt, yymsp[-1].minor.yy137, &yymsp[0].minor.yy209); } +{ pCxt->pRootNode = createDropDatabaseStmt(pCxt, yymsp[-1].minor.yy621, &yymsp[0].minor.yy421); } break; case 61: /* cmd ::= USE db_name */ -{ pCxt->pRootNode = createUseDatabaseStmt(pCxt, &yymsp[0].minor.yy209); } +{ pCxt->pRootNode = createUseDatabaseStmt(pCxt, &yymsp[0].minor.yy421); } break; case 62: /* cmd ::= ALTER DATABASE db_name alter_db_options */ -{ pCxt->pRootNode = createAlterDatabaseStmt(pCxt, &yymsp[-1].minor.yy209, yymsp[0].minor.yy632); } +{ pCxt->pRootNode = createAlterDatabaseStmt(pCxt, &yymsp[-1].minor.yy421, yymsp[0].minor.yy674); } break; case 63: /* not_exists_opt ::= IF NOT EXISTS */ -{ yymsp[-2].minor.yy137 = true; } +{ yymsp[-2].minor.yy621 = true; } break; case 64: /* not_exists_opt ::= */ case 66: /* exists_opt ::= */ yytestcase(yyruleno==66); - case 232: /* analyze_opt ::= */ yytestcase(yyruleno==232); - case 240: /* agg_func_opt ::= */ yytestcase(yyruleno==240); - case 391: /* set_quantifier_opt ::= */ yytestcase(yyruleno==391); -{ yymsp[1].minor.yy137 = false; } + case 237: /* analyze_opt ::= */ yytestcase(yyruleno==237); + case 245: /* agg_func_opt ::= */ yytestcase(yyruleno==245); + case 396: /* set_quantifier_opt ::= */ yytestcase(yyruleno==396); +{ yymsp[1].minor.yy621 = false; } break; case 65: /* exists_opt ::= IF EXISTS */ -{ yymsp[-1].minor.yy137 = true; } +{ yymsp[-1].minor.yy621 = true; } break; case 67: /* db_options ::= */ -{ yymsp[1].minor.yy632 = createDefaultDatabaseOptions(pCxt); } +{ yymsp[1].minor.yy674 = createDefaultDatabaseOptions(pCxt); } break; case 68: /* db_options ::= db_options BUFFER NK_INTEGER */ -{ yylhsminor.yy632 = setDatabaseOption(pCxt, yymsp[-2].minor.yy632, DB_OPTION_BUFFER, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = setDatabaseOption(pCxt, yymsp[-2].minor.yy674, DB_OPTION_BUFFER, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; case 69: /* db_options ::= db_options CACHELAST NK_INTEGER */ -{ yylhsminor.yy632 = setDatabaseOption(pCxt, yymsp[-2].minor.yy632, DB_OPTION_CACHELAST, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = setDatabaseOption(pCxt, yymsp[-2].minor.yy674, DB_OPTION_CACHELAST, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; case 70: /* db_options ::= db_options COMP NK_INTEGER */ -{ yylhsminor.yy632 = setDatabaseOption(pCxt, yymsp[-2].minor.yy632, DB_OPTION_COMP, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = setDatabaseOption(pCxt, yymsp[-2].minor.yy674, DB_OPTION_COMP, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; case 71: /* db_options ::= db_options DURATION NK_INTEGER */ case 72: /* db_options ::= db_options DURATION NK_VARIABLE */ yytestcase(yyruleno==72); -{ yylhsminor.yy632 = setDatabaseOption(pCxt, yymsp[-2].minor.yy632, DB_OPTION_DAYS, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = setDatabaseOption(pCxt, yymsp[-2].minor.yy674, DB_OPTION_DAYS, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; case 73: /* db_options ::= db_options FSYNC NK_INTEGER */ -{ yylhsminor.yy632 = setDatabaseOption(pCxt, yymsp[-2].minor.yy632, DB_OPTION_FSYNC, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = setDatabaseOption(pCxt, yymsp[-2].minor.yy674, DB_OPTION_FSYNC, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; case 74: /* db_options ::= db_options MAXROWS NK_INTEGER */ -{ yylhsminor.yy632 = setDatabaseOption(pCxt, yymsp[-2].minor.yy632, DB_OPTION_MAXROWS, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = setDatabaseOption(pCxt, yymsp[-2].minor.yy674, DB_OPTION_MAXROWS, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; case 75: /* db_options ::= db_options MINROWS NK_INTEGER */ -{ yylhsminor.yy632 = setDatabaseOption(pCxt, yymsp[-2].minor.yy632, DB_OPTION_MINROWS, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = setDatabaseOption(pCxt, yymsp[-2].minor.yy674, DB_OPTION_MINROWS, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; case 76: /* db_options ::= db_options KEEP integer_list */ case 77: /* db_options ::= db_options KEEP variable_list */ yytestcase(yyruleno==77); -{ yylhsminor.yy632 = setDatabaseOption(pCxt, yymsp[-2].minor.yy632, DB_OPTION_KEEP, yymsp[0].minor.yy424); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = setDatabaseOption(pCxt, yymsp[-2].minor.yy674, DB_OPTION_KEEP, yymsp[0].minor.yy530); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; case 78: /* db_options ::= db_options PAGES NK_INTEGER */ -{ yylhsminor.yy632 = setDatabaseOption(pCxt, yymsp[-2].minor.yy632, DB_OPTION_PAGES, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = setDatabaseOption(pCxt, yymsp[-2].minor.yy674, DB_OPTION_PAGES, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; case 79: /* db_options ::= db_options PAGESIZE NK_INTEGER */ -{ yylhsminor.yy632 = setDatabaseOption(pCxt, yymsp[-2].minor.yy632, DB_OPTION_PAGESIZE, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = setDatabaseOption(pCxt, yymsp[-2].minor.yy674, DB_OPTION_PAGESIZE, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; case 80: /* db_options ::= db_options PRECISION NK_STRING */ -{ yylhsminor.yy632 = setDatabaseOption(pCxt, yymsp[-2].minor.yy632, DB_OPTION_PRECISION, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = setDatabaseOption(pCxt, yymsp[-2].minor.yy674, DB_OPTION_PRECISION, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; case 81: /* db_options ::= db_options REPLICA NK_INTEGER */ -{ yylhsminor.yy632 = setDatabaseOption(pCxt, yymsp[-2].minor.yy632, DB_OPTION_REPLICA, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = setDatabaseOption(pCxt, yymsp[-2].minor.yy674, DB_OPTION_REPLICA, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; case 82: /* db_options ::= db_options STRICT NK_INTEGER */ -{ yylhsminor.yy632 = setDatabaseOption(pCxt, yymsp[-2].minor.yy632, DB_OPTION_STRICT, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = setDatabaseOption(pCxt, yymsp[-2].minor.yy674, DB_OPTION_STRICT, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; case 83: /* db_options ::= db_options WAL NK_INTEGER */ -{ yylhsminor.yy632 = setDatabaseOption(pCxt, yymsp[-2].minor.yy632, DB_OPTION_WAL, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = setDatabaseOption(pCxt, yymsp[-2].minor.yy674, DB_OPTION_WAL, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; case 84: /* db_options ::= db_options VGROUPS NK_INTEGER */ -{ yylhsminor.yy632 = setDatabaseOption(pCxt, yymsp[-2].minor.yy632, DB_OPTION_VGROUPS, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = setDatabaseOption(pCxt, yymsp[-2].minor.yy674, DB_OPTION_VGROUPS, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; case 85: /* db_options ::= db_options SINGLE_STABLE NK_INTEGER */ -{ yylhsminor.yy632 = setDatabaseOption(pCxt, yymsp[-2].minor.yy632, DB_OPTION_SINGLE_STABLE, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = setDatabaseOption(pCxt, yymsp[-2].minor.yy674, DB_OPTION_SINGLE_STABLE, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; case 86: /* db_options ::= db_options RETENTIONS retention_list */ -{ yylhsminor.yy632 = setDatabaseOption(pCxt, yymsp[-2].minor.yy632, DB_OPTION_RETENTIONS, yymsp[0].minor.yy424); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = setDatabaseOption(pCxt, yymsp[-2].minor.yy674, DB_OPTION_RETENTIONS, yymsp[0].minor.yy530); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; case 87: /* db_options ::= db_options SCHEMALESS NK_INTEGER */ -{ yylhsminor.yy632 = setDatabaseOption(pCxt, yymsp[-2].minor.yy632, DB_OPTION_SCHEMALESS, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = setDatabaseOption(pCxt, yymsp[-2].minor.yy674, DB_OPTION_SCHEMALESS, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; case 88: /* alter_db_options ::= alter_db_option */ -{ yylhsminor.yy632 = createAlterDatabaseOptions(pCxt); yylhsminor.yy632 = setAlterDatabaseOption(pCxt, yylhsminor.yy632, &yymsp[0].minor.yy605); } - yymsp[0].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = createAlterDatabaseOptions(pCxt); yylhsminor.yy674 = setAlterDatabaseOption(pCxt, yylhsminor.yy674, &yymsp[0].minor.yy557); } + yymsp[0].minor.yy674 = yylhsminor.yy674; break; case 89: /* alter_db_options ::= alter_db_options alter_db_option */ -{ yylhsminor.yy632 = setAlterDatabaseOption(pCxt, yymsp[-1].minor.yy632, &yymsp[0].minor.yy605); } - yymsp[-1].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = setAlterDatabaseOption(pCxt, yymsp[-1].minor.yy674, &yymsp[0].minor.yy557); } + yymsp[-1].minor.yy674 = yylhsminor.yy674; break; case 90: /* alter_db_option ::= BUFFER NK_INTEGER */ -{ yymsp[-1].minor.yy605.type = DB_OPTION_BUFFER; yymsp[-1].minor.yy605.val = yymsp[0].minor.yy0; } +{ yymsp[-1].minor.yy557.type = DB_OPTION_BUFFER; yymsp[-1].minor.yy557.val = yymsp[0].minor.yy0; } break; case 91: /* alter_db_option ::= CACHELAST NK_INTEGER */ -{ yymsp[-1].minor.yy605.type = DB_OPTION_CACHELAST; yymsp[-1].minor.yy605.val = yymsp[0].minor.yy0; } +{ yymsp[-1].minor.yy557.type = DB_OPTION_CACHELAST; yymsp[-1].minor.yy557.val = yymsp[0].minor.yy0; } break; case 92: /* alter_db_option ::= FSYNC NK_INTEGER */ -{ yymsp[-1].minor.yy605.type = DB_OPTION_FSYNC; yymsp[-1].minor.yy605.val = yymsp[0].minor.yy0; } +{ yymsp[-1].minor.yy557.type = DB_OPTION_FSYNC; yymsp[-1].minor.yy557.val = yymsp[0].minor.yy0; } break; case 93: /* alter_db_option ::= KEEP integer_list */ case 94: /* alter_db_option ::= KEEP variable_list */ yytestcase(yyruleno==94); -{ yymsp[-1].minor.yy605.type = DB_OPTION_KEEP; yymsp[-1].minor.yy605.pList = yymsp[0].minor.yy424; } +{ yymsp[-1].minor.yy557.type = DB_OPTION_KEEP; yymsp[-1].minor.yy557.pList = yymsp[0].minor.yy530; } break; case 95: /* alter_db_option ::= PAGES NK_INTEGER */ -{ yymsp[-1].minor.yy605.type = DB_OPTION_PAGES; yymsp[-1].minor.yy605.val = yymsp[0].minor.yy0; } +{ yymsp[-1].minor.yy557.type = DB_OPTION_PAGES; yymsp[-1].minor.yy557.val = yymsp[0].minor.yy0; } break; case 96: /* alter_db_option ::= REPLICA NK_INTEGER */ -{ yymsp[-1].minor.yy605.type = DB_OPTION_REPLICA; yymsp[-1].minor.yy605.val = yymsp[0].minor.yy0; } +{ yymsp[-1].minor.yy557.type = DB_OPTION_REPLICA; yymsp[-1].minor.yy557.val = yymsp[0].minor.yy0; } break; case 97: /* alter_db_option ::= STRICT NK_INTEGER */ -{ yymsp[-1].minor.yy605.type = DB_OPTION_STRICT; yymsp[-1].minor.yy605.val = yymsp[0].minor.yy0; } +{ yymsp[-1].minor.yy557.type = DB_OPTION_STRICT; yymsp[-1].minor.yy557.val = yymsp[0].minor.yy0; } break; case 98: /* alter_db_option ::= WAL NK_INTEGER */ -{ yymsp[-1].minor.yy605.type = DB_OPTION_WAL; yymsp[-1].minor.yy605.val = yymsp[0].minor.yy0; } +{ yymsp[-1].minor.yy557.type = DB_OPTION_WAL; yymsp[-1].minor.yy557.val = yymsp[0].minor.yy0; } break; case 99: /* integer_list ::= NK_INTEGER */ -{ yylhsminor.yy424 = createNodeList(pCxt, createValueNode(pCxt, TSDB_DATA_TYPE_BIGINT, &yymsp[0].minor.yy0)); } - yymsp[0].minor.yy424 = yylhsminor.yy424; +{ yylhsminor.yy530 = createNodeList(pCxt, createValueNode(pCxt, TSDB_DATA_TYPE_BIGINT, &yymsp[0].minor.yy0)); } + yymsp[0].minor.yy530 = yylhsminor.yy530; break; case 100: /* integer_list ::= integer_list NK_COMMA NK_INTEGER */ - case 261: /* dnode_list ::= dnode_list DNODE NK_INTEGER */ yytestcase(yyruleno==261); -{ yylhsminor.yy424 = addNodeToList(pCxt, yymsp[-2].minor.yy424, createValueNode(pCxt, TSDB_DATA_TYPE_BIGINT, &yymsp[0].minor.yy0)); } - yymsp[-2].minor.yy424 = yylhsminor.yy424; + case 266: /* dnode_list ::= dnode_list DNODE NK_INTEGER */ yytestcase(yyruleno==266); +{ yylhsminor.yy530 = addNodeToList(pCxt, yymsp[-2].minor.yy530, createValueNode(pCxt, TSDB_DATA_TYPE_BIGINT, &yymsp[0].minor.yy0)); } + yymsp[-2].minor.yy530 = yylhsminor.yy530; break; case 101: /* variable_list ::= NK_VARIABLE */ -{ yylhsminor.yy424 = createNodeList(pCxt, createDurationValueNode(pCxt, &yymsp[0].minor.yy0)); } - yymsp[0].minor.yy424 = yylhsminor.yy424; +{ yylhsminor.yy530 = createNodeList(pCxt, createDurationValueNode(pCxt, &yymsp[0].minor.yy0)); } + yymsp[0].minor.yy530 = yylhsminor.yy530; break; case 102: /* variable_list ::= variable_list NK_COMMA NK_VARIABLE */ -{ yylhsminor.yy424 = addNodeToList(pCxt, yymsp[-2].minor.yy424, createDurationValueNode(pCxt, &yymsp[0].minor.yy0)); } - yymsp[-2].minor.yy424 = yylhsminor.yy424; +{ yylhsminor.yy530 = addNodeToList(pCxt, yymsp[-2].minor.yy530, createDurationValueNode(pCxt, &yymsp[0].minor.yy0)); } + yymsp[-2].minor.yy530 = yylhsminor.yy530; break; case 103: /* retention_list ::= retention */ case 123: /* multi_create_clause ::= create_subtable_clause */ yytestcase(yyruleno==123); case 126: /* multi_drop_clause ::= drop_table_clause */ yytestcase(yyruleno==126); case 133: /* column_def_list ::= column_def */ yytestcase(yyruleno==133); - case 173: /* col_name_list ::= col_name */ yytestcase(yyruleno==173); - case 211: /* func_name_list ::= func_name */ yytestcase(yyruleno==211); - case 220: /* func_list ::= func */ yytestcase(yyruleno==220); - case 287: /* literal_list ::= signed_literal */ yytestcase(yyruleno==287); - case 341: /* other_para_list ::= star_func_para */ yytestcase(yyruleno==341); - case 396: /* select_sublist ::= select_item */ yytestcase(yyruleno==396); - case 445: /* sort_specification_list ::= sort_specification */ yytestcase(yyruleno==445); -{ yylhsminor.yy424 = createNodeList(pCxt, yymsp[0].minor.yy632); } - yymsp[0].minor.yy424 = yylhsminor.yy424; + case 176: /* rollup_func_list ::= rollup_func_name */ yytestcase(yyruleno==176); + case 181: /* col_name_list ::= col_name */ yytestcase(yyruleno==181); + case 225: /* func_list ::= func */ yytestcase(yyruleno==225); + case 292: /* literal_list ::= signed_literal */ yytestcase(yyruleno==292); + case 346: /* other_para_list ::= star_func_para */ yytestcase(yyruleno==346); + case 401: /* select_sublist ::= select_item */ yytestcase(yyruleno==401); + case 450: /* sort_specification_list ::= sort_specification */ yytestcase(yyruleno==450); +{ yylhsminor.yy530 = createNodeList(pCxt, yymsp[0].minor.yy674); } + yymsp[0].minor.yy530 = yylhsminor.yy530; break; case 104: /* retention_list ::= retention_list NK_COMMA retention */ case 134: /* column_def_list ::= column_def_list NK_COMMA column_def */ yytestcase(yyruleno==134); - case 174: /* col_name_list ::= col_name_list NK_COMMA col_name */ yytestcase(yyruleno==174); - case 212: /* func_name_list ::= func_name_list NK_COMMA func_name */ yytestcase(yyruleno==212); - case 221: /* func_list ::= func_list NK_COMMA func */ yytestcase(yyruleno==221); - case 288: /* literal_list ::= literal_list NK_COMMA signed_literal */ yytestcase(yyruleno==288); - case 342: /* other_para_list ::= other_para_list NK_COMMA star_func_para */ yytestcase(yyruleno==342); - case 397: /* select_sublist ::= select_sublist NK_COMMA select_item */ yytestcase(yyruleno==397); - case 446: /* sort_specification_list ::= sort_specification_list NK_COMMA sort_specification */ yytestcase(yyruleno==446); -{ yylhsminor.yy424 = addNodeToList(pCxt, yymsp[-2].minor.yy424, yymsp[0].minor.yy632); } - yymsp[-2].minor.yy424 = yylhsminor.yy424; + case 177: /* rollup_func_list ::= rollup_func_list NK_COMMA rollup_func_name */ yytestcase(yyruleno==177); + case 182: /* col_name_list ::= col_name_list NK_COMMA col_name */ yytestcase(yyruleno==182); + case 226: /* func_list ::= func_list NK_COMMA func */ yytestcase(yyruleno==226); + case 293: /* literal_list ::= literal_list NK_COMMA signed_literal */ yytestcase(yyruleno==293); + case 347: /* other_para_list ::= other_para_list NK_COMMA star_func_para */ yytestcase(yyruleno==347); + case 402: /* select_sublist ::= select_sublist NK_COMMA select_item */ yytestcase(yyruleno==402); + case 451: /* sort_specification_list ::= sort_specification_list NK_COMMA sort_specification */ yytestcase(yyruleno==451); +{ yylhsminor.yy530 = addNodeToList(pCxt, yymsp[-2].minor.yy530, yymsp[0].minor.yy674); } + yymsp[-2].minor.yy530 = yylhsminor.yy530; break; case 105: /* retention ::= NK_VARIABLE NK_COLON NK_VARIABLE */ -{ yylhsminor.yy632 = createNodeListNodeEx(pCxt, createDurationValueNode(pCxt, &yymsp[-2].minor.yy0), createDurationValueNode(pCxt, &yymsp[0].minor.yy0)); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = createNodeListNodeEx(pCxt, createDurationValueNode(pCxt, &yymsp[-2].minor.yy0), createDurationValueNode(pCxt, &yymsp[0].minor.yy0)); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; case 106: /* cmd ::= CREATE TABLE not_exists_opt full_table_name NK_LP column_def_list NK_RP tags_def_opt table_options */ case 108: /* cmd ::= CREATE STABLE not_exists_opt full_table_name NK_LP column_def_list NK_RP tags_def table_options */ yytestcase(yyruleno==108); -{ pCxt->pRootNode = createCreateTableStmt(pCxt, yymsp[-6].minor.yy137, yymsp[-5].minor.yy632, yymsp[-3].minor.yy424, yymsp[-1].minor.yy424, yymsp[0].minor.yy632); } +{ pCxt->pRootNode = createCreateTableStmt(pCxt, yymsp[-6].minor.yy621, yymsp[-5].minor.yy674, yymsp[-3].minor.yy530, yymsp[-1].minor.yy530, yymsp[0].minor.yy674); } break; case 107: /* cmd ::= CREATE TABLE multi_create_clause */ -{ pCxt->pRootNode = createCreateMultiTableStmt(pCxt, yymsp[0].minor.yy424); } +{ pCxt->pRootNode = createCreateMultiTableStmt(pCxt, yymsp[0].minor.yy530); } break; case 109: /* cmd ::= DROP TABLE multi_drop_clause */ -{ pCxt->pRootNode = createDropTableStmt(pCxt, yymsp[0].minor.yy424); } +{ pCxt->pRootNode = createDropTableStmt(pCxt, yymsp[0].minor.yy530); } break; case 110: /* cmd ::= DROP STABLE exists_opt full_table_name */ -{ pCxt->pRootNode = createDropSuperTableStmt(pCxt, yymsp[-1].minor.yy137, yymsp[0].minor.yy632); } +{ pCxt->pRootNode = createDropSuperTableStmt(pCxt, yymsp[-1].minor.yy621, yymsp[0].minor.yy674); } break; case 111: /* cmd ::= ALTER TABLE alter_table_clause */ case 112: /* cmd ::= ALTER STABLE alter_table_clause */ yytestcase(yyruleno==112); - case 264: /* cmd ::= query_expression */ yytestcase(yyruleno==264); -{ pCxt->pRootNode = yymsp[0].minor.yy632; } + case 269: /* cmd ::= query_expression */ yytestcase(yyruleno==269); +{ pCxt->pRootNode = yymsp[0].minor.yy674; } break; case 113: /* alter_table_clause ::= full_table_name alter_table_options */ -{ yylhsminor.yy632 = createAlterTableModifyOptions(pCxt, yymsp[-1].minor.yy632, yymsp[0].minor.yy632); } - yymsp[-1].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = createAlterTableModifyOptions(pCxt, yymsp[-1].minor.yy674, yymsp[0].minor.yy674); } + yymsp[-1].minor.yy674 = yylhsminor.yy674; break; case 114: /* alter_table_clause ::= full_table_name ADD COLUMN column_name type_name */ -{ yylhsminor.yy632 = createAlterTableAddModifyCol(pCxt, yymsp[-4].minor.yy632, TSDB_ALTER_TABLE_ADD_COLUMN, &yymsp[-1].minor.yy209, yymsp[0].minor.yy304); } - yymsp[-4].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = createAlterTableAddModifyCol(pCxt, yymsp[-4].minor.yy674, TSDB_ALTER_TABLE_ADD_COLUMN, &yymsp[-1].minor.yy421, yymsp[0].minor.yy690); } + yymsp[-4].minor.yy674 = yylhsminor.yy674; break; case 115: /* alter_table_clause ::= full_table_name DROP COLUMN column_name */ -{ yylhsminor.yy632 = createAlterTableDropCol(pCxt, yymsp[-3].minor.yy632, TSDB_ALTER_TABLE_DROP_COLUMN, &yymsp[0].minor.yy209); } - yymsp[-3].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = createAlterTableDropCol(pCxt, yymsp[-3].minor.yy674, TSDB_ALTER_TABLE_DROP_COLUMN, &yymsp[0].minor.yy421); } + yymsp[-3].minor.yy674 = yylhsminor.yy674; break; case 116: /* alter_table_clause ::= full_table_name MODIFY COLUMN column_name type_name */ -{ yylhsminor.yy632 = createAlterTableAddModifyCol(pCxt, yymsp[-4].minor.yy632, TSDB_ALTER_TABLE_UPDATE_COLUMN_BYTES, &yymsp[-1].minor.yy209, yymsp[0].minor.yy304); } - yymsp[-4].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = createAlterTableAddModifyCol(pCxt, yymsp[-4].minor.yy674, TSDB_ALTER_TABLE_UPDATE_COLUMN_BYTES, &yymsp[-1].minor.yy421, yymsp[0].minor.yy690); } + yymsp[-4].minor.yy674 = yylhsminor.yy674; break; case 117: /* alter_table_clause ::= full_table_name RENAME COLUMN column_name column_name */ -{ yylhsminor.yy632 = createAlterTableRenameCol(pCxt, yymsp[-4].minor.yy632, TSDB_ALTER_TABLE_UPDATE_COLUMN_NAME, &yymsp[-1].minor.yy209, &yymsp[0].minor.yy209); } - yymsp[-4].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = createAlterTableRenameCol(pCxt, yymsp[-4].minor.yy674, TSDB_ALTER_TABLE_UPDATE_COLUMN_NAME, &yymsp[-1].minor.yy421, &yymsp[0].minor.yy421); } + yymsp[-4].minor.yy674 = yylhsminor.yy674; break; case 118: /* alter_table_clause ::= full_table_name ADD TAG column_name type_name */ -{ yylhsminor.yy632 = createAlterTableAddModifyCol(pCxt, yymsp[-4].minor.yy632, TSDB_ALTER_TABLE_ADD_TAG, &yymsp[-1].minor.yy209, yymsp[0].minor.yy304); } - yymsp[-4].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = createAlterTableAddModifyCol(pCxt, yymsp[-4].minor.yy674, TSDB_ALTER_TABLE_ADD_TAG, &yymsp[-1].minor.yy421, yymsp[0].minor.yy690); } + yymsp[-4].minor.yy674 = yylhsminor.yy674; break; case 119: /* alter_table_clause ::= full_table_name DROP TAG column_name */ -{ yylhsminor.yy632 = createAlterTableDropCol(pCxt, yymsp[-3].minor.yy632, TSDB_ALTER_TABLE_DROP_TAG, &yymsp[0].minor.yy209); } - yymsp[-3].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = createAlterTableDropCol(pCxt, yymsp[-3].minor.yy674, TSDB_ALTER_TABLE_DROP_TAG, &yymsp[0].minor.yy421); } + yymsp[-3].minor.yy674 = yylhsminor.yy674; break; case 120: /* alter_table_clause ::= full_table_name MODIFY TAG column_name type_name */ -{ yylhsminor.yy632 = createAlterTableAddModifyCol(pCxt, yymsp[-4].minor.yy632, TSDB_ALTER_TABLE_UPDATE_TAG_BYTES, &yymsp[-1].minor.yy209, yymsp[0].minor.yy304); } - yymsp[-4].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = createAlterTableAddModifyCol(pCxt, yymsp[-4].minor.yy674, TSDB_ALTER_TABLE_UPDATE_TAG_BYTES, &yymsp[-1].minor.yy421, yymsp[0].minor.yy690); } + yymsp[-4].minor.yy674 = yylhsminor.yy674; break; case 121: /* alter_table_clause ::= full_table_name RENAME TAG column_name column_name */ -{ yylhsminor.yy632 = createAlterTableRenameCol(pCxt, yymsp[-4].minor.yy632, TSDB_ALTER_TABLE_UPDATE_TAG_NAME, &yymsp[-1].minor.yy209, &yymsp[0].minor.yy209); } - yymsp[-4].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = createAlterTableRenameCol(pCxt, yymsp[-4].minor.yy674, TSDB_ALTER_TABLE_UPDATE_TAG_NAME, &yymsp[-1].minor.yy421, &yymsp[0].minor.yy421); } + yymsp[-4].minor.yy674 = yylhsminor.yy674; break; case 122: /* alter_table_clause ::= full_table_name SET TAG column_name NK_EQ signed_literal */ -{ yylhsminor.yy632 = createAlterTableSetTag(pCxt, yymsp[-5].minor.yy632, &yymsp[-2].minor.yy209, yymsp[0].minor.yy632); } - yymsp[-5].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = createAlterTableSetTag(pCxt, yymsp[-5].minor.yy674, &yymsp[-2].minor.yy421, yymsp[0].minor.yy674); } + yymsp[-5].minor.yy674 = yylhsminor.yy674; break; case 124: /* multi_create_clause ::= multi_create_clause create_subtable_clause */ case 127: /* multi_drop_clause ::= multi_drop_clause drop_table_clause */ yytestcase(yyruleno==127); -{ yylhsminor.yy424 = addNodeToList(pCxt, yymsp[-1].minor.yy424, yymsp[0].minor.yy632); } - yymsp[-1].minor.yy424 = yylhsminor.yy424; +{ yylhsminor.yy530 = addNodeToList(pCxt, yymsp[-1].minor.yy530, yymsp[0].minor.yy674); } + yymsp[-1].minor.yy530 = yylhsminor.yy530; break; case 125: /* create_subtable_clause ::= not_exists_opt full_table_name USING full_table_name specific_tags_opt TAGS NK_LP literal_list NK_RP table_options */ -{ yylhsminor.yy632 = createCreateSubTableClause(pCxt, yymsp[-9].minor.yy137, yymsp[-8].minor.yy632, yymsp[-6].minor.yy632, yymsp[-5].minor.yy424, yymsp[-2].minor.yy424, yymsp[0].minor.yy632); } - yymsp[-9].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = createCreateSubTableClause(pCxt, yymsp[-9].minor.yy621, yymsp[-8].minor.yy674, yymsp[-6].minor.yy674, yymsp[-5].minor.yy530, yymsp[-2].minor.yy530, yymsp[0].minor.yy674); } + yymsp[-9].minor.yy674 = yylhsminor.yy674; break; case 128: /* drop_table_clause ::= exists_opt full_table_name */ -{ yylhsminor.yy632 = createDropTableClause(pCxt, yymsp[-1].minor.yy137, yymsp[0].minor.yy632); } - yymsp[-1].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = createDropTableClause(pCxt, yymsp[-1].minor.yy621, yymsp[0].minor.yy674); } + yymsp[-1].minor.yy674 = yylhsminor.yy674; break; case 129: /* specific_tags_opt ::= */ case 160: /* tags_def_opt ::= */ yytestcase(yyruleno==160); - case 404: /* partition_by_clause_opt ::= */ yytestcase(yyruleno==404); - case 421: /* group_by_clause_opt ::= */ yytestcase(yyruleno==421); - case 433: /* order_by_clause_opt ::= */ yytestcase(yyruleno==433); -{ yymsp[1].minor.yy424 = NULL; } + case 409: /* partition_by_clause_opt ::= */ yytestcase(yyruleno==409); + case 426: /* group_by_clause_opt ::= */ yytestcase(yyruleno==426); + case 438: /* order_by_clause_opt ::= */ yytestcase(yyruleno==438); +{ yymsp[1].minor.yy530 = NULL; } break; case 130: /* specific_tags_opt ::= NK_LP col_name_list NK_RP */ -{ yymsp[-2].minor.yy424 = yymsp[-1].minor.yy424; } +{ yymsp[-2].minor.yy530 = yymsp[-1].minor.yy530; } break; case 131: /* full_table_name ::= table_name */ -{ yylhsminor.yy632 = createRealTableNode(pCxt, NULL, &yymsp[0].minor.yy209, NULL); } - yymsp[0].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = createRealTableNode(pCxt, NULL, &yymsp[0].minor.yy421, NULL); } + yymsp[0].minor.yy674 = yylhsminor.yy674; break; case 132: /* full_table_name ::= db_name NK_DOT table_name */ -{ yylhsminor.yy632 = createRealTableNode(pCxt, &yymsp[-2].minor.yy209, &yymsp[0].minor.yy209, NULL); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = createRealTableNode(pCxt, &yymsp[-2].minor.yy421, &yymsp[0].minor.yy421, NULL); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; case 135: /* column_def ::= column_name type_name */ -{ yylhsminor.yy632 = createColumnDefNode(pCxt, &yymsp[-1].minor.yy209, yymsp[0].minor.yy304, NULL); } - yymsp[-1].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = createColumnDefNode(pCxt, &yymsp[-1].minor.yy421, yymsp[0].minor.yy690, NULL); } + yymsp[-1].minor.yy674 = yylhsminor.yy674; break; case 136: /* column_def ::= column_name type_name COMMENT NK_STRING */ -{ yylhsminor.yy632 = createColumnDefNode(pCxt, &yymsp[-3].minor.yy209, yymsp[-2].minor.yy304, &yymsp[0].minor.yy0); } - yymsp[-3].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = createColumnDefNode(pCxt, &yymsp[-3].minor.yy421, yymsp[-2].minor.yy690, &yymsp[0].minor.yy0); } + yymsp[-3].minor.yy674 = yylhsminor.yy674; break; case 137: /* type_name ::= BOOL */ -{ yymsp[0].minor.yy304 = createDataType(TSDB_DATA_TYPE_BOOL); } +{ yymsp[0].minor.yy690 = createDataType(TSDB_DATA_TYPE_BOOL); } break; case 138: /* type_name ::= TINYINT */ -{ yymsp[0].minor.yy304 = createDataType(TSDB_DATA_TYPE_TINYINT); } +{ yymsp[0].minor.yy690 = createDataType(TSDB_DATA_TYPE_TINYINT); } break; case 139: /* type_name ::= SMALLINT */ -{ yymsp[0].minor.yy304 = createDataType(TSDB_DATA_TYPE_SMALLINT); } +{ yymsp[0].minor.yy690 = createDataType(TSDB_DATA_TYPE_SMALLINT); } break; case 140: /* type_name ::= INT */ case 141: /* type_name ::= INTEGER */ yytestcase(yyruleno==141); -{ yymsp[0].minor.yy304 = createDataType(TSDB_DATA_TYPE_INT); } +{ yymsp[0].minor.yy690 = createDataType(TSDB_DATA_TYPE_INT); } break; case 142: /* type_name ::= BIGINT */ -{ yymsp[0].minor.yy304 = createDataType(TSDB_DATA_TYPE_BIGINT); } +{ yymsp[0].minor.yy690 = createDataType(TSDB_DATA_TYPE_BIGINT); } break; case 143: /* type_name ::= FLOAT */ -{ yymsp[0].minor.yy304 = createDataType(TSDB_DATA_TYPE_FLOAT); } +{ yymsp[0].minor.yy690 = createDataType(TSDB_DATA_TYPE_FLOAT); } break; case 144: /* type_name ::= DOUBLE */ -{ yymsp[0].minor.yy304 = createDataType(TSDB_DATA_TYPE_DOUBLE); } +{ yymsp[0].minor.yy690 = createDataType(TSDB_DATA_TYPE_DOUBLE); } break; case 145: /* type_name ::= BINARY NK_LP NK_INTEGER NK_RP */ -{ yymsp[-3].minor.yy304 = createVarLenDataType(TSDB_DATA_TYPE_BINARY, &yymsp[-1].minor.yy0); } +{ yymsp[-3].minor.yy690 = createVarLenDataType(TSDB_DATA_TYPE_BINARY, &yymsp[-1].minor.yy0); } break; case 146: /* type_name ::= TIMESTAMP */ -{ yymsp[0].minor.yy304 = createDataType(TSDB_DATA_TYPE_TIMESTAMP); } +{ yymsp[0].minor.yy690 = createDataType(TSDB_DATA_TYPE_TIMESTAMP); } break; case 147: /* type_name ::= NCHAR NK_LP NK_INTEGER NK_RP */ -{ yymsp[-3].minor.yy304 = createVarLenDataType(TSDB_DATA_TYPE_NCHAR, &yymsp[-1].minor.yy0); } +{ yymsp[-3].minor.yy690 = createVarLenDataType(TSDB_DATA_TYPE_NCHAR, &yymsp[-1].minor.yy0); } break; case 148: /* type_name ::= TINYINT UNSIGNED */ -{ yymsp[-1].minor.yy304 = createDataType(TSDB_DATA_TYPE_UTINYINT); } +{ yymsp[-1].minor.yy690 = createDataType(TSDB_DATA_TYPE_UTINYINT); } break; case 149: /* type_name ::= SMALLINT UNSIGNED */ -{ yymsp[-1].minor.yy304 = createDataType(TSDB_DATA_TYPE_USMALLINT); } +{ yymsp[-1].minor.yy690 = createDataType(TSDB_DATA_TYPE_USMALLINT); } break; case 150: /* type_name ::= INT UNSIGNED */ -{ yymsp[-1].minor.yy304 = createDataType(TSDB_DATA_TYPE_UINT); } +{ yymsp[-1].minor.yy690 = createDataType(TSDB_DATA_TYPE_UINT); } break; case 151: /* type_name ::= BIGINT UNSIGNED */ -{ yymsp[-1].minor.yy304 = createDataType(TSDB_DATA_TYPE_UBIGINT); } +{ yymsp[-1].minor.yy690 = createDataType(TSDB_DATA_TYPE_UBIGINT); } break; case 152: /* type_name ::= JSON */ -{ yymsp[0].minor.yy304 = createDataType(TSDB_DATA_TYPE_JSON); } +{ yymsp[0].minor.yy690 = createDataType(TSDB_DATA_TYPE_JSON); } break; case 153: /* type_name ::= VARCHAR NK_LP NK_INTEGER NK_RP */ -{ yymsp[-3].minor.yy304 = createVarLenDataType(TSDB_DATA_TYPE_VARCHAR, &yymsp[-1].minor.yy0); } +{ yymsp[-3].minor.yy690 = createVarLenDataType(TSDB_DATA_TYPE_VARCHAR, &yymsp[-1].minor.yy0); } break; case 154: /* type_name ::= MEDIUMBLOB */ -{ yymsp[0].minor.yy304 = createDataType(TSDB_DATA_TYPE_MEDIUMBLOB); } +{ yymsp[0].minor.yy690 = createDataType(TSDB_DATA_TYPE_MEDIUMBLOB); } break; case 155: /* type_name ::= BLOB */ -{ yymsp[0].minor.yy304 = createDataType(TSDB_DATA_TYPE_BLOB); } +{ yymsp[0].minor.yy690 = createDataType(TSDB_DATA_TYPE_BLOB); } break; case 156: /* type_name ::= VARBINARY NK_LP NK_INTEGER NK_RP */ -{ yymsp[-3].minor.yy304 = createVarLenDataType(TSDB_DATA_TYPE_VARBINARY, &yymsp[-1].minor.yy0); } +{ yymsp[-3].minor.yy690 = createVarLenDataType(TSDB_DATA_TYPE_VARBINARY, &yymsp[-1].minor.yy0); } break; case 157: /* type_name ::= DECIMAL */ -{ yymsp[0].minor.yy304 = createDataType(TSDB_DATA_TYPE_DECIMAL); } +{ yymsp[0].minor.yy690 = createDataType(TSDB_DATA_TYPE_DECIMAL); } break; case 158: /* type_name ::= DECIMAL NK_LP NK_INTEGER NK_RP */ -{ yymsp[-3].minor.yy304 = createDataType(TSDB_DATA_TYPE_DECIMAL); } +{ yymsp[-3].minor.yy690 = createDataType(TSDB_DATA_TYPE_DECIMAL); } break; case 159: /* type_name ::= DECIMAL NK_LP NK_INTEGER NK_COMMA NK_INTEGER NK_RP */ -{ yymsp[-5].minor.yy304 = createDataType(TSDB_DATA_TYPE_DECIMAL); } +{ yymsp[-5].minor.yy690 = createDataType(TSDB_DATA_TYPE_DECIMAL); } break; case 161: /* tags_def_opt ::= tags_def */ - case 340: /* star_func_para_list ::= other_para_list */ yytestcase(yyruleno==340); - case 395: /* select_list ::= select_sublist */ yytestcase(yyruleno==395); -{ yylhsminor.yy424 = yymsp[0].minor.yy424; } - yymsp[0].minor.yy424 = yylhsminor.yy424; + case 345: /* star_func_para_list ::= other_para_list */ yytestcase(yyruleno==345); + case 400: /* select_list ::= select_sublist */ yytestcase(yyruleno==400); +{ yylhsminor.yy530 = yymsp[0].minor.yy530; } + yymsp[0].minor.yy530 = yylhsminor.yy530; break; case 162: /* tags_def ::= TAGS NK_LP column_def_list NK_RP */ -{ yymsp[-3].minor.yy424 = yymsp[-1].minor.yy424; } +{ yymsp[-3].minor.yy530 = yymsp[-1].minor.yy530; } break; case 163: /* table_options ::= */ -{ yymsp[1].minor.yy632 = createDefaultTableOptions(pCxt); } +{ yymsp[1].minor.yy674 = createDefaultTableOptions(pCxt); } break; case 164: /* table_options ::= table_options COMMENT NK_STRING */ -{ yylhsminor.yy632 = setTableOption(pCxt, yymsp[-2].minor.yy632, TABLE_OPTION_COMMENT, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; +{ yylhsminor.yy674 = setTableOption(pCxt, yymsp[-2].minor.yy674, TABLE_OPTION_COMMENT, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; - case 165: /* table_options ::= table_options FILE_FACTOR NK_FLOAT */ -{ yylhsminor.yy632 = setTableOption(pCxt, yymsp[-2].minor.yy632, TABLE_OPTION_FILE_FACTOR, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; + case 165: /* table_options ::= table_options MAX_DELAY duration_list */ +{ yylhsminor.yy674 = setTableOption(pCxt, yymsp[-2].minor.yy674, TABLE_OPTION_MAXDELAY, yymsp[0].minor.yy530); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; - case 166: /* table_options ::= table_options ROLLUP NK_LP func_name_list NK_RP */ -{ yylhsminor.yy632 = setTableOption(pCxt, yymsp[-4].minor.yy632, TABLE_OPTION_ROLLUP, yymsp[-1].minor.yy424); } - yymsp[-4].minor.yy632 = yylhsminor.yy632; + case 166: /* table_options ::= table_options WATERMARK duration_list */ +{ yylhsminor.yy674 = setTableOption(pCxt, yymsp[-2].minor.yy674, TABLE_OPTION_WATERMARK, yymsp[0].minor.yy530); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; - case 167: /* table_options ::= table_options TTL NK_INTEGER */ -{ yylhsminor.yy632 = setTableOption(pCxt, yymsp[-2].minor.yy632, TABLE_OPTION_TTL, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; + case 167: /* table_options ::= table_options ROLLUP NK_LP rollup_func_list NK_RP */ +{ yylhsminor.yy674 = setTableOption(pCxt, yymsp[-4].minor.yy674, TABLE_OPTION_ROLLUP, yymsp[-1].minor.yy530); } + yymsp[-4].minor.yy674 = yylhsminor.yy674; break; - case 168: /* table_options ::= table_options SMA NK_LP col_name_list NK_RP */ -{ yylhsminor.yy632 = setTableOption(pCxt, yymsp[-4].minor.yy632, TABLE_OPTION_SMA, yymsp[-1].minor.yy424); } - yymsp[-4].minor.yy632 = yylhsminor.yy632; + case 168: /* table_options ::= table_options TTL NK_INTEGER */ +{ yylhsminor.yy674 = setTableOption(pCxt, yymsp[-2].minor.yy674, TABLE_OPTION_TTL, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; - case 169: /* alter_table_options ::= alter_table_option */ -{ yylhsminor.yy632 = createAlterTableOptions(pCxt); yylhsminor.yy632 = setTableOption(pCxt, yylhsminor.yy632, yymsp[0].minor.yy605.type, &yymsp[0].minor.yy605.val); } - yymsp[0].minor.yy632 = yylhsminor.yy632; + case 169: /* table_options ::= table_options SMA NK_LP col_name_list NK_RP */ +{ yylhsminor.yy674 = setTableOption(pCxt, yymsp[-4].minor.yy674, TABLE_OPTION_SMA, yymsp[-1].minor.yy530); } + yymsp[-4].minor.yy674 = yylhsminor.yy674; break; - case 170: /* alter_table_options ::= alter_table_options alter_table_option */ -{ yylhsminor.yy632 = setTableOption(pCxt, yymsp[-1].minor.yy632, yymsp[0].minor.yy605.type, &yymsp[0].minor.yy605.val); } - yymsp[-1].minor.yy632 = yylhsminor.yy632; + case 170: /* alter_table_options ::= alter_table_option */ +{ yylhsminor.yy674 = createAlterTableOptions(pCxt); yylhsminor.yy674 = setTableOption(pCxt, yylhsminor.yy674, yymsp[0].minor.yy557.type, &yymsp[0].minor.yy557.val); } + yymsp[0].minor.yy674 = yylhsminor.yy674; break; - case 171: /* alter_table_option ::= COMMENT NK_STRING */ -{ yymsp[-1].minor.yy605.type = TABLE_OPTION_COMMENT; yymsp[-1].minor.yy605.val = yymsp[0].minor.yy0; } + case 171: /* alter_table_options ::= alter_table_options alter_table_option */ +{ yylhsminor.yy674 = setTableOption(pCxt, yymsp[-1].minor.yy674, yymsp[0].minor.yy557.type, &yymsp[0].minor.yy557.val); } + yymsp[-1].minor.yy674 = yylhsminor.yy674; break; - case 172: /* alter_table_option ::= TTL NK_INTEGER */ -{ yymsp[-1].minor.yy605.type = TABLE_OPTION_TTL; yymsp[-1].minor.yy605.val = yymsp[0].minor.yy0; } + case 172: /* alter_table_option ::= COMMENT NK_STRING */ +{ yymsp[-1].minor.yy557.type = TABLE_OPTION_COMMENT; yymsp[-1].minor.yy557.val = yymsp[0].minor.yy0; } break; - case 175: /* col_name ::= column_name */ -{ yylhsminor.yy632 = createColumnNode(pCxt, NULL, &yymsp[0].minor.yy209); } - yymsp[0].minor.yy632 = yylhsminor.yy632; + case 173: /* alter_table_option ::= TTL NK_INTEGER */ +{ yymsp[-1].minor.yy557.type = TABLE_OPTION_TTL; yymsp[-1].minor.yy557.val = yymsp[0].minor.yy0; } break; - case 176: /* cmd ::= SHOW DNODES */ + case 174: /* duration_list ::= duration_literal */ + case 319: /* expression_list ::= expression */ yytestcase(yyruleno==319); +{ yylhsminor.yy530 = createNodeList(pCxt, releaseRawExprNode(pCxt, yymsp[0].minor.yy674)); } + yymsp[0].minor.yy530 = yylhsminor.yy530; + break; + case 175: /* duration_list ::= duration_list NK_COMMA duration_literal */ + case 320: /* expression_list ::= expression_list NK_COMMA expression */ yytestcase(yyruleno==320); +{ yylhsminor.yy530 = addNodeToList(pCxt, yymsp[-2].minor.yy530, releaseRawExprNode(pCxt, yymsp[0].minor.yy674)); } + yymsp[-2].minor.yy530 = yylhsminor.yy530; + break; + case 178: /* rollup_func_name ::= function_name */ +{ yylhsminor.yy674 = createFunctionNode(pCxt, &yymsp[0].minor.yy421, NULL); } + yymsp[0].minor.yy674 = yylhsminor.yy674; + break; + case 179: /* rollup_func_name ::= FIRST */ + case 180: /* rollup_func_name ::= LAST */ yytestcase(yyruleno==180); +{ yylhsminor.yy674 = createFunctionNode(pCxt, &yymsp[0].minor.yy0, NULL); } + yymsp[0].minor.yy674 = yylhsminor.yy674; + break; + case 183: /* col_name ::= column_name */ +{ yylhsminor.yy674 = createColumnNode(pCxt, NULL, &yymsp[0].minor.yy421); } + yymsp[0].minor.yy674 = yylhsminor.yy674; + break; + case 184: /* cmd ::= SHOW DNODES */ { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_DNODES_STMT, NULL, NULL); } break; - case 177: /* cmd ::= SHOW USERS */ + case 185: /* cmd ::= SHOW USERS */ { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_USERS_STMT, NULL, NULL); } break; - case 178: /* cmd ::= SHOW DATABASES */ + case 186: /* cmd ::= SHOW DATABASES */ { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_DATABASES_STMT, NULL, NULL); } break; - case 179: /* cmd ::= SHOW db_name_cond_opt TABLES like_pattern_opt */ -{ pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_TABLES_STMT, yymsp[-2].minor.yy632, yymsp[0].minor.yy632); } + case 187: /* cmd ::= SHOW db_name_cond_opt TABLES like_pattern_opt */ +{ pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_TABLES_STMT, yymsp[-2].minor.yy674, yymsp[0].minor.yy674); } break; - case 180: /* cmd ::= SHOW db_name_cond_opt STABLES like_pattern_opt */ -{ pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_STABLES_STMT, yymsp[-2].minor.yy632, yymsp[0].minor.yy632); } + case 188: /* cmd ::= SHOW db_name_cond_opt STABLES like_pattern_opt */ +{ pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_STABLES_STMT, yymsp[-2].minor.yy674, yymsp[0].minor.yy674); } break; - case 181: /* cmd ::= SHOW db_name_cond_opt VGROUPS */ -{ pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_VGROUPS_STMT, yymsp[-1].minor.yy632, NULL); } + case 189: /* cmd ::= SHOW db_name_cond_opt VGROUPS */ +{ pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_VGROUPS_STMT, yymsp[-1].minor.yy674, NULL); } break; - case 182: /* cmd ::= SHOW MNODES */ + case 190: /* cmd ::= SHOW MNODES */ { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_MNODES_STMT, NULL, NULL); } break; - case 183: /* cmd ::= SHOW MODULES */ + case 191: /* cmd ::= SHOW MODULES */ { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_MODULES_STMT, NULL, NULL); } break; - case 184: /* cmd ::= SHOW QNODES */ + case 192: /* cmd ::= SHOW QNODES */ { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_QNODES_STMT, NULL, NULL); } break; - case 185: /* cmd ::= SHOW FUNCTIONS */ + case 193: /* cmd ::= SHOW FUNCTIONS */ { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_FUNCTIONS_STMT, NULL, NULL); } break; - case 186: /* cmd ::= SHOW INDEXES FROM table_name_cond from_db_opt */ -{ pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_INDEXES_STMT, yymsp[-1].minor.yy632, yymsp[0].minor.yy632); } + case 194: /* cmd ::= SHOW INDEXES FROM table_name_cond from_db_opt */ +{ pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_INDEXES_STMT, yymsp[-1].minor.yy674, yymsp[0].minor.yy674); } break; - case 187: /* cmd ::= SHOW STREAMS */ + case 195: /* cmd ::= SHOW STREAMS */ { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_STREAMS_STMT, NULL, NULL); } break; - case 188: /* cmd ::= SHOW ACCOUNTS */ + case 196: /* cmd ::= SHOW ACCOUNTS */ { pCxt->errCode = generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_EXPRIE_STATEMENT); } break; - case 189: /* cmd ::= SHOW APPS */ + case 197: /* cmd ::= SHOW APPS */ { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_APPS_STMT, NULL, NULL); } break; - case 190: /* cmd ::= SHOW CONNECTIONS */ + case 198: /* cmd ::= SHOW CONNECTIONS */ { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_CONNECTIONS_STMT, NULL, NULL); } break; - case 191: /* cmd ::= SHOW LICENCE */ - case 192: /* cmd ::= SHOW GRANTS */ yytestcase(yyruleno==192); + case 199: /* cmd ::= SHOW LICENCE */ + case 200: /* cmd ::= SHOW GRANTS */ yytestcase(yyruleno==200); { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_LICENCE_STMT, NULL, NULL); } break; - case 193: /* cmd ::= SHOW CREATE DATABASE db_name */ -{ pCxt->pRootNode = createShowCreateDatabaseStmt(pCxt, &yymsp[0].minor.yy209); } + case 201: /* cmd ::= SHOW CREATE DATABASE db_name */ +{ pCxt->pRootNode = createShowCreateDatabaseStmt(pCxt, &yymsp[0].minor.yy421); } break; - case 194: /* cmd ::= SHOW CREATE TABLE full_table_name */ -{ pCxt->pRootNode = createShowCreateTableStmt(pCxt, QUERY_NODE_SHOW_CREATE_TABLE_STMT, yymsp[0].minor.yy632); } + case 202: /* cmd ::= SHOW CREATE TABLE full_table_name */ +{ pCxt->pRootNode = createShowCreateTableStmt(pCxt, QUERY_NODE_SHOW_CREATE_TABLE_STMT, yymsp[0].minor.yy674); } break; - case 195: /* cmd ::= SHOW CREATE STABLE full_table_name */ -{ pCxt->pRootNode = createShowCreateTableStmt(pCxt, QUERY_NODE_SHOW_CREATE_STABLE_STMT, yymsp[0].minor.yy632); } + case 203: /* cmd ::= SHOW CREATE STABLE full_table_name */ +{ pCxt->pRootNode = createShowCreateTableStmt(pCxt, QUERY_NODE_SHOW_CREATE_STABLE_STMT, yymsp[0].minor.yy674); } break; - case 196: /* cmd ::= SHOW QUERIES */ + case 204: /* cmd ::= SHOW QUERIES */ { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_QUERIES_STMT, NULL, NULL); } break; - case 197: /* cmd ::= SHOW SCORES */ + case 205: /* cmd ::= SHOW SCORES */ { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_SCORES_STMT, NULL, NULL); } break; - case 198: /* cmd ::= SHOW TOPICS */ + case 206: /* cmd ::= SHOW TOPICS */ { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_TOPICS_STMT, NULL, NULL); } break; - case 199: /* cmd ::= SHOW VARIABLES */ + case 207: /* cmd ::= SHOW VARIABLES */ { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_VARIABLE_STMT, NULL, NULL); } break; - case 200: /* cmd ::= SHOW BNODES */ + case 208: /* cmd ::= SHOW BNODES */ { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_BNODES_STMT, NULL, NULL); } break; - case 201: /* cmd ::= SHOW SNODES */ + case 209: /* cmd ::= SHOW SNODES */ { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_SNODES_STMT, NULL, NULL); } break; - case 202: /* cmd ::= SHOW CLUSTER */ + case 210: /* cmd ::= SHOW CLUSTER */ { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_CLUSTER_STMT, NULL, NULL); } break; - case 203: /* cmd ::= SHOW TRANSACTIONS */ + case 211: /* cmd ::= SHOW TRANSACTIONS */ { pCxt->pRootNode = createShowStmt(pCxt, QUERY_NODE_SHOW_TRANSACTIONS_STMT, NULL, NULL); } break; - case 204: /* db_name_cond_opt ::= */ - case 209: /* from_db_opt ::= */ yytestcase(yyruleno==209); -{ yymsp[1].minor.yy632 = createDefaultDatabaseCondValue(pCxt); } - break; - case 205: /* db_name_cond_opt ::= db_name NK_DOT */ -{ yylhsminor.yy632 = createValueNode(pCxt, TSDB_DATA_TYPE_BINARY, &yymsp[-1].minor.yy209); } - yymsp[-1].minor.yy632 = yylhsminor.yy632; + case 212: /* db_name_cond_opt ::= */ + case 217: /* from_db_opt ::= */ yytestcase(yyruleno==217); +{ yymsp[1].minor.yy674 = createDefaultDatabaseCondValue(pCxt); } break; - case 206: /* like_pattern_opt ::= */ - case 217: /* index_options ::= */ yytestcase(yyruleno==217); - case 246: /* into_opt ::= */ yytestcase(yyruleno==246); - case 402: /* where_clause_opt ::= */ yytestcase(yyruleno==402); - case 406: /* twindow_clause_opt ::= */ yytestcase(yyruleno==406); - case 411: /* sliding_opt ::= */ yytestcase(yyruleno==411); - case 413: /* fill_opt ::= */ yytestcase(yyruleno==413); - case 425: /* having_clause_opt ::= */ yytestcase(yyruleno==425); - case 435: /* slimit_clause_opt ::= */ yytestcase(yyruleno==435); - case 439: /* limit_clause_opt ::= */ yytestcase(yyruleno==439); -{ yymsp[1].minor.yy632 = NULL; } + case 213: /* db_name_cond_opt ::= db_name NK_DOT */ +{ yylhsminor.yy674 = createValueNode(pCxt, TSDB_DATA_TYPE_BINARY, &yymsp[-1].minor.yy421); } + yymsp[-1].minor.yy674 = yylhsminor.yy674; break; - case 207: /* like_pattern_opt ::= LIKE NK_STRING */ -{ yymsp[-1].minor.yy632 = createValueNode(pCxt, TSDB_DATA_TYPE_BINARY, &yymsp[0].minor.yy0); } + case 214: /* like_pattern_opt ::= */ + case 222: /* index_options ::= */ yytestcase(yyruleno==222); + case 251: /* into_opt ::= */ yytestcase(yyruleno==251); + case 407: /* where_clause_opt ::= */ yytestcase(yyruleno==407); + case 411: /* twindow_clause_opt ::= */ yytestcase(yyruleno==411); + case 416: /* sliding_opt ::= */ yytestcase(yyruleno==416); + case 418: /* fill_opt ::= */ yytestcase(yyruleno==418); + case 430: /* having_clause_opt ::= */ yytestcase(yyruleno==430); + case 440: /* slimit_clause_opt ::= */ yytestcase(yyruleno==440); + case 444: /* limit_clause_opt ::= */ yytestcase(yyruleno==444); +{ yymsp[1].minor.yy674 = NULL; } break; - case 208: /* table_name_cond ::= table_name */ -{ yylhsminor.yy632 = createValueNode(pCxt, TSDB_DATA_TYPE_BINARY, &yymsp[0].minor.yy209); } - yymsp[0].minor.yy632 = yylhsminor.yy632; + case 215: /* like_pattern_opt ::= LIKE NK_STRING */ +{ yymsp[-1].minor.yy674 = createValueNode(pCxt, TSDB_DATA_TYPE_BINARY, &yymsp[0].minor.yy0); } break; - case 210: /* from_db_opt ::= FROM db_name */ -{ yymsp[-1].minor.yy632 = createValueNode(pCxt, TSDB_DATA_TYPE_BINARY, &yymsp[0].minor.yy209); } + case 216: /* table_name_cond ::= table_name */ +{ yylhsminor.yy674 = createValueNode(pCxt, TSDB_DATA_TYPE_BINARY, &yymsp[0].minor.yy421); } + yymsp[0].minor.yy674 = yylhsminor.yy674; break; - case 213: /* func_name ::= function_name */ -{ yylhsminor.yy632 = createFunctionNode(pCxt, &yymsp[0].minor.yy209, NULL); } - yymsp[0].minor.yy632 = yylhsminor.yy632; + case 218: /* from_db_opt ::= FROM db_name */ +{ yymsp[-1].minor.yy674 = createValueNode(pCxt, TSDB_DATA_TYPE_BINARY, &yymsp[0].minor.yy421); } break; - case 214: /* cmd ::= CREATE SMA INDEX not_exists_opt index_name ON table_name index_options */ -{ pCxt->pRootNode = createCreateIndexStmt(pCxt, INDEX_TYPE_SMA, yymsp[-4].minor.yy137, &yymsp[-3].minor.yy209, &yymsp[-1].minor.yy209, NULL, yymsp[0].minor.yy632); } + case 219: /* cmd ::= CREATE SMA INDEX not_exists_opt index_name ON table_name index_options */ +{ pCxt->pRootNode = createCreateIndexStmt(pCxt, INDEX_TYPE_SMA, yymsp[-4].minor.yy621, &yymsp[-3].minor.yy421, &yymsp[-1].minor.yy421, NULL, yymsp[0].minor.yy674); } break; - case 215: /* cmd ::= CREATE FULLTEXT INDEX not_exists_opt index_name ON table_name NK_LP col_name_list NK_RP */ -{ pCxt->pRootNode = createCreateIndexStmt(pCxt, INDEX_TYPE_FULLTEXT, yymsp[-6].minor.yy137, &yymsp[-5].minor.yy209, &yymsp[-3].minor.yy209, yymsp[-1].minor.yy424, NULL); } + case 220: /* cmd ::= CREATE FULLTEXT INDEX not_exists_opt index_name ON table_name NK_LP col_name_list NK_RP */ +{ pCxt->pRootNode = createCreateIndexStmt(pCxt, INDEX_TYPE_FULLTEXT, yymsp[-6].minor.yy621, &yymsp[-5].minor.yy421, &yymsp[-3].minor.yy421, yymsp[-1].minor.yy530, NULL); } break; - case 216: /* cmd ::= DROP INDEX exists_opt index_name ON table_name */ -{ pCxt->pRootNode = createDropIndexStmt(pCxt, yymsp[-3].minor.yy137, &yymsp[-2].minor.yy209, &yymsp[0].minor.yy209); } + case 221: /* cmd ::= DROP INDEX exists_opt index_name ON table_name */ +{ pCxt->pRootNode = createDropIndexStmt(pCxt, yymsp[-3].minor.yy621, &yymsp[-2].minor.yy421, &yymsp[0].minor.yy421); } break; - case 218: /* index_options ::= FUNCTION NK_LP func_list NK_RP INTERVAL NK_LP duration_literal NK_RP sliding_opt */ -{ yymsp[-8].minor.yy632 = createIndexOption(pCxt, yymsp[-6].minor.yy424, releaseRawExprNode(pCxt, yymsp[-2].minor.yy632), NULL, yymsp[0].minor.yy632); } + case 223: /* index_options ::= FUNCTION NK_LP func_list NK_RP INTERVAL NK_LP duration_literal NK_RP sliding_opt */ +{ yymsp[-8].minor.yy674 = createIndexOption(pCxt, yymsp[-6].minor.yy530, releaseRawExprNode(pCxt, yymsp[-2].minor.yy674), NULL, yymsp[0].minor.yy674); } break; - case 219: /* index_options ::= FUNCTION NK_LP func_list NK_RP INTERVAL NK_LP duration_literal NK_COMMA duration_literal NK_RP sliding_opt */ -{ yymsp[-10].minor.yy632 = createIndexOption(pCxt, yymsp[-8].minor.yy424, releaseRawExprNode(pCxt, yymsp[-4].minor.yy632), releaseRawExprNode(pCxt, yymsp[-2].minor.yy632), yymsp[0].minor.yy632); } + case 224: /* index_options ::= FUNCTION NK_LP func_list NK_RP INTERVAL NK_LP duration_literal NK_COMMA duration_literal NK_RP sliding_opt */ +{ yymsp[-10].minor.yy674 = createIndexOption(pCxt, yymsp[-8].minor.yy530, releaseRawExprNode(pCxt, yymsp[-4].minor.yy674), releaseRawExprNode(pCxt, yymsp[-2].minor.yy674), yymsp[0].minor.yy674); } break; - case 222: /* func ::= function_name NK_LP expression_list NK_RP */ -{ yylhsminor.yy632 = createFunctionNode(pCxt, &yymsp[-3].minor.yy209, yymsp[-1].minor.yy424); } - yymsp[-3].minor.yy632 = yylhsminor.yy632; + case 227: /* func ::= function_name NK_LP expression_list NK_RP */ +{ yylhsminor.yy674 = createFunctionNode(pCxt, &yymsp[-3].minor.yy421, yymsp[-1].minor.yy530); } + yymsp[-3].minor.yy674 = yylhsminor.yy674; break; - case 223: /* cmd ::= CREATE TOPIC not_exists_opt topic_name AS query_expression */ -{ pCxt->pRootNode = createCreateTopicStmt(pCxt, yymsp[-3].minor.yy137, &yymsp[-2].minor.yy209, yymsp[0].minor.yy632, NULL, NULL); } + case 228: /* cmd ::= CREATE TOPIC not_exists_opt topic_name AS query_expression */ +{ pCxt->pRootNode = createCreateTopicStmt(pCxt, yymsp[-3].minor.yy621, &yymsp[-2].minor.yy421, yymsp[0].minor.yy674, NULL, NULL); } break; - case 224: /* cmd ::= CREATE TOPIC not_exists_opt topic_name AS DATABASE db_name */ -{ pCxt->pRootNode = createCreateTopicStmt(pCxt, yymsp[-4].minor.yy137, &yymsp[-3].minor.yy209, NULL, &yymsp[0].minor.yy209, NULL); } + case 229: /* cmd ::= CREATE TOPIC not_exists_opt topic_name AS DATABASE db_name */ +{ pCxt->pRootNode = createCreateTopicStmt(pCxt, yymsp[-4].minor.yy621, &yymsp[-3].minor.yy421, NULL, &yymsp[0].minor.yy421, NULL); } break; - case 225: /* cmd ::= CREATE TOPIC not_exists_opt topic_name AS STABLE full_table_name */ -{ pCxt->pRootNode = createCreateTopicStmt(pCxt, yymsp[-4].minor.yy137, &yymsp[-3].minor.yy209, NULL, NULL, yymsp[0].minor.yy632); } + case 230: /* cmd ::= CREATE TOPIC not_exists_opt topic_name AS STABLE full_table_name */ +{ pCxt->pRootNode = createCreateTopicStmt(pCxt, yymsp[-4].minor.yy621, &yymsp[-3].minor.yy421, NULL, NULL, yymsp[0].minor.yy674); } break; - case 226: /* cmd ::= DROP TOPIC exists_opt topic_name */ -{ pCxt->pRootNode = createDropTopicStmt(pCxt, yymsp[-1].minor.yy137, &yymsp[0].minor.yy209); } + case 231: /* cmd ::= DROP TOPIC exists_opt topic_name */ +{ pCxt->pRootNode = createDropTopicStmt(pCxt, yymsp[-1].minor.yy621, &yymsp[0].minor.yy421); } break; - case 227: /* cmd ::= DROP CONSUMER GROUP exists_opt cgroup_name ON topic_name */ -{ pCxt->pRootNode = createDropCGroupStmt(pCxt, yymsp[-3].minor.yy137, &yymsp[-2].minor.yy209, &yymsp[0].minor.yy209); } + case 232: /* cmd ::= DROP CONSUMER GROUP exists_opt cgroup_name ON topic_name */ +{ pCxt->pRootNode = createDropCGroupStmt(pCxt, yymsp[-3].minor.yy621, &yymsp[-2].minor.yy421, &yymsp[0].minor.yy421); } break; - case 228: /* cmd ::= DESC full_table_name */ - case 229: /* cmd ::= DESCRIBE full_table_name */ yytestcase(yyruleno==229); -{ pCxt->pRootNode = createDescribeStmt(pCxt, yymsp[0].minor.yy632); } + case 233: /* cmd ::= DESC full_table_name */ + case 234: /* cmd ::= DESCRIBE full_table_name */ yytestcase(yyruleno==234); +{ pCxt->pRootNode = createDescribeStmt(pCxt, yymsp[0].minor.yy674); } break; - case 230: /* cmd ::= RESET QUERY CACHE */ + case 235: /* cmd ::= RESET QUERY CACHE */ { pCxt->pRootNode = createResetQueryCacheStmt(pCxt); } break; - case 231: /* cmd ::= EXPLAIN analyze_opt explain_options query_expression */ -{ pCxt->pRootNode = createExplainStmt(pCxt, yymsp[-2].minor.yy137, yymsp[-1].minor.yy632, yymsp[0].minor.yy632); } + case 236: /* cmd ::= EXPLAIN analyze_opt explain_options query_expression */ +{ pCxt->pRootNode = createExplainStmt(pCxt, yymsp[-2].minor.yy621, yymsp[-1].minor.yy674, yymsp[0].minor.yy674); } break; - case 233: /* analyze_opt ::= ANALYZE */ - case 241: /* agg_func_opt ::= AGGREGATE */ yytestcase(yyruleno==241); - case 392: /* set_quantifier_opt ::= DISTINCT */ yytestcase(yyruleno==392); -{ yymsp[0].minor.yy137 = true; } + case 238: /* analyze_opt ::= ANALYZE */ + case 246: /* agg_func_opt ::= AGGREGATE */ yytestcase(yyruleno==246); + case 397: /* set_quantifier_opt ::= DISTINCT */ yytestcase(yyruleno==397); +{ yymsp[0].minor.yy621 = true; } break; - case 234: /* explain_options ::= */ -{ yymsp[1].minor.yy632 = createDefaultExplainOptions(pCxt); } + case 239: /* explain_options ::= */ +{ yymsp[1].minor.yy674 = createDefaultExplainOptions(pCxt); } break; - case 235: /* explain_options ::= explain_options VERBOSE NK_BOOL */ -{ yylhsminor.yy632 = setExplainVerbose(pCxt, yymsp[-2].minor.yy632, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; + case 240: /* explain_options ::= explain_options VERBOSE NK_BOOL */ +{ yylhsminor.yy674 = setExplainVerbose(pCxt, yymsp[-2].minor.yy674, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; - case 236: /* explain_options ::= explain_options RATIO NK_FLOAT */ -{ yylhsminor.yy632 = setExplainRatio(pCxt, yymsp[-2].minor.yy632, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; + case 241: /* explain_options ::= explain_options RATIO NK_FLOAT */ +{ yylhsminor.yy674 = setExplainRatio(pCxt, yymsp[-2].minor.yy674, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; - case 237: /* cmd ::= COMPACT VNODES IN NK_LP integer_list NK_RP */ -{ pCxt->pRootNode = createCompactStmt(pCxt, yymsp[-1].minor.yy424); } + case 242: /* cmd ::= COMPACT VNODES IN NK_LP integer_list NK_RP */ +{ pCxt->pRootNode = createCompactStmt(pCxt, yymsp[-1].minor.yy530); } break; - case 238: /* cmd ::= CREATE agg_func_opt FUNCTION not_exists_opt function_name AS NK_STRING OUTPUTTYPE type_name bufsize_opt */ -{ pCxt->pRootNode = createCreateFunctionStmt(pCxt, yymsp[-6].minor.yy137, yymsp[-8].minor.yy137, &yymsp[-5].minor.yy209, &yymsp[-3].minor.yy0, yymsp[-1].minor.yy304, yymsp[0].minor.yy100); } + case 243: /* cmd ::= CREATE agg_func_opt FUNCTION not_exists_opt function_name AS NK_STRING OUTPUTTYPE type_name bufsize_opt */ +{ pCxt->pRootNode = createCreateFunctionStmt(pCxt, yymsp[-6].minor.yy621, yymsp[-8].minor.yy621, &yymsp[-5].minor.yy421, &yymsp[-3].minor.yy0, yymsp[-1].minor.yy690, yymsp[0].minor.yy42); } break; - case 239: /* cmd ::= DROP FUNCTION exists_opt function_name */ -{ pCxt->pRootNode = createDropFunctionStmt(pCxt, yymsp[-1].minor.yy137, &yymsp[0].minor.yy209); } + case 244: /* cmd ::= DROP FUNCTION exists_opt function_name */ +{ pCxt->pRootNode = createDropFunctionStmt(pCxt, yymsp[-1].minor.yy621, &yymsp[0].minor.yy421); } break; - case 242: /* bufsize_opt ::= */ -{ yymsp[1].minor.yy100 = 0; } + case 247: /* bufsize_opt ::= */ +{ yymsp[1].minor.yy42 = 0; } break; - case 243: /* bufsize_opt ::= BUFSIZE NK_INTEGER */ -{ yymsp[-1].minor.yy100 = taosStr2Int32(yymsp[0].minor.yy0.z, NULL, 10); } + case 248: /* bufsize_opt ::= BUFSIZE NK_INTEGER */ +{ yymsp[-1].minor.yy42 = taosStr2Int32(yymsp[0].minor.yy0.z, NULL, 10); } break; - case 244: /* cmd ::= CREATE STREAM not_exists_opt stream_name stream_options into_opt AS query_expression */ -{ pCxt->pRootNode = createCreateStreamStmt(pCxt, yymsp[-5].minor.yy137, &yymsp[-4].minor.yy209, yymsp[-2].minor.yy632, yymsp[-3].minor.yy632, yymsp[0].minor.yy632); } + case 249: /* cmd ::= CREATE STREAM not_exists_opt stream_name stream_options into_opt AS query_expression */ +{ pCxt->pRootNode = createCreateStreamStmt(pCxt, yymsp[-5].minor.yy621, &yymsp[-4].minor.yy421, yymsp[-2].minor.yy674, yymsp[-3].minor.yy674, yymsp[0].minor.yy674); } break; - case 245: /* cmd ::= DROP STREAM exists_opt stream_name */ -{ pCxt->pRootNode = createDropStreamStmt(pCxt, yymsp[-1].minor.yy137, &yymsp[0].minor.yy209); } + case 250: /* cmd ::= DROP STREAM exists_opt stream_name */ +{ pCxt->pRootNode = createDropStreamStmt(pCxt, yymsp[-1].minor.yy621, &yymsp[0].minor.yy421); } break; - case 247: /* into_opt ::= INTO full_table_name */ - case 373: /* from_clause ::= FROM table_reference_list */ yytestcase(yyruleno==373); - case 403: /* where_clause_opt ::= WHERE search_condition */ yytestcase(yyruleno==403); - case 426: /* having_clause_opt ::= HAVING search_condition */ yytestcase(yyruleno==426); -{ yymsp[-1].minor.yy632 = yymsp[0].minor.yy632; } + case 252: /* into_opt ::= INTO full_table_name */ + case 378: /* from_clause ::= FROM table_reference_list */ yytestcase(yyruleno==378); + case 408: /* where_clause_opt ::= WHERE search_condition */ yytestcase(yyruleno==408); + case 431: /* having_clause_opt ::= HAVING search_condition */ yytestcase(yyruleno==431); +{ yymsp[-1].minor.yy674 = yymsp[0].minor.yy674; } break; - case 248: /* stream_options ::= */ -{ yymsp[1].minor.yy632 = createStreamOptions(pCxt); } + case 253: /* stream_options ::= */ +{ yymsp[1].minor.yy674 = createStreamOptions(pCxt); } break; - case 249: /* stream_options ::= stream_options TRIGGER AT_ONCE */ -{ ((SStreamOptions*)yymsp[-2].minor.yy632)->triggerType = STREAM_TRIGGER_AT_ONCE; yylhsminor.yy632 = yymsp[-2].minor.yy632; } - yymsp[-2].minor.yy632 = yylhsminor.yy632; + case 254: /* stream_options ::= stream_options TRIGGER AT_ONCE */ +{ ((SStreamOptions*)yymsp[-2].minor.yy674)->triggerType = STREAM_TRIGGER_AT_ONCE; yylhsminor.yy674 = yymsp[-2].minor.yy674; } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; - case 250: /* stream_options ::= stream_options TRIGGER WINDOW_CLOSE */ -{ ((SStreamOptions*)yymsp[-2].minor.yy632)->triggerType = STREAM_TRIGGER_WINDOW_CLOSE; yylhsminor.yy632 = yymsp[-2].minor.yy632; } - yymsp[-2].minor.yy632 = yylhsminor.yy632; + case 255: /* stream_options ::= stream_options TRIGGER WINDOW_CLOSE */ +{ ((SStreamOptions*)yymsp[-2].minor.yy674)->triggerType = STREAM_TRIGGER_WINDOW_CLOSE; yylhsminor.yy674 = yymsp[-2].minor.yy674; } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; - case 251: /* stream_options ::= stream_options TRIGGER MAX_DELAY duration_literal */ -{ ((SStreamOptions*)yymsp[-3].minor.yy632)->triggerType = STREAM_TRIGGER_MAX_DELAY; ((SStreamOptions*)yymsp[-3].minor.yy632)->pDelay = releaseRawExprNode(pCxt, yymsp[0].minor.yy632); yylhsminor.yy632 = yymsp[-3].minor.yy632; } - yymsp[-3].minor.yy632 = yylhsminor.yy632; + case 256: /* stream_options ::= stream_options TRIGGER MAX_DELAY duration_literal */ +{ ((SStreamOptions*)yymsp[-3].minor.yy674)->triggerType = STREAM_TRIGGER_MAX_DELAY; ((SStreamOptions*)yymsp[-3].minor.yy674)->pDelay = releaseRawExprNode(pCxt, yymsp[0].minor.yy674); yylhsminor.yy674 = yymsp[-3].minor.yy674; } + yymsp[-3].minor.yy674 = yylhsminor.yy674; break; - case 252: /* stream_options ::= stream_options WATERMARK duration_literal */ -{ ((SStreamOptions*)yymsp[-2].minor.yy632)->pWatermark = releaseRawExprNode(pCxt, yymsp[0].minor.yy632); yylhsminor.yy632 = yymsp[-2].minor.yy632; } - yymsp[-2].minor.yy632 = yylhsminor.yy632; + case 257: /* stream_options ::= stream_options WATERMARK duration_literal */ +{ ((SStreamOptions*)yymsp[-2].minor.yy674)->pWatermark = releaseRawExprNode(pCxt, yymsp[0].minor.yy674); yylhsminor.yy674 = yymsp[-2].minor.yy674; } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; - case 253: /* cmd ::= KILL CONNECTION NK_INTEGER */ + case 258: /* cmd ::= KILL CONNECTION NK_INTEGER */ { pCxt->pRootNode = createKillStmt(pCxt, QUERY_NODE_KILL_CONNECTION_STMT, &yymsp[0].minor.yy0); } break; - case 254: /* cmd ::= KILL QUERY NK_STRING */ + case 259: /* cmd ::= KILL QUERY NK_STRING */ { pCxt->pRootNode = createKillQueryStmt(pCxt, &yymsp[0].minor.yy0); } break; - case 255: /* cmd ::= KILL TRANSACTION NK_INTEGER */ + case 260: /* cmd ::= KILL TRANSACTION NK_INTEGER */ { pCxt->pRootNode = createKillStmt(pCxt, QUERY_NODE_KILL_TRANSACTION_STMT, &yymsp[0].minor.yy0); } break; - case 256: /* cmd ::= BALANCE VGROUP */ + case 261: /* cmd ::= BALANCE VGROUP */ { pCxt->pRootNode = createBalanceVgroupStmt(pCxt); } break; - case 257: /* cmd ::= MERGE VGROUP NK_INTEGER NK_INTEGER */ + case 262: /* cmd ::= MERGE VGROUP NK_INTEGER NK_INTEGER */ { pCxt->pRootNode = createMergeVgroupStmt(pCxt, &yymsp[-1].minor.yy0, &yymsp[0].minor.yy0); } break; - case 258: /* cmd ::= REDISTRIBUTE VGROUP NK_INTEGER dnode_list */ -{ pCxt->pRootNode = createRedistributeVgroupStmt(pCxt, &yymsp[-1].minor.yy0, yymsp[0].minor.yy424); } + case 263: /* cmd ::= REDISTRIBUTE VGROUP NK_INTEGER dnode_list */ +{ pCxt->pRootNode = createRedistributeVgroupStmt(pCxt, &yymsp[-1].minor.yy0, yymsp[0].minor.yy530); } break; - case 259: /* cmd ::= SPLIT VGROUP NK_INTEGER */ + case 264: /* cmd ::= SPLIT VGROUP NK_INTEGER */ { pCxt->pRootNode = createSplitVgroupStmt(pCxt, &yymsp[0].minor.yy0); } break; - case 260: /* dnode_list ::= DNODE NK_INTEGER */ -{ yymsp[-1].minor.yy424 = createNodeList(pCxt, createValueNode(pCxt, TSDB_DATA_TYPE_BIGINT, &yymsp[0].minor.yy0)); } - break; - case 262: /* cmd ::= SYNCDB db_name REPLICA */ -{ pCxt->pRootNode = createSyncdbStmt(pCxt, &yymsp[-1].minor.yy209); } - break; - case 263: /* cmd ::= DELETE FROM full_table_name where_clause_opt */ -{ pCxt->pRootNode = createDeleteStmt(pCxt, yymsp[-1].minor.yy632, yymsp[0].minor.yy632); } - break; - case 265: /* literal ::= NK_INTEGER */ -{ yylhsminor.yy632 = createRawExprNode(pCxt, &yymsp[0].minor.yy0, createValueNode(pCxt, TSDB_DATA_TYPE_BIGINT, &yymsp[0].minor.yy0)); } - yymsp[0].minor.yy632 = yylhsminor.yy632; - break; - case 266: /* literal ::= NK_FLOAT */ -{ yylhsminor.yy632 = createRawExprNode(pCxt, &yymsp[0].minor.yy0, createValueNode(pCxt, TSDB_DATA_TYPE_DOUBLE, &yymsp[0].minor.yy0)); } - yymsp[0].minor.yy632 = yylhsminor.yy632; - break; - case 267: /* literal ::= NK_STRING */ -{ yylhsminor.yy632 = createRawExprNode(pCxt, &yymsp[0].minor.yy0, createValueNode(pCxt, TSDB_DATA_TYPE_BINARY, &yymsp[0].minor.yy0)); } - yymsp[0].minor.yy632 = yylhsminor.yy632; - break; - case 268: /* literal ::= NK_BOOL */ -{ yylhsminor.yy632 = createRawExprNode(pCxt, &yymsp[0].minor.yy0, createValueNode(pCxt, TSDB_DATA_TYPE_BOOL, &yymsp[0].minor.yy0)); } - yymsp[0].minor.yy632 = yylhsminor.yy632; - break; - case 269: /* literal ::= TIMESTAMP NK_STRING */ -{ yylhsminor.yy632 = createRawExprNodeExt(pCxt, &yymsp[-1].minor.yy0, &yymsp[0].minor.yy0, createValueNode(pCxt, TSDB_DATA_TYPE_TIMESTAMP, &yymsp[0].minor.yy0)); } - yymsp[-1].minor.yy632 = yylhsminor.yy632; - break; - case 270: /* literal ::= duration_literal */ - case 280: /* signed_literal ::= signed */ yytestcase(yyruleno==280); - case 300: /* expression ::= literal */ yytestcase(yyruleno==300); - case 301: /* expression ::= pseudo_column */ yytestcase(yyruleno==301); - case 302: /* expression ::= column_reference */ yytestcase(yyruleno==302); - case 303: /* expression ::= function_expression */ yytestcase(yyruleno==303); - case 304: /* expression ::= subquery */ yytestcase(yyruleno==304); - case 329: /* function_expression ::= literal_func */ yytestcase(yyruleno==329); - case 365: /* boolean_value_expression ::= boolean_primary */ yytestcase(yyruleno==365); - case 369: /* boolean_primary ::= predicate */ yytestcase(yyruleno==369); - case 371: /* common_expression ::= expression */ yytestcase(yyruleno==371); - case 372: /* common_expression ::= boolean_value_expression */ yytestcase(yyruleno==372); - case 374: /* table_reference_list ::= table_reference */ yytestcase(yyruleno==374); - case 376: /* table_reference ::= table_primary */ yytestcase(yyruleno==376); - case 377: /* table_reference ::= joined_table */ yytestcase(yyruleno==377); - case 381: /* table_primary ::= parenthesized_joined_table */ yytestcase(yyruleno==381); - case 428: /* query_expression_body ::= query_primary */ yytestcase(yyruleno==428); - case 431: /* query_primary ::= query_specification */ yytestcase(yyruleno==431); -{ yylhsminor.yy632 = yymsp[0].minor.yy632; } - yymsp[0].minor.yy632 = yylhsminor.yy632; - break; - case 271: /* literal ::= NULL */ -{ yylhsminor.yy632 = createRawExprNode(pCxt, &yymsp[0].minor.yy0, createValueNode(pCxt, TSDB_DATA_TYPE_NULL, &yymsp[0].minor.yy0)); } - yymsp[0].minor.yy632 = yylhsminor.yy632; - break; - case 272: /* literal ::= NK_QUESTION */ -{ yylhsminor.yy632 = createRawExprNode(pCxt, &yymsp[0].minor.yy0, createPlaceholderValueNode(pCxt, &yymsp[0].minor.yy0)); } - yymsp[0].minor.yy632 = yylhsminor.yy632; - break; - case 273: /* duration_literal ::= NK_VARIABLE */ -{ yylhsminor.yy632 = createRawExprNode(pCxt, &yymsp[0].minor.yy0, createDurationValueNode(pCxt, &yymsp[0].minor.yy0)); } - yymsp[0].minor.yy632 = yylhsminor.yy632; - break; - case 274: /* signed ::= NK_INTEGER */ -{ yylhsminor.yy632 = createValueNode(pCxt, TSDB_DATA_TYPE_BIGINT, &yymsp[0].minor.yy0); } - yymsp[0].minor.yy632 = yylhsminor.yy632; - break; - case 275: /* signed ::= NK_PLUS NK_INTEGER */ -{ yymsp[-1].minor.yy632 = createValueNode(pCxt, TSDB_DATA_TYPE_BIGINT, &yymsp[0].minor.yy0); } - break; - case 276: /* signed ::= NK_MINUS NK_INTEGER */ + case 265: /* dnode_list ::= DNODE NK_INTEGER */ +{ yymsp[-1].minor.yy530 = createNodeList(pCxt, createValueNode(pCxt, TSDB_DATA_TYPE_BIGINT, &yymsp[0].minor.yy0)); } + break; + case 267: /* cmd ::= SYNCDB db_name REPLICA */ +{ pCxt->pRootNode = createSyncdbStmt(pCxt, &yymsp[-1].minor.yy421); } + break; + case 268: /* cmd ::= DELETE FROM full_table_name where_clause_opt */ +{ pCxt->pRootNode = createDeleteStmt(pCxt, yymsp[-1].minor.yy674, yymsp[0].minor.yy674); } + break; + case 270: /* literal ::= NK_INTEGER */ +{ yylhsminor.yy674 = createRawExprNode(pCxt, &yymsp[0].minor.yy0, createValueNode(pCxt, TSDB_DATA_TYPE_BIGINT, &yymsp[0].minor.yy0)); } + yymsp[0].minor.yy674 = yylhsminor.yy674; + break; + case 271: /* literal ::= NK_FLOAT */ +{ yylhsminor.yy674 = createRawExprNode(pCxt, &yymsp[0].minor.yy0, createValueNode(pCxt, TSDB_DATA_TYPE_DOUBLE, &yymsp[0].minor.yy0)); } + yymsp[0].minor.yy674 = yylhsminor.yy674; + break; + case 272: /* literal ::= NK_STRING */ +{ yylhsminor.yy674 = createRawExprNode(pCxt, &yymsp[0].minor.yy0, createValueNode(pCxt, TSDB_DATA_TYPE_BINARY, &yymsp[0].minor.yy0)); } + yymsp[0].minor.yy674 = yylhsminor.yy674; + break; + case 273: /* literal ::= NK_BOOL */ +{ yylhsminor.yy674 = createRawExprNode(pCxt, &yymsp[0].minor.yy0, createValueNode(pCxt, TSDB_DATA_TYPE_BOOL, &yymsp[0].minor.yy0)); } + yymsp[0].minor.yy674 = yylhsminor.yy674; + break; + case 274: /* literal ::= TIMESTAMP NK_STRING */ +{ yylhsminor.yy674 = createRawExprNodeExt(pCxt, &yymsp[-1].minor.yy0, &yymsp[0].minor.yy0, createValueNode(pCxt, TSDB_DATA_TYPE_TIMESTAMP, &yymsp[0].minor.yy0)); } + yymsp[-1].minor.yy674 = yylhsminor.yy674; + break; + case 275: /* literal ::= duration_literal */ + case 285: /* signed_literal ::= signed */ yytestcase(yyruleno==285); + case 305: /* expression ::= literal */ yytestcase(yyruleno==305); + case 306: /* expression ::= pseudo_column */ yytestcase(yyruleno==306); + case 307: /* expression ::= column_reference */ yytestcase(yyruleno==307); + case 308: /* expression ::= function_expression */ yytestcase(yyruleno==308); + case 309: /* expression ::= subquery */ yytestcase(yyruleno==309); + case 334: /* function_expression ::= literal_func */ yytestcase(yyruleno==334); + case 370: /* boolean_value_expression ::= boolean_primary */ yytestcase(yyruleno==370); + case 374: /* boolean_primary ::= predicate */ yytestcase(yyruleno==374); + case 376: /* common_expression ::= expression */ yytestcase(yyruleno==376); + case 377: /* common_expression ::= boolean_value_expression */ yytestcase(yyruleno==377); + case 379: /* table_reference_list ::= table_reference */ yytestcase(yyruleno==379); + case 381: /* table_reference ::= table_primary */ yytestcase(yyruleno==381); + case 382: /* table_reference ::= joined_table */ yytestcase(yyruleno==382); + case 386: /* table_primary ::= parenthesized_joined_table */ yytestcase(yyruleno==386); + case 433: /* query_expression_body ::= query_primary */ yytestcase(yyruleno==433); + case 436: /* query_primary ::= query_specification */ yytestcase(yyruleno==436); +{ yylhsminor.yy674 = yymsp[0].minor.yy674; } + yymsp[0].minor.yy674 = yylhsminor.yy674; + break; + case 276: /* literal ::= NULL */ +{ yylhsminor.yy674 = createRawExprNode(pCxt, &yymsp[0].minor.yy0, createValueNode(pCxt, TSDB_DATA_TYPE_NULL, &yymsp[0].minor.yy0)); } + yymsp[0].minor.yy674 = yylhsminor.yy674; + break; + case 277: /* literal ::= NK_QUESTION */ +{ yylhsminor.yy674 = createRawExprNode(pCxt, &yymsp[0].minor.yy0, createPlaceholderValueNode(pCxt, &yymsp[0].minor.yy0)); } + yymsp[0].minor.yy674 = yylhsminor.yy674; + break; + case 278: /* duration_literal ::= NK_VARIABLE */ +{ yylhsminor.yy674 = createRawExprNode(pCxt, &yymsp[0].minor.yy0, createDurationValueNode(pCxt, &yymsp[0].minor.yy0)); } + yymsp[0].minor.yy674 = yylhsminor.yy674; + break; + case 279: /* signed ::= NK_INTEGER */ +{ yylhsminor.yy674 = createValueNode(pCxt, TSDB_DATA_TYPE_BIGINT, &yymsp[0].minor.yy0); } + yymsp[0].minor.yy674 = yylhsminor.yy674; + break; + case 280: /* signed ::= NK_PLUS NK_INTEGER */ +{ yymsp[-1].minor.yy674 = createValueNode(pCxt, TSDB_DATA_TYPE_BIGINT, &yymsp[0].minor.yy0); } + break; + case 281: /* signed ::= NK_MINUS NK_INTEGER */ { SToken t = yymsp[-1].minor.yy0; t.n = (yymsp[0].minor.yy0.z + yymsp[0].minor.yy0.n) - yymsp[-1].minor.yy0.z; - yylhsminor.yy632 = createValueNode(pCxt, TSDB_DATA_TYPE_BIGINT, &t); + yylhsminor.yy674 = createValueNode(pCxt, TSDB_DATA_TYPE_BIGINT, &t); } - yymsp[-1].minor.yy632 = yylhsminor.yy632; + yymsp[-1].minor.yy674 = yylhsminor.yy674; break; - case 277: /* signed ::= NK_FLOAT */ -{ yylhsminor.yy632 = createValueNode(pCxt, TSDB_DATA_TYPE_DOUBLE, &yymsp[0].minor.yy0); } - yymsp[0].minor.yy632 = yylhsminor.yy632; + case 282: /* signed ::= NK_FLOAT */ +{ yylhsminor.yy674 = createValueNode(pCxt, TSDB_DATA_TYPE_DOUBLE, &yymsp[0].minor.yy0); } + yymsp[0].minor.yy674 = yylhsminor.yy674; break; - case 278: /* signed ::= NK_PLUS NK_FLOAT */ -{ yymsp[-1].minor.yy632 = createValueNode(pCxt, TSDB_DATA_TYPE_DOUBLE, &yymsp[0].minor.yy0); } + case 283: /* signed ::= NK_PLUS NK_FLOAT */ +{ yymsp[-1].minor.yy674 = createValueNode(pCxt, TSDB_DATA_TYPE_DOUBLE, &yymsp[0].minor.yy0); } break; - case 279: /* signed ::= NK_MINUS NK_FLOAT */ + case 284: /* signed ::= NK_MINUS NK_FLOAT */ { SToken t = yymsp[-1].minor.yy0; t.n = (yymsp[0].minor.yy0.z + yymsp[0].minor.yy0.n) - yymsp[-1].minor.yy0.z; - yylhsminor.yy632 = createValueNode(pCxt, TSDB_DATA_TYPE_DOUBLE, &t); + yylhsminor.yy674 = createValueNode(pCxt, TSDB_DATA_TYPE_DOUBLE, &t); } - yymsp[-1].minor.yy632 = yylhsminor.yy632; + yymsp[-1].minor.yy674 = yylhsminor.yy674; break; - case 281: /* signed_literal ::= NK_STRING */ -{ yylhsminor.yy632 = createValueNode(pCxt, TSDB_DATA_TYPE_BINARY, &yymsp[0].minor.yy0); } - yymsp[0].minor.yy632 = yylhsminor.yy632; + case 286: /* signed_literal ::= NK_STRING */ +{ yylhsminor.yy674 = createValueNode(pCxt, TSDB_DATA_TYPE_BINARY, &yymsp[0].minor.yy0); } + yymsp[0].minor.yy674 = yylhsminor.yy674; break; - case 282: /* signed_literal ::= NK_BOOL */ -{ yylhsminor.yy632 = createValueNode(pCxt, TSDB_DATA_TYPE_BOOL, &yymsp[0].minor.yy0); } - yymsp[0].minor.yy632 = yylhsminor.yy632; + case 287: /* signed_literal ::= NK_BOOL */ +{ yylhsminor.yy674 = createValueNode(pCxt, TSDB_DATA_TYPE_BOOL, &yymsp[0].minor.yy0); } + yymsp[0].minor.yy674 = yylhsminor.yy674; break; - case 283: /* signed_literal ::= TIMESTAMP NK_STRING */ -{ yymsp[-1].minor.yy632 = createValueNode(pCxt, TSDB_DATA_TYPE_TIMESTAMP, &yymsp[0].minor.yy0); } + case 288: /* signed_literal ::= TIMESTAMP NK_STRING */ +{ yymsp[-1].minor.yy674 = createValueNode(pCxt, TSDB_DATA_TYPE_TIMESTAMP, &yymsp[0].minor.yy0); } break; - case 284: /* signed_literal ::= duration_literal */ - case 286: /* signed_literal ::= literal_func */ yytestcase(yyruleno==286); - case 343: /* star_func_para ::= expression */ yytestcase(yyruleno==343); - case 398: /* select_item ::= common_expression */ yytestcase(yyruleno==398); - case 444: /* search_condition ::= common_expression */ yytestcase(yyruleno==444); -{ yylhsminor.yy632 = releaseRawExprNode(pCxt, yymsp[0].minor.yy632); } - yymsp[0].minor.yy632 = yylhsminor.yy632; + case 289: /* signed_literal ::= duration_literal */ + case 291: /* signed_literal ::= literal_func */ yytestcase(yyruleno==291); + case 348: /* star_func_para ::= expression */ yytestcase(yyruleno==348); + case 403: /* select_item ::= common_expression */ yytestcase(yyruleno==403); + case 449: /* search_condition ::= common_expression */ yytestcase(yyruleno==449); +{ yylhsminor.yy674 = releaseRawExprNode(pCxt, yymsp[0].minor.yy674); } + yymsp[0].minor.yy674 = yylhsminor.yy674; break; - case 285: /* signed_literal ::= NULL */ -{ yylhsminor.yy632 = createValueNode(pCxt, TSDB_DATA_TYPE_NULL, &yymsp[0].minor.yy0); } - yymsp[0].minor.yy632 = yylhsminor.yy632; + case 290: /* signed_literal ::= NULL */ +{ yylhsminor.yy674 = createValueNode(pCxt, TSDB_DATA_TYPE_NULL, &yymsp[0].minor.yy0); } + yymsp[0].minor.yy674 = yylhsminor.yy674; break; - case 305: /* expression ::= NK_LP expression NK_RP */ - case 370: /* boolean_primary ::= NK_LP boolean_value_expression NK_RP */ yytestcase(yyruleno==370); -{ yylhsminor.yy632 = createRawExprNodeExt(pCxt, &yymsp[-2].minor.yy0, &yymsp[0].minor.yy0, releaseRawExprNode(pCxt, yymsp[-1].minor.yy632)); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; + case 310: /* expression ::= NK_LP expression NK_RP */ + case 375: /* boolean_primary ::= NK_LP boolean_value_expression NK_RP */ yytestcase(yyruleno==375); +{ yylhsminor.yy674 = createRawExprNodeExt(pCxt, &yymsp[-2].minor.yy0, &yymsp[0].minor.yy0, releaseRawExprNode(pCxt, yymsp[-1].minor.yy674)); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; - case 306: /* expression ::= NK_PLUS expression */ + case 311: /* expression ::= NK_PLUS expression */ { - SToken t = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy632); - yylhsminor.yy632 = createRawExprNodeExt(pCxt, &yymsp[-1].minor.yy0, &t, releaseRawExprNode(pCxt, yymsp[0].minor.yy632)); + SToken t = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy674); + yylhsminor.yy674 = createRawExprNodeExt(pCxt, &yymsp[-1].minor.yy0, &t, releaseRawExprNode(pCxt, yymsp[0].minor.yy674)); } - yymsp[-1].minor.yy632 = yylhsminor.yy632; + yymsp[-1].minor.yy674 = yylhsminor.yy674; break; - case 307: /* expression ::= NK_MINUS expression */ + case 312: /* expression ::= NK_MINUS expression */ { - SToken t = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy632); - yylhsminor.yy632 = createRawExprNodeExt(pCxt, &yymsp[-1].minor.yy0, &t, createOperatorNode(pCxt, OP_TYPE_MINUS, releaseRawExprNode(pCxt, yymsp[0].minor.yy632), NULL)); + SToken t = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy674); + yylhsminor.yy674 = createRawExprNodeExt(pCxt, &yymsp[-1].minor.yy0, &t, createOperatorNode(pCxt, OP_TYPE_MINUS, releaseRawExprNode(pCxt, yymsp[0].minor.yy674), NULL)); } - yymsp[-1].minor.yy632 = yylhsminor.yy632; + yymsp[-1].minor.yy674 = yylhsminor.yy674; break; - case 308: /* expression ::= expression NK_PLUS expression */ + case 313: /* expression ::= expression NK_PLUS expression */ { - SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy632); - SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy632); - yylhsminor.yy632 = createRawExprNodeExt(pCxt, &s, &e, createOperatorNode(pCxt, OP_TYPE_ADD, releaseRawExprNode(pCxt, yymsp[-2].minor.yy632), releaseRawExprNode(pCxt, yymsp[0].minor.yy632))); + SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy674); + SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy674); + yylhsminor.yy674 = createRawExprNodeExt(pCxt, &s, &e, createOperatorNode(pCxt, OP_TYPE_ADD, releaseRawExprNode(pCxt, yymsp[-2].minor.yy674), releaseRawExprNode(pCxt, yymsp[0].minor.yy674))); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; - case 309: /* expression ::= expression NK_MINUS expression */ + case 314: /* expression ::= expression NK_MINUS expression */ { - SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy632); - SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy632); - yylhsminor.yy632 = createRawExprNodeExt(pCxt, &s, &e, createOperatorNode(pCxt, OP_TYPE_SUB, releaseRawExprNode(pCxt, yymsp[-2].minor.yy632), releaseRawExprNode(pCxt, yymsp[0].minor.yy632))); + SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy674); + SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy674); + yylhsminor.yy674 = createRawExprNodeExt(pCxt, &s, &e, createOperatorNode(pCxt, OP_TYPE_SUB, releaseRawExprNode(pCxt, yymsp[-2].minor.yy674), releaseRawExprNode(pCxt, yymsp[0].minor.yy674))); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; - case 310: /* expression ::= expression NK_STAR expression */ + case 315: /* expression ::= expression NK_STAR expression */ { - SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy632); - SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy632); - yylhsminor.yy632 = createRawExprNodeExt(pCxt, &s, &e, createOperatorNode(pCxt, OP_TYPE_MULTI, releaseRawExprNode(pCxt, yymsp[-2].minor.yy632), releaseRawExprNode(pCxt, yymsp[0].minor.yy632))); + SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy674); + SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy674); + yylhsminor.yy674 = createRawExprNodeExt(pCxt, &s, &e, createOperatorNode(pCxt, OP_TYPE_MULTI, releaseRawExprNode(pCxt, yymsp[-2].minor.yy674), releaseRawExprNode(pCxt, yymsp[0].minor.yy674))); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; - case 311: /* expression ::= expression NK_SLASH expression */ + case 316: /* expression ::= expression NK_SLASH expression */ { - SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy632); - SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy632); - yylhsminor.yy632 = createRawExprNodeExt(pCxt, &s, &e, createOperatorNode(pCxt, OP_TYPE_DIV, releaseRawExprNode(pCxt, yymsp[-2].minor.yy632), releaseRawExprNode(pCxt, yymsp[0].minor.yy632))); + SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy674); + SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy674); + yylhsminor.yy674 = createRawExprNodeExt(pCxt, &s, &e, createOperatorNode(pCxt, OP_TYPE_DIV, releaseRawExprNode(pCxt, yymsp[-2].minor.yy674), releaseRawExprNode(pCxt, yymsp[0].minor.yy674))); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; - case 312: /* expression ::= expression NK_REM expression */ + case 317: /* expression ::= expression NK_REM expression */ { - SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy632); - SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy632); - yylhsminor.yy632 = createRawExprNodeExt(pCxt, &s, &e, createOperatorNode(pCxt, OP_TYPE_REM, releaseRawExprNode(pCxt, yymsp[-2].minor.yy632), releaseRawExprNode(pCxt, yymsp[0].minor.yy632))); + SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy674); + SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy674); + yylhsminor.yy674 = createRawExprNodeExt(pCxt, &s, &e, createOperatorNode(pCxt, OP_TYPE_REM, releaseRawExprNode(pCxt, yymsp[-2].minor.yy674), releaseRawExprNode(pCxt, yymsp[0].minor.yy674))); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; - case 313: /* expression ::= column_reference NK_ARROW NK_STRING */ + case 318: /* expression ::= column_reference NK_ARROW NK_STRING */ { - SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy632); - yylhsminor.yy632 = createRawExprNodeExt(pCxt, &s, &yymsp[0].minor.yy0, createOperatorNode(pCxt, OP_TYPE_JSON_GET_VALUE, releaseRawExprNode(pCxt, yymsp[-2].minor.yy632), createValueNode(pCxt, TSDB_DATA_TYPE_BINARY, &yymsp[0].minor.yy0))); + SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy674); + yylhsminor.yy674 = createRawExprNodeExt(pCxt, &s, &yymsp[0].minor.yy0, createOperatorNode(pCxt, OP_TYPE_JSON_GET_VALUE, releaseRawExprNode(pCxt, yymsp[-2].minor.yy674), createValueNode(pCxt, TSDB_DATA_TYPE_BINARY, &yymsp[0].minor.yy0))); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; - break; - case 314: /* expression_list ::= expression */ -{ yylhsminor.yy424 = createNodeList(pCxt, releaseRawExprNode(pCxt, yymsp[0].minor.yy632)); } - yymsp[0].minor.yy424 = yylhsminor.yy424; - break; - case 315: /* expression_list ::= expression_list NK_COMMA expression */ -{ yylhsminor.yy424 = addNodeToList(pCxt, yymsp[-2].minor.yy424, releaseRawExprNode(pCxt, yymsp[0].minor.yy632)); } - yymsp[-2].minor.yy424 = yylhsminor.yy424; - break; - case 316: /* column_reference ::= column_name */ -{ yylhsminor.yy632 = createRawExprNode(pCxt, &yymsp[0].minor.yy209, createColumnNode(pCxt, NULL, &yymsp[0].minor.yy209)); } - yymsp[0].minor.yy632 = yylhsminor.yy632; - break; - case 317: /* column_reference ::= table_name NK_DOT column_name */ -{ yylhsminor.yy632 = createRawExprNodeExt(pCxt, &yymsp[-2].minor.yy209, &yymsp[0].minor.yy209, createColumnNode(pCxt, &yymsp[-2].minor.yy209, &yymsp[0].minor.yy209)); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; - break; - case 318: /* pseudo_column ::= ROWTS */ - case 319: /* pseudo_column ::= TBNAME */ yytestcase(yyruleno==319); - case 321: /* pseudo_column ::= QSTARTTS */ yytestcase(yyruleno==321); - case 322: /* pseudo_column ::= QENDTS */ yytestcase(yyruleno==322); - case 323: /* pseudo_column ::= WSTARTTS */ yytestcase(yyruleno==323); - case 324: /* pseudo_column ::= WENDTS */ yytestcase(yyruleno==324); - case 325: /* pseudo_column ::= WDURATION */ yytestcase(yyruleno==325); - case 331: /* literal_func ::= NOW */ yytestcase(yyruleno==331); -{ yylhsminor.yy632 = createRawExprNode(pCxt, &yymsp[0].minor.yy0, createFunctionNode(pCxt, &yymsp[0].minor.yy0, NULL)); } - yymsp[0].minor.yy632 = yylhsminor.yy632; - break; - case 320: /* pseudo_column ::= table_name NK_DOT TBNAME */ -{ yylhsminor.yy632 = createRawExprNodeExt(pCxt, &yymsp[-2].minor.yy209, &yymsp[0].minor.yy0, createFunctionNode(pCxt, &yymsp[0].minor.yy0, createNodeList(pCxt, createValueNode(pCxt, TSDB_DATA_TYPE_BINARY, &yymsp[-2].minor.yy209)))); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; - break; - case 326: /* function_expression ::= function_name NK_LP expression_list NK_RP */ - case 327: /* function_expression ::= star_func NK_LP star_func_para_list NK_RP */ yytestcase(yyruleno==327); -{ yylhsminor.yy632 = createRawExprNodeExt(pCxt, &yymsp[-3].minor.yy209, &yymsp[0].minor.yy0, createFunctionNode(pCxt, &yymsp[-3].minor.yy209, yymsp[-1].minor.yy424)); } - yymsp[-3].minor.yy632 = yylhsminor.yy632; - break; - case 328: /* function_expression ::= CAST NK_LP expression AS type_name NK_RP */ -{ yylhsminor.yy632 = createRawExprNodeExt(pCxt, &yymsp[-5].minor.yy0, &yymsp[0].minor.yy0, createCastFunctionNode(pCxt, releaseRawExprNode(pCxt, yymsp[-3].minor.yy632), yymsp[-1].minor.yy304)); } - yymsp[-5].minor.yy632 = yylhsminor.yy632; - break; - case 330: /* literal_func ::= noarg_func NK_LP NK_RP */ -{ yylhsminor.yy632 = createRawExprNodeExt(pCxt, &yymsp[-2].minor.yy209, &yymsp[0].minor.yy0, createFunctionNode(pCxt, &yymsp[-2].minor.yy209, NULL)); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; - break; - case 339: /* star_func_para_list ::= NK_STAR */ -{ yylhsminor.yy424 = createNodeList(pCxt, createColumnNode(pCxt, NULL, &yymsp[0].minor.yy0)); } - yymsp[0].minor.yy424 = yylhsminor.yy424; - break; - case 344: /* star_func_para ::= table_name NK_DOT NK_STAR */ - case 401: /* select_item ::= table_name NK_DOT NK_STAR */ yytestcase(yyruleno==401); -{ yylhsminor.yy632 = createColumnNode(pCxt, &yymsp[-2].minor.yy209, &yymsp[0].minor.yy0); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; - break; - case 345: /* predicate ::= expression compare_op expression */ - case 350: /* predicate ::= expression in_op in_predicate_value */ yytestcase(yyruleno==350); + yymsp[-2].minor.yy674 = yylhsminor.yy674; + break; + case 321: /* column_reference ::= column_name */ +{ yylhsminor.yy674 = createRawExprNode(pCxt, &yymsp[0].minor.yy421, createColumnNode(pCxt, NULL, &yymsp[0].minor.yy421)); } + yymsp[0].minor.yy674 = yylhsminor.yy674; + break; + case 322: /* column_reference ::= table_name NK_DOT column_name */ +{ yylhsminor.yy674 = createRawExprNodeExt(pCxt, &yymsp[-2].minor.yy421, &yymsp[0].minor.yy421, createColumnNode(pCxt, &yymsp[-2].minor.yy421, &yymsp[0].minor.yy421)); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; + break; + case 323: /* pseudo_column ::= ROWTS */ + case 324: /* pseudo_column ::= TBNAME */ yytestcase(yyruleno==324); + case 326: /* pseudo_column ::= QSTARTTS */ yytestcase(yyruleno==326); + case 327: /* pseudo_column ::= QENDTS */ yytestcase(yyruleno==327); + case 328: /* pseudo_column ::= WSTARTTS */ yytestcase(yyruleno==328); + case 329: /* pseudo_column ::= WENDTS */ yytestcase(yyruleno==329); + case 330: /* pseudo_column ::= WDURATION */ yytestcase(yyruleno==330); + case 336: /* literal_func ::= NOW */ yytestcase(yyruleno==336); +{ yylhsminor.yy674 = createRawExprNode(pCxt, &yymsp[0].minor.yy0, createFunctionNode(pCxt, &yymsp[0].minor.yy0, NULL)); } + yymsp[0].minor.yy674 = yylhsminor.yy674; + break; + case 325: /* pseudo_column ::= table_name NK_DOT TBNAME */ +{ yylhsminor.yy674 = createRawExprNodeExt(pCxt, &yymsp[-2].minor.yy421, &yymsp[0].minor.yy0, createFunctionNode(pCxt, &yymsp[0].minor.yy0, createNodeList(pCxt, createValueNode(pCxt, TSDB_DATA_TYPE_BINARY, &yymsp[-2].minor.yy421)))); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; + break; + case 331: /* function_expression ::= function_name NK_LP expression_list NK_RP */ + case 332: /* function_expression ::= star_func NK_LP star_func_para_list NK_RP */ yytestcase(yyruleno==332); +{ yylhsminor.yy674 = createRawExprNodeExt(pCxt, &yymsp[-3].minor.yy421, &yymsp[0].minor.yy0, createFunctionNode(pCxt, &yymsp[-3].minor.yy421, yymsp[-1].minor.yy530)); } + yymsp[-3].minor.yy674 = yylhsminor.yy674; + break; + case 333: /* function_expression ::= CAST NK_LP expression AS type_name NK_RP */ +{ yylhsminor.yy674 = createRawExprNodeExt(pCxt, &yymsp[-5].minor.yy0, &yymsp[0].minor.yy0, createCastFunctionNode(pCxt, releaseRawExprNode(pCxt, yymsp[-3].minor.yy674), yymsp[-1].minor.yy690)); } + yymsp[-5].minor.yy674 = yylhsminor.yy674; + break; + case 335: /* literal_func ::= noarg_func NK_LP NK_RP */ +{ yylhsminor.yy674 = createRawExprNodeExt(pCxt, &yymsp[-2].minor.yy421, &yymsp[0].minor.yy0, createFunctionNode(pCxt, &yymsp[-2].minor.yy421, NULL)); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; + break; + case 344: /* star_func_para_list ::= NK_STAR */ +{ yylhsminor.yy530 = createNodeList(pCxt, createColumnNode(pCxt, NULL, &yymsp[0].minor.yy0)); } + yymsp[0].minor.yy530 = yylhsminor.yy530; + break; + case 349: /* star_func_para ::= table_name NK_DOT NK_STAR */ + case 406: /* select_item ::= table_name NK_DOT NK_STAR */ yytestcase(yyruleno==406); +{ yylhsminor.yy674 = createColumnNode(pCxt, &yymsp[-2].minor.yy421, &yymsp[0].minor.yy0); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; + break; + case 350: /* predicate ::= expression compare_op expression */ + case 355: /* predicate ::= expression in_op in_predicate_value */ yytestcase(yyruleno==355); { - SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy632); - SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy632); - yylhsminor.yy632 = createRawExprNodeExt(pCxt, &s, &e, createOperatorNode(pCxt, yymsp[-1].minor.yy380, releaseRawExprNode(pCxt, yymsp[-2].minor.yy632), releaseRawExprNode(pCxt, yymsp[0].minor.yy632))); + SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy674); + SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy674); + yylhsminor.yy674 = createRawExprNodeExt(pCxt, &s, &e, createOperatorNode(pCxt, yymsp[-1].minor.yy28, releaseRawExprNode(pCxt, yymsp[-2].minor.yy674), releaseRawExprNode(pCxt, yymsp[0].minor.yy674))); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; - case 346: /* predicate ::= expression BETWEEN expression AND expression */ + case 351: /* predicate ::= expression BETWEEN expression AND expression */ { - SToken s = getTokenFromRawExprNode(pCxt, yymsp[-4].minor.yy632); - SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy632); - yylhsminor.yy632 = createRawExprNodeExt(pCxt, &s, &e, createBetweenAnd(pCxt, releaseRawExprNode(pCxt, yymsp[-4].minor.yy632), releaseRawExprNode(pCxt, yymsp[-2].minor.yy632), releaseRawExprNode(pCxt, yymsp[0].minor.yy632))); + SToken s = getTokenFromRawExprNode(pCxt, yymsp[-4].minor.yy674); + SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy674); + yylhsminor.yy674 = createRawExprNodeExt(pCxt, &s, &e, createBetweenAnd(pCxt, releaseRawExprNode(pCxt, yymsp[-4].minor.yy674), releaseRawExprNode(pCxt, yymsp[-2].minor.yy674), releaseRawExprNode(pCxt, yymsp[0].minor.yy674))); } - yymsp[-4].minor.yy632 = yylhsminor.yy632; + yymsp[-4].minor.yy674 = yylhsminor.yy674; break; - case 347: /* predicate ::= expression NOT BETWEEN expression AND expression */ + case 352: /* predicate ::= expression NOT BETWEEN expression AND expression */ { - SToken s = getTokenFromRawExprNode(pCxt, yymsp[-5].minor.yy632); - SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy632); - yylhsminor.yy632 = createRawExprNodeExt(pCxt, &s, &e, createNotBetweenAnd(pCxt, releaseRawExprNode(pCxt, yymsp[-5].minor.yy632), releaseRawExprNode(pCxt, yymsp[-2].minor.yy632), releaseRawExprNode(pCxt, yymsp[0].minor.yy632))); + SToken s = getTokenFromRawExprNode(pCxt, yymsp[-5].minor.yy674); + SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy674); + yylhsminor.yy674 = createRawExprNodeExt(pCxt, &s, &e, createNotBetweenAnd(pCxt, releaseRawExprNode(pCxt, yymsp[-5].minor.yy674), releaseRawExprNode(pCxt, yymsp[-2].minor.yy674), releaseRawExprNode(pCxt, yymsp[0].minor.yy674))); } - yymsp[-5].minor.yy632 = yylhsminor.yy632; + yymsp[-5].minor.yy674 = yylhsminor.yy674; break; - case 348: /* predicate ::= expression IS NULL */ + case 353: /* predicate ::= expression IS NULL */ { - SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy632); - yylhsminor.yy632 = createRawExprNodeExt(pCxt, &s, &yymsp[0].minor.yy0, createOperatorNode(pCxt, OP_TYPE_IS_NULL, releaseRawExprNode(pCxt, yymsp[-2].minor.yy632), NULL)); + SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy674); + yylhsminor.yy674 = createRawExprNodeExt(pCxt, &s, &yymsp[0].minor.yy0, createOperatorNode(pCxt, OP_TYPE_IS_NULL, releaseRawExprNode(pCxt, yymsp[-2].minor.yy674), NULL)); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; - case 349: /* predicate ::= expression IS NOT NULL */ + case 354: /* predicate ::= expression IS NOT NULL */ { - SToken s = getTokenFromRawExprNode(pCxt, yymsp[-3].minor.yy632); - yylhsminor.yy632 = createRawExprNodeExt(pCxt, &s, &yymsp[0].minor.yy0, createOperatorNode(pCxt, OP_TYPE_IS_NOT_NULL, releaseRawExprNode(pCxt, yymsp[-3].minor.yy632), NULL)); + SToken s = getTokenFromRawExprNode(pCxt, yymsp[-3].minor.yy674); + yylhsminor.yy674 = createRawExprNodeExt(pCxt, &s, &yymsp[0].minor.yy0, createOperatorNode(pCxt, OP_TYPE_IS_NOT_NULL, releaseRawExprNode(pCxt, yymsp[-3].minor.yy674), NULL)); } - yymsp[-3].minor.yy632 = yylhsminor.yy632; + yymsp[-3].minor.yy674 = yylhsminor.yy674; break; - case 351: /* compare_op ::= NK_LT */ -{ yymsp[0].minor.yy380 = OP_TYPE_LOWER_THAN; } + case 356: /* compare_op ::= NK_LT */ +{ yymsp[0].minor.yy28 = OP_TYPE_LOWER_THAN; } break; - case 352: /* compare_op ::= NK_GT */ -{ yymsp[0].minor.yy380 = OP_TYPE_GREATER_THAN; } + case 357: /* compare_op ::= NK_GT */ +{ yymsp[0].minor.yy28 = OP_TYPE_GREATER_THAN; } break; - case 353: /* compare_op ::= NK_LE */ -{ yymsp[0].minor.yy380 = OP_TYPE_LOWER_EQUAL; } + case 358: /* compare_op ::= NK_LE */ +{ yymsp[0].minor.yy28 = OP_TYPE_LOWER_EQUAL; } break; - case 354: /* compare_op ::= NK_GE */ -{ yymsp[0].minor.yy380 = OP_TYPE_GREATER_EQUAL; } + case 359: /* compare_op ::= NK_GE */ +{ yymsp[0].minor.yy28 = OP_TYPE_GREATER_EQUAL; } break; - case 355: /* compare_op ::= NK_NE */ -{ yymsp[0].minor.yy380 = OP_TYPE_NOT_EQUAL; } + case 360: /* compare_op ::= NK_NE */ +{ yymsp[0].minor.yy28 = OP_TYPE_NOT_EQUAL; } break; - case 356: /* compare_op ::= NK_EQ */ -{ yymsp[0].minor.yy380 = OP_TYPE_EQUAL; } + case 361: /* compare_op ::= NK_EQ */ +{ yymsp[0].minor.yy28 = OP_TYPE_EQUAL; } break; - case 357: /* compare_op ::= LIKE */ -{ yymsp[0].minor.yy380 = OP_TYPE_LIKE; } + case 362: /* compare_op ::= LIKE */ +{ yymsp[0].minor.yy28 = OP_TYPE_LIKE; } break; - case 358: /* compare_op ::= NOT LIKE */ -{ yymsp[-1].minor.yy380 = OP_TYPE_NOT_LIKE; } + case 363: /* compare_op ::= NOT LIKE */ +{ yymsp[-1].minor.yy28 = OP_TYPE_NOT_LIKE; } break; - case 359: /* compare_op ::= MATCH */ -{ yymsp[0].minor.yy380 = OP_TYPE_MATCH; } + case 364: /* compare_op ::= MATCH */ +{ yymsp[0].minor.yy28 = OP_TYPE_MATCH; } break; - case 360: /* compare_op ::= NMATCH */ -{ yymsp[0].minor.yy380 = OP_TYPE_NMATCH; } + case 365: /* compare_op ::= NMATCH */ +{ yymsp[0].minor.yy28 = OP_TYPE_NMATCH; } break; - case 361: /* compare_op ::= CONTAINS */ -{ yymsp[0].minor.yy380 = OP_TYPE_JSON_CONTAINS; } + case 366: /* compare_op ::= CONTAINS */ +{ yymsp[0].minor.yy28 = OP_TYPE_JSON_CONTAINS; } break; - case 362: /* in_op ::= IN */ -{ yymsp[0].minor.yy380 = OP_TYPE_IN; } + case 367: /* in_op ::= IN */ +{ yymsp[0].minor.yy28 = OP_TYPE_IN; } break; - case 363: /* in_op ::= NOT IN */ -{ yymsp[-1].minor.yy380 = OP_TYPE_NOT_IN; } + case 368: /* in_op ::= NOT IN */ +{ yymsp[-1].minor.yy28 = OP_TYPE_NOT_IN; } break; - case 364: /* in_predicate_value ::= NK_LP expression_list NK_RP */ -{ yylhsminor.yy632 = createRawExprNodeExt(pCxt, &yymsp[-2].minor.yy0, &yymsp[0].minor.yy0, createNodeListNode(pCxt, yymsp[-1].minor.yy424)); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; + case 369: /* in_predicate_value ::= NK_LP expression_list NK_RP */ +{ yylhsminor.yy674 = createRawExprNodeExt(pCxt, &yymsp[-2].minor.yy0, &yymsp[0].minor.yy0, createNodeListNode(pCxt, yymsp[-1].minor.yy530)); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; - case 366: /* boolean_value_expression ::= NOT boolean_primary */ + case 371: /* boolean_value_expression ::= NOT boolean_primary */ { - SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy632); - yylhsminor.yy632 = createRawExprNodeExt(pCxt, &yymsp[-1].minor.yy0, &e, createLogicConditionNode(pCxt, LOGIC_COND_TYPE_NOT, releaseRawExprNode(pCxt, yymsp[0].minor.yy632), NULL)); + SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy674); + yylhsminor.yy674 = createRawExprNodeExt(pCxt, &yymsp[-1].minor.yy0, &e, createLogicConditionNode(pCxt, LOGIC_COND_TYPE_NOT, releaseRawExprNode(pCxt, yymsp[0].minor.yy674), NULL)); } - yymsp[-1].minor.yy632 = yylhsminor.yy632; + yymsp[-1].minor.yy674 = yylhsminor.yy674; break; - case 367: /* boolean_value_expression ::= boolean_value_expression OR boolean_value_expression */ + case 372: /* boolean_value_expression ::= boolean_value_expression OR boolean_value_expression */ { - SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy632); - SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy632); - yylhsminor.yy632 = createRawExprNodeExt(pCxt, &s, &e, createLogicConditionNode(pCxt, LOGIC_COND_TYPE_OR, releaseRawExprNode(pCxt, yymsp[-2].minor.yy632), releaseRawExprNode(pCxt, yymsp[0].minor.yy632))); + SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy674); + SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy674); + yylhsminor.yy674 = createRawExprNodeExt(pCxt, &s, &e, createLogicConditionNode(pCxt, LOGIC_COND_TYPE_OR, releaseRawExprNode(pCxt, yymsp[-2].minor.yy674), releaseRawExprNode(pCxt, yymsp[0].minor.yy674))); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; - case 368: /* boolean_value_expression ::= boolean_value_expression AND boolean_value_expression */ + case 373: /* boolean_value_expression ::= boolean_value_expression AND boolean_value_expression */ { - SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy632); - SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy632); - yylhsminor.yy632 = createRawExprNodeExt(pCxt, &s, &e, createLogicConditionNode(pCxt, LOGIC_COND_TYPE_AND, releaseRawExprNode(pCxt, yymsp[-2].minor.yy632), releaseRawExprNode(pCxt, yymsp[0].minor.yy632))); + SToken s = getTokenFromRawExprNode(pCxt, yymsp[-2].minor.yy674); + SToken e = getTokenFromRawExprNode(pCxt, yymsp[0].minor.yy674); + yylhsminor.yy674 = createRawExprNodeExt(pCxt, &s, &e, createLogicConditionNode(pCxt, LOGIC_COND_TYPE_AND, releaseRawExprNode(pCxt, yymsp[-2].minor.yy674), releaseRawExprNode(pCxt, yymsp[0].minor.yy674))); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; - case 375: /* table_reference_list ::= table_reference_list NK_COMMA table_reference */ -{ yylhsminor.yy632 = createJoinTableNode(pCxt, JOIN_TYPE_INNER, yymsp[-2].minor.yy632, yymsp[0].minor.yy632, NULL); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; + case 380: /* table_reference_list ::= table_reference_list NK_COMMA table_reference */ +{ yylhsminor.yy674 = createJoinTableNode(pCxt, JOIN_TYPE_INNER, yymsp[-2].minor.yy674, yymsp[0].minor.yy674, NULL); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; - case 378: /* table_primary ::= table_name alias_opt */ -{ yylhsminor.yy632 = createRealTableNode(pCxt, NULL, &yymsp[-1].minor.yy209, &yymsp[0].minor.yy209); } - yymsp[-1].minor.yy632 = yylhsminor.yy632; + case 383: /* table_primary ::= table_name alias_opt */ +{ yylhsminor.yy674 = createRealTableNode(pCxt, NULL, &yymsp[-1].minor.yy421, &yymsp[0].minor.yy421); } + yymsp[-1].minor.yy674 = yylhsminor.yy674; break; - case 379: /* table_primary ::= db_name NK_DOT table_name alias_opt */ -{ yylhsminor.yy632 = createRealTableNode(pCxt, &yymsp[-3].minor.yy209, &yymsp[-1].minor.yy209, &yymsp[0].minor.yy209); } - yymsp[-3].minor.yy632 = yylhsminor.yy632; + case 384: /* table_primary ::= db_name NK_DOT table_name alias_opt */ +{ yylhsminor.yy674 = createRealTableNode(pCxt, &yymsp[-3].minor.yy421, &yymsp[-1].minor.yy421, &yymsp[0].minor.yy421); } + yymsp[-3].minor.yy674 = yylhsminor.yy674; break; - case 380: /* table_primary ::= subquery alias_opt */ -{ yylhsminor.yy632 = createTempTableNode(pCxt, releaseRawExprNode(pCxt, yymsp[-1].minor.yy632), &yymsp[0].minor.yy209); } - yymsp[-1].minor.yy632 = yylhsminor.yy632; + case 385: /* table_primary ::= subquery alias_opt */ +{ yylhsminor.yy674 = createTempTableNode(pCxt, releaseRawExprNode(pCxt, yymsp[-1].minor.yy674), &yymsp[0].minor.yy421); } + yymsp[-1].minor.yy674 = yylhsminor.yy674; break; - case 382: /* alias_opt ::= */ -{ yymsp[1].minor.yy209 = nil_token; } + case 387: /* alias_opt ::= */ +{ yymsp[1].minor.yy421 = nil_token; } break; - case 383: /* alias_opt ::= table_alias */ -{ yylhsminor.yy209 = yymsp[0].minor.yy209; } - yymsp[0].minor.yy209 = yylhsminor.yy209; + case 388: /* alias_opt ::= table_alias */ +{ yylhsminor.yy421 = yymsp[0].minor.yy421; } + yymsp[0].minor.yy421 = yylhsminor.yy421; break; - case 384: /* alias_opt ::= AS table_alias */ -{ yymsp[-1].minor.yy209 = yymsp[0].minor.yy209; } + case 389: /* alias_opt ::= AS table_alias */ +{ yymsp[-1].minor.yy421 = yymsp[0].minor.yy421; } break; - case 385: /* parenthesized_joined_table ::= NK_LP joined_table NK_RP */ - case 386: /* parenthesized_joined_table ::= NK_LP parenthesized_joined_table NK_RP */ yytestcase(yyruleno==386); -{ yymsp[-2].minor.yy632 = yymsp[-1].minor.yy632; } + case 390: /* parenthesized_joined_table ::= NK_LP joined_table NK_RP */ + case 391: /* parenthesized_joined_table ::= NK_LP parenthesized_joined_table NK_RP */ yytestcase(yyruleno==391); +{ yymsp[-2].minor.yy674 = yymsp[-1].minor.yy674; } break; - case 387: /* joined_table ::= table_reference join_type JOIN table_reference ON search_condition */ -{ yylhsminor.yy632 = createJoinTableNode(pCxt, yymsp[-4].minor.yy612, yymsp[-5].minor.yy632, yymsp[-2].minor.yy632, yymsp[0].minor.yy632); } - yymsp[-5].minor.yy632 = yylhsminor.yy632; + case 392: /* joined_table ::= table_reference join_type JOIN table_reference ON search_condition */ +{ yylhsminor.yy674 = createJoinTableNode(pCxt, yymsp[-4].minor.yy636, yymsp[-5].minor.yy674, yymsp[-2].minor.yy674, yymsp[0].minor.yy674); } + yymsp[-5].minor.yy674 = yylhsminor.yy674; break; - case 388: /* join_type ::= */ -{ yymsp[1].minor.yy612 = JOIN_TYPE_INNER; } + case 393: /* join_type ::= */ +{ yymsp[1].minor.yy636 = JOIN_TYPE_INNER; } break; - case 389: /* join_type ::= INNER */ -{ yymsp[0].minor.yy612 = JOIN_TYPE_INNER; } + case 394: /* join_type ::= INNER */ +{ yymsp[0].minor.yy636 = JOIN_TYPE_INNER; } break; - case 390: /* query_specification ::= SELECT set_quantifier_opt select_list from_clause where_clause_opt partition_by_clause_opt twindow_clause_opt group_by_clause_opt having_clause_opt */ + case 395: /* query_specification ::= SELECT set_quantifier_opt select_list from_clause where_clause_opt partition_by_clause_opt twindow_clause_opt group_by_clause_opt having_clause_opt */ { - yymsp[-8].minor.yy632 = createSelectStmt(pCxt, yymsp[-7].minor.yy137, yymsp[-6].minor.yy424, yymsp[-5].minor.yy632); - yymsp[-8].minor.yy632 = addWhereClause(pCxt, yymsp[-8].minor.yy632, yymsp[-4].minor.yy632); - yymsp[-8].minor.yy632 = addPartitionByClause(pCxt, yymsp[-8].minor.yy632, yymsp[-3].minor.yy424); - yymsp[-8].minor.yy632 = addWindowClauseClause(pCxt, yymsp[-8].minor.yy632, yymsp[-2].minor.yy632); - yymsp[-8].minor.yy632 = addGroupByClause(pCxt, yymsp[-8].minor.yy632, yymsp[-1].minor.yy424); - yymsp[-8].minor.yy632 = addHavingClause(pCxt, yymsp[-8].minor.yy632, yymsp[0].minor.yy632); + yymsp[-8].minor.yy674 = createSelectStmt(pCxt, yymsp[-7].minor.yy621, yymsp[-6].minor.yy530, yymsp[-5].minor.yy674); + yymsp[-8].minor.yy674 = addWhereClause(pCxt, yymsp[-8].minor.yy674, yymsp[-4].minor.yy674); + yymsp[-8].minor.yy674 = addPartitionByClause(pCxt, yymsp[-8].minor.yy674, yymsp[-3].minor.yy530); + yymsp[-8].minor.yy674 = addWindowClauseClause(pCxt, yymsp[-8].minor.yy674, yymsp[-2].minor.yy674); + yymsp[-8].minor.yy674 = addGroupByClause(pCxt, yymsp[-8].minor.yy674, yymsp[-1].minor.yy530); + yymsp[-8].minor.yy674 = addHavingClause(pCxt, yymsp[-8].minor.yy674, yymsp[0].minor.yy674); } break; - case 393: /* set_quantifier_opt ::= ALL */ -{ yymsp[0].minor.yy137 = false; } + case 398: /* set_quantifier_opt ::= ALL */ +{ yymsp[0].minor.yy621 = false; } break; - case 394: /* select_list ::= NK_STAR */ -{ yymsp[0].minor.yy424 = NULL; } + case 399: /* select_list ::= NK_STAR */ +{ yymsp[0].minor.yy530 = NULL; } break; - case 399: /* select_item ::= common_expression column_alias */ -{ yylhsminor.yy632 = setProjectionAlias(pCxt, releaseRawExprNode(pCxt, yymsp[-1].minor.yy632), &yymsp[0].minor.yy209); } - yymsp[-1].minor.yy632 = yylhsminor.yy632; + case 404: /* select_item ::= common_expression column_alias */ +{ yylhsminor.yy674 = setProjectionAlias(pCxt, releaseRawExprNode(pCxt, yymsp[-1].minor.yy674), &yymsp[0].minor.yy421); } + yymsp[-1].minor.yy674 = yylhsminor.yy674; break; - case 400: /* select_item ::= common_expression AS column_alias */ -{ yylhsminor.yy632 = setProjectionAlias(pCxt, releaseRawExprNode(pCxt, yymsp[-2].minor.yy632), &yymsp[0].minor.yy209); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; + case 405: /* select_item ::= common_expression AS column_alias */ +{ yylhsminor.yy674 = setProjectionAlias(pCxt, releaseRawExprNode(pCxt, yymsp[-2].minor.yy674), &yymsp[0].minor.yy421); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; - case 405: /* partition_by_clause_opt ::= PARTITION BY expression_list */ - case 422: /* group_by_clause_opt ::= GROUP BY group_by_list */ yytestcase(yyruleno==422); - case 434: /* order_by_clause_opt ::= ORDER BY sort_specification_list */ yytestcase(yyruleno==434); -{ yymsp[-2].minor.yy424 = yymsp[0].minor.yy424; } + case 410: /* partition_by_clause_opt ::= PARTITION BY expression_list */ + case 427: /* group_by_clause_opt ::= GROUP BY group_by_list */ yytestcase(yyruleno==427); + case 439: /* order_by_clause_opt ::= ORDER BY sort_specification_list */ yytestcase(yyruleno==439); +{ yymsp[-2].minor.yy530 = yymsp[0].minor.yy530; } break; - case 407: /* twindow_clause_opt ::= SESSION NK_LP column_reference NK_COMMA duration_literal NK_RP */ -{ yymsp[-5].minor.yy632 = createSessionWindowNode(pCxt, releaseRawExprNode(pCxt, yymsp[-3].minor.yy632), releaseRawExprNode(pCxt, yymsp[-1].minor.yy632)); } + case 412: /* twindow_clause_opt ::= SESSION NK_LP column_reference NK_COMMA duration_literal NK_RP */ +{ yymsp[-5].minor.yy674 = createSessionWindowNode(pCxt, releaseRawExprNode(pCxt, yymsp[-3].minor.yy674), releaseRawExprNode(pCxt, yymsp[-1].minor.yy674)); } break; - case 408: /* twindow_clause_opt ::= STATE_WINDOW NK_LP expression NK_RP */ -{ yymsp[-3].minor.yy632 = createStateWindowNode(pCxt, releaseRawExprNode(pCxt, yymsp[-1].minor.yy632)); } + case 413: /* twindow_clause_opt ::= STATE_WINDOW NK_LP expression NK_RP */ +{ yymsp[-3].minor.yy674 = createStateWindowNode(pCxt, releaseRawExprNode(pCxt, yymsp[-1].minor.yy674)); } break; - case 409: /* twindow_clause_opt ::= INTERVAL NK_LP duration_literal NK_RP sliding_opt fill_opt */ -{ yymsp[-5].minor.yy632 = createIntervalWindowNode(pCxt, releaseRawExprNode(pCxt, yymsp[-3].minor.yy632), NULL, yymsp[-1].minor.yy632, yymsp[0].minor.yy632); } + case 414: /* twindow_clause_opt ::= INTERVAL NK_LP duration_literal NK_RP sliding_opt fill_opt */ +{ yymsp[-5].minor.yy674 = createIntervalWindowNode(pCxt, releaseRawExprNode(pCxt, yymsp[-3].minor.yy674), NULL, yymsp[-1].minor.yy674, yymsp[0].minor.yy674); } break; - case 410: /* twindow_clause_opt ::= INTERVAL NK_LP duration_literal NK_COMMA duration_literal NK_RP sliding_opt fill_opt */ -{ yymsp[-7].minor.yy632 = createIntervalWindowNode(pCxt, releaseRawExprNode(pCxt, yymsp[-5].minor.yy632), releaseRawExprNode(pCxt, yymsp[-3].minor.yy632), yymsp[-1].minor.yy632, yymsp[0].minor.yy632); } + case 415: /* twindow_clause_opt ::= INTERVAL NK_LP duration_literal NK_COMMA duration_literal NK_RP sliding_opt fill_opt */ +{ yymsp[-7].minor.yy674 = createIntervalWindowNode(pCxt, releaseRawExprNode(pCxt, yymsp[-5].minor.yy674), releaseRawExprNode(pCxt, yymsp[-3].minor.yy674), yymsp[-1].minor.yy674, yymsp[0].minor.yy674); } break; - case 412: /* sliding_opt ::= SLIDING NK_LP duration_literal NK_RP */ -{ yymsp[-3].minor.yy632 = releaseRawExprNode(pCxt, yymsp[-1].minor.yy632); } + case 417: /* sliding_opt ::= SLIDING NK_LP duration_literal NK_RP */ +{ yymsp[-3].minor.yy674 = releaseRawExprNode(pCxt, yymsp[-1].minor.yy674); } break; - case 414: /* fill_opt ::= FILL NK_LP fill_mode NK_RP */ -{ yymsp[-3].minor.yy632 = createFillNode(pCxt, yymsp[-1].minor.yy54, NULL); } + case 419: /* fill_opt ::= FILL NK_LP fill_mode NK_RP */ +{ yymsp[-3].minor.yy674 = createFillNode(pCxt, yymsp[-1].minor.yy320, NULL); } break; - case 415: /* fill_opt ::= FILL NK_LP VALUE NK_COMMA literal_list NK_RP */ -{ yymsp[-5].minor.yy632 = createFillNode(pCxt, FILL_MODE_VALUE, createNodeListNode(pCxt, yymsp[-1].minor.yy424)); } + case 420: /* fill_opt ::= FILL NK_LP VALUE NK_COMMA literal_list NK_RP */ +{ yymsp[-5].minor.yy674 = createFillNode(pCxt, FILL_MODE_VALUE, createNodeListNode(pCxt, yymsp[-1].minor.yy530)); } break; - case 416: /* fill_mode ::= NONE */ -{ yymsp[0].minor.yy54 = FILL_MODE_NONE; } + case 421: /* fill_mode ::= NONE */ +{ yymsp[0].minor.yy320 = FILL_MODE_NONE; } break; - case 417: /* fill_mode ::= PREV */ -{ yymsp[0].minor.yy54 = FILL_MODE_PREV; } + case 422: /* fill_mode ::= PREV */ +{ yymsp[0].minor.yy320 = FILL_MODE_PREV; } break; - case 418: /* fill_mode ::= NULL */ -{ yymsp[0].minor.yy54 = FILL_MODE_NULL; } + case 423: /* fill_mode ::= NULL */ +{ yymsp[0].minor.yy320 = FILL_MODE_NULL; } break; - case 419: /* fill_mode ::= LINEAR */ -{ yymsp[0].minor.yy54 = FILL_MODE_LINEAR; } + case 424: /* fill_mode ::= LINEAR */ +{ yymsp[0].minor.yy320 = FILL_MODE_LINEAR; } break; - case 420: /* fill_mode ::= NEXT */ -{ yymsp[0].minor.yy54 = FILL_MODE_NEXT; } + case 425: /* fill_mode ::= NEXT */ +{ yymsp[0].minor.yy320 = FILL_MODE_NEXT; } break; - case 423: /* group_by_list ::= expression */ -{ yylhsminor.yy424 = createNodeList(pCxt, createGroupingSetNode(pCxt, releaseRawExprNode(pCxt, yymsp[0].minor.yy632))); } - yymsp[0].minor.yy424 = yylhsminor.yy424; + case 428: /* group_by_list ::= expression */ +{ yylhsminor.yy530 = createNodeList(pCxt, createGroupingSetNode(pCxt, releaseRawExprNode(pCxt, yymsp[0].minor.yy674))); } + yymsp[0].minor.yy530 = yylhsminor.yy530; break; - case 424: /* group_by_list ::= group_by_list NK_COMMA expression */ -{ yylhsminor.yy424 = addNodeToList(pCxt, yymsp[-2].minor.yy424, createGroupingSetNode(pCxt, releaseRawExprNode(pCxt, yymsp[0].minor.yy632))); } - yymsp[-2].minor.yy424 = yylhsminor.yy424; + case 429: /* group_by_list ::= group_by_list NK_COMMA expression */ +{ yylhsminor.yy530 = addNodeToList(pCxt, yymsp[-2].minor.yy530, createGroupingSetNode(pCxt, releaseRawExprNode(pCxt, yymsp[0].minor.yy674))); } + yymsp[-2].minor.yy530 = yylhsminor.yy530; break; - case 427: /* query_expression ::= query_expression_body order_by_clause_opt slimit_clause_opt limit_clause_opt */ + case 432: /* query_expression ::= query_expression_body order_by_clause_opt slimit_clause_opt limit_clause_opt */ { - yylhsminor.yy632 = addOrderByClause(pCxt, yymsp[-3].minor.yy632, yymsp[-2].minor.yy424); - yylhsminor.yy632 = addSlimitClause(pCxt, yylhsminor.yy632, yymsp[-1].minor.yy632); - yylhsminor.yy632 = addLimitClause(pCxt, yylhsminor.yy632, yymsp[0].minor.yy632); + yylhsminor.yy674 = addOrderByClause(pCxt, yymsp[-3].minor.yy674, yymsp[-2].minor.yy530); + yylhsminor.yy674 = addSlimitClause(pCxt, yylhsminor.yy674, yymsp[-1].minor.yy674); + yylhsminor.yy674 = addLimitClause(pCxt, yylhsminor.yy674, yymsp[0].minor.yy674); } - yymsp[-3].minor.yy632 = yylhsminor.yy632; + yymsp[-3].minor.yy674 = yylhsminor.yy674; break; - case 429: /* query_expression_body ::= query_expression_body UNION ALL query_expression_body */ -{ yylhsminor.yy632 = createSetOperator(pCxt, SET_OP_TYPE_UNION_ALL, yymsp[-3].minor.yy632, yymsp[0].minor.yy632); } - yymsp[-3].minor.yy632 = yylhsminor.yy632; + case 434: /* query_expression_body ::= query_expression_body UNION ALL query_expression_body */ +{ yylhsminor.yy674 = createSetOperator(pCxt, SET_OP_TYPE_UNION_ALL, yymsp[-3].minor.yy674, yymsp[0].minor.yy674); } + yymsp[-3].minor.yy674 = yylhsminor.yy674; break; - case 430: /* query_expression_body ::= query_expression_body UNION query_expression_body */ -{ yylhsminor.yy632 = createSetOperator(pCxt, SET_OP_TYPE_UNION, yymsp[-2].minor.yy632, yymsp[0].minor.yy632); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; + case 435: /* query_expression_body ::= query_expression_body UNION query_expression_body */ +{ yylhsminor.yy674 = createSetOperator(pCxt, SET_OP_TYPE_UNION, yymsp[-2].minor.yy674, yymsp[0].minor.yy674); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; - case 432: /* query_primary ::= NK_LP query_expression_body order_by_clause_opt slimit_clause_opt limit_clause_opt NK_RP */ -{ yymsp[-5].minor.yy632 = yymsp[-4].minor.yy632; } - yy_destructor(yypParser,352,&yymsp[-3].minor); - yy_destructor(yypParser,353,&yymsp[-2].minor); - yy_destructor(yypParser,354,&yymsp[-1].minor); + case 437: /* query_primary ::= NK_LP query_expression_body order_by_clause_opt slimit_clause_opt limit_clause_opt NK_RP */ +{ yymsp[-5].minor.yy674 = yymsp[-4].minor.yy674; } + yy_destructor(yypParser,351,&yymsp[-3].minor); + yy_destructor(yypParser,352,&yymsp[-2].minor); + yy_destructor(yypParser,353,&yymsp[-1].minor); break; - case 436: /* slimit_clause_opt ::= SLIMIT NK_INTEGER */ - case 440: /* limit_clause_opt ::= LIMIT NK_INTEGER */ yytestcase(yyruleno==440); -{ yymsp[-1].minor.yy632 = createLimitNode(pCxt, &yymsp[0].minor.yy0, NULL); } + case 441: /* slimit_clause_opt ::= SLIMIT NK_INTEGER */ + case 445: /* limit_clause_opt ::= LIMIT NK_INTEGER */ yytestcase(yyruleno==445); +{ yymsp[-1].minor.yy674 = createLimitNode(pCxt, &yymsp[0].minor.yy0, NULL); } break; - case 437: /* slimit_clause_opt ::= SLIMIT NK_INTEGER SOFFSET NK_INTEGER */ - case 441: /* limit_clause_opt ::= LIMIT NK_INTEGER OFFSET NK_INTEGER */ yytestcase(yyruleno==441); -{ yymsp[-3].minor.yy632 = createLimitNode(pCxt, &yymsp[-2].minor.yy0, &yymsp[0].minor.yy0); } + case 442: /* slimit_clause_opt ::= SLIMIT NK_INTEGER SOFFSET NK_INTEGER */ + case 446: /* limit_clause_opt ::= LIMIT NK_INTEGER OFFSET NK_INTEGER */ yytestcase(yyruleno==446); +{ yymsp[-3].minor.yy674 = createLimitNode(pCxt, &yymsp[-2].minor.yy0, &yymsp[0].minor.yy0); } break; - case 438: /* slimit_clause_opt ::= SLIMIT NK_INTEGER NK_COMMA NK_INTEGER */ - case 442: /* limit_clause_opt ::= LIMIT NK_INTEGER NK_COMMA NK_INTEGER */ yytestcase(yyruleno==442); -{ yymsp[-3].minor.yy632 = createLimitNode(pCxt, &yymsp[0].minor.yy0, &yymsp[-2].minor.yy0); } + case 443: /* slimit_clause_opt ::= SLIMIT NK_INTEGER NK_COMMA NK_INTEGER */ + case 447: /* limit_clause_opt ::= LIMIT NK_INTEGER NK_COMMA NK_INTEGER */ yytestcase(yyruleno==447); +{ yymsp[-3].minor.yy674 = createLimitNode(pCxt, &yymsp[0].minor.yy0, &yymsp[-2].minor.yy0); } break; - case 443: /* subquery ::= NK_LP query_expression NK_RP */ -{ yylhsminor.yy632 = createRawExprNodeExt(pCxt, &yymsp[-2].minor.yy0, &yymsp[0].minor.yy0, yymsp[-1].minor.yy632); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; + case 448: /* subquery ::= NK_LP query_expression NK_RP */ +{ yylhsminor.yy674 = createRawExprNodeExt(pCxt, &yymsp[-2].minor.yy0, &yymsp[0].minor.yy0, yymsp[-1].minor.yy674); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; - case 447: /* sort_specification ::= expression ordering_specification_opt null_ordering_opt */ -{ yylhsminor.yy632 = createOrderByExprNode(pCxt, releaseRawExprNode(pCxt, yymsp[-2].minor.yy632), yymsp[-1].minor.yy578, yymsp[0].minor.yy217); } - yymsp[-2].minor.yy632 = yylhsminor.yy632; + case 452: /* sort_specification ::= expression ordering_specification_opt null_ordering_opt */ +{ yylhsminor.yy674 = createOrderByExprNode(pCxt, releaseRawExprNode(pCxt, yymsp[-2].minor.yy674), yymsp[-1].minor.yy610, yymsp[0].minor.yy107); } + yymsp[-2].minor.yy674 = yylhsminor.yy674; break; - case 448: /* ordering_specification_opt ::= */ -{ yymsp[1].minor.yy578 = ORDER_ASC; } + case 453: /* ordering_specification_opt ::= */ +{ yymsp[1].minor.yy610 = ORDER_ASC; } break; - case 449: /* ordering_specification_opt ::= ASC */ -{ yymsp[0].minor.yy578 = ORDER_ASC; } + case 454: /* ordering_specification_opt ::= ASC */ +{ yymsp[0].minor.yy610 = ORDER_ASC; } break; - case 450: /* ordering_specification_opt ::= DESC */ -{ yymsp[0].minor.yy578 = ORDER_DESC; } + case 455: /* ordering_specification_opt ::= DESC */ +{ yymsp[0].minor.yy610 = ORDER_DESC; } break; - case 451: /* null_ordering_opt ::= */ -{ yymsp[1].minor.yy217 = NULL_ORDER_DEFAULT; } + case 456: /* null_ordering_opt ::= */ +{ yymsp[1].minor.yy107 = NULL_ORDER_DEFAULT; } break; - case 452: /* null_ordering_opt ::= NULLS FIRST */ -{ yymsp[-1].minor.yy217 = NULL_ORDER_FIRST; } + case 457: /* null_ordering_opt ::= NULLS FIRST */ +{ yymsp[-1].minor.yy107 = NULL_ORDER_FIRST; } break; - case 453: /* null_ordering_opt ::= NULLS LAST */ -{ yymsp[-1].minor.yy217 = NULL_ORDER_LAST; } + case 458: /* null_ordering_opt ::= NULLS LAST */ +{ yymsp[-1].minor.yy107 = NULL_ORDER_LAST; } break; default: break; diff --git a/source/libs/parser/test/mockCatalogService.cpp b/source/libs/parser/test/mockCatalogService.cpp index 68f9e9d36d517a0cbfd34417c8276aadd680f49a..9758da78991adccfffedaa06ba4b0f0d7f58ac2f 100644 --- a/source/libs/parser/test/mockCatalogService.cpp +++ b/source/libs/parser/test/mockCatalogService.cpp @@ -285,7 +285,7 @@ class MockCatalogServiceImpl { } void createSmaIndex(const SMCreateSmaReq* pReq) { - STableIndexInfo info; + STableIndexInfo info = {0}; info.intervalUnit = pReq->intervalUnit; info.slidingUnit = pReq->slidingUnit; info.interval = pReq->interval; diff --git a/source/libs/parser/test/parInitialATest.cpp b/source/libs/parser/test/parInitialATest.cpp index cc23fbbc60ce2ff8b457af05f78721bc9f1c7563..3247ff352e3335795d1727394094d9971ee64cee 100644 --- a/source/libs/parser/test/parInitialATest.cpp +++ b/source/libs/parser/test/parInitialATest.cpp @@ -43,7 +43,40 @@ TEST_F(ParserInitialATest, alterDatabase) { run("ALTER DATABASE wxy_db KEEP 2400"); } -// todo ALTER local +TEST_F(ParserInitialATest, alterLocal) { + useDb("root", "test"); + + pair expect; + + auto clearAlterLocal = [&]() { + expect.first.clear(); + expect.second.clear(); + }; + + auto setAlterLocalFunc = [&](const char* pConfig, const char* pValue = nullptr) { + expect.first.assign(pConfig); + if (nullptr != pValue) { + expect.second.assign(pValue); + } + }; + + setCheckDdlFunc([&](const SQuery* pQuery, ParserStage stage) { + ASSERT_EQ(nodeType(pQuery->pRoot), QUERY_NODE_ALTER_LOCAL_STMT); + ASSERT_EQ(pQuery->execMode, QUERY_EXEC_MODE_LOCAL); + SAlterLocalStmt* pStmt = (SAlterLocalStmt*)pQuery->pRoot; + ASSERT_EQ(string(pStmt->config), expect.first); + ASSERT_EQ(string(pStmt->value), expect.second); + }); + + setAlterLocalFunc("resetlog"); + run("ALTER LOCAL 'resetlog'"); + clearAlterLocal(); + + setAlterLocalFunc("querypolicy", "2"); + run("ALTER LOCAL 'querypolicy' '2'"); + clearAlterLocal(); +} + // todo ALTER stable /* diff --git a/source/libs/parser/test/parInitialCTest.cpp b/source/libs/parser/test/parInitialCTest.cpp index 8b2c2aade7ae7df398841279184da53e15608825..1ab251fa4839fe3396338e3ce6f322bacaac8cdc 100644 --- a/source/libs/parser/test/parInitialCTest.cpp +++ b/source/libs/parser/test/parInitialCTest.cpp @@ -14,7 +14,6 @@ */ #include "parTestUtil.h" -#include "ttime.h" using namespace std; @@ -242,9 +241,47 @@ TEST_F(ParserInitialCTest, createDatabaseSemanticCheck) { TEST_F(ParserInitialCTest, createDnode) { useDb("root", "test"); - run("CREATE DNODE abc1 PORT 7000"); + SCreateDnodeReq expect = {0}; - run("CREATE DNODE 1.1.1.1 PORT 9000"); + auto clearCreateDnodeReq = [&]() { memset(&expect, 0, sizeof(SCreateDnodeReq)); }; + + auto setCreateDnodeReqFunc = [&](const char* pFqdn, int32_t port = tsServerPort) { + strcpy(expect.fqdn, pFqdn); + expect.port = port; + }; + + setCheckDdlFunc([&](const SQuery* pQuery, ParserStage stage) { + ASSERT_EQ(nodeType(pQuery->pRoot), QUERY_NODE_CREATE_DNODE_STMT); + SCreateDnodeReq req = {0}; + ASSERT_TRUE(TSDB_CODE_SUCCESS == tDeserializeSCreateDnodeReq(pQuery->pCmdMsg->pMsg, pQuery->pCmdMsg->msgLen, &req)); + + ASSERT_EQ(std::string(req.fqdn), std::string(expect.fqdn)); + ASSERT_EQ(req.port, expect.port); + }); + + setCreateDnodeReqFunc("abc1", 7030); + run("CREATE DNODE 'abc1' PORT 7030"); + clearCreateDnodeReq(); + + setCreateDnodeReqFunc("1.1.1.1", 8030); + run("CREATE DNODE 1.1.1.1 PORT 8030"); + clearCreateDnodeReq(); + + setCreateDnodeReqFunc("host1", 9030); + run("CREATE DNODE host1 PORT 9030"); + clearCreateDnodeReq(); + + setCreateDnodeReqFunc("abc2", 7040); + run("CREATE DNODE 'abc2:7040'"); + clearCreateDnodeReq(); + + setCreateDnodeReqFunc("1.1.1.2"); + run("CREATE DNODE 1.1.1.2"); + clearCreateDnodeReq(); + + setCreateDnodeReqFunc("host2"); + run("CREATE DNODE host2"); + clearCreateDnodeReq(); } // CREATE [AGGREGATE] FUNCTION [IF NOT EXISTS] func_name AS library_path OUTPUTTYPE type_name [BUFSIZE value] @@ -322,13 +359,17 @@ TEST_F(ParserInitialCTest, createStable) { memset(&expect, 0, sizeof(SMCreateStbReq)); }; - auto setCreateStbReqFunc = [&](const char* pTbname, int8_t igExists = 0, - float xFilesFactor = TSDB_DEFAULT_ROLLUP_FILE_FACTOR, + auto setCreateStbReqFunc = [&](const char* pTbname, int8_t igExists = 0, int64_t delay1 = -1, int64_t delay2 = -1, + int64_t watermark1 = TSDB_DEFAULT_ROLLUP_WATERMARK, + int64_t watermark2 = TSDB_DEFAULT_ROLLUP_WATERMARK, int32_t ttl = TSDB_DEFAULT_TABLE_TTL, const char* pComment = nullptr) { int32_t len = snprintf(expect.name, sizeof(expect.name), "0.test.%s", pTbname); expect.name[len] = '\0'; expect.igExists = igExists; - expect.xFilesFactor = xFilesFactor; + expect.delay1 = delay1; + expect.delay2 = delay2; + expect.watermark1 = watermark1; + expect.watermark2 = watermark2; expect.ttl = ttl; if (nullptr != pComment) { expect.comment = strdup(pComment); @@ -366,8 +407,10 @@ TEST_F(ParserInitialCTest, createStable) { ASSERT_EQ(std::string(req.name), std::string(expect.name)); ASSERT_EQ(req.igExists, expect.igExists); - ASSERT_EQ(req.xFilesFactor, expect.xFilesFactor); - ASSERT_EQ(req.delay, expect.delay); + ASSERT_EQ(req.delay1, expect.delay1); + ASSERT_EQ(req.delay2, expect.delay2); + ASSERT_EQ(req.watermark1, expect.watermark1); + ASSERT_EQ(req.watermark2, expect.watermark2); ASSERT_EQ(req.ttl, expect.ttl); ASSERT_EQ(req.numOfColumns, expect.numOfColumns); ASSERT_EQ(req.numOfTags, expect.numOfTags); @@ -418,7 +461,8 @@ TEST_F(ParserInitialCTest, createStable) { run("CREATE STABLE t1(ts TIMESTAMP, c1 INT) TAGS(id INT)"); clearCreateStbReq(); - setCreateStbReqFunc("t1", 1, 0.1, 100, "test create table"); + setCreateStbReqFunc("t1", 1, 100 * MILLISECOND_PER_SECOND, 10 * MILLISECOND_PER_MINUTE, 10, + 1 * MILLISECOND_PER_MINUTE, 100, "test create table"); addFieldToCreateStbReqFunc(true, "ts", TSDB_DATA_TYPE_TIMESTAMP, 0, 0); addFieldToCreateStbReqFunc(true, "c1", TSDB_DATA_TYPE_INT); addFieldToCreateStbReqFunc(true, "c2", TSDB_DATA_TYPE_UINT); @@ -456,15 +500,20 @@ TEST_F(ParserInitialCTest, createStable) { "TAGS (a1 TIMESTAMP, a2 INT, a3 INT UNSIGNED, a4 BIGINT, a5 BIGINT UNSIGNED, a6 FLOAT, a7 DOUBLE, " "a8 BINARY(20), a9 SMALLINT, a10 SMALLINT UNSIGNED COMMENT 'test column comment', a11 TINYINT, " "a12 TINYINT UNSIGNED, a13 BOOL, a14 NCHAR(30), a15 VARCHAR(50)) " - "TTL 100 COMMENT 'test create table' SMA(c1, c2, c3) ROLLUP (MIN) FILE_FACTOR 0.1"); + "TTL 100 COMMENT 'test create table' SMA(c1, c2, c3) ROLLUP (MIN) MAX_DELAY 100s,10m WATERMARK 10a,1m"); clearCreateStbReq(); } TEST_F(ParserInitialCTest, createStableSemanticCheck) { useDb("root", "test"); - run("CREATE STABLE stb2 (ts TIMESTAMP, c1 INT) TAGS (tag1 INT) ROLLUP(CEIL) FILE_FACTOR 0.1", - TSDB_CODE_PAR_INVALID_ROLLUP_OPTION, PARSER_STAGE_TRANSLATE); + run("CREATE STABLE stb2 (ts TIMESTAMP, c1 INT) TAGS (tag1 INT) ROLLUP(CEIL)", TSDB_CODE_PAR_INVALID_ROLLUP_OPTION); + + run("CREATE STABLE stb2 (ts TIMESTAMP, c1 INT) TAGS (tag1 INT) ROLLUP(MAX) MAX_DELAY 0s WATERMARK 1m", + TSDB_CODE_PAR_INVALID_RANGE_OPTION); + + run("CREATE STABLE stb2 (ts TIMESTAMP, c1 INT) TAGS (tag1 INT) ROLLUP(MAX) MAX_DELAY 10s WATERMARK 18m", + TSDB_CODE_PAR_INVALID_RANGE_OPTION); } TEST_F(ParserInitialCTest, createStream) { @@ -477,7 +526,7 @@ TEST_F(ParserInitialCTest, createStream) { memset(&expect, 0, sizeof(SCMCreateStreamReq)); }; - auto setCreateStbReqFunc = + auto setCreateStreamReqFunc = [&](const char* pStream, const char* pSrcDb, const char* pSql, const char* pDstStb = nullptr, int8_t igExists = 0, int8_t triggerType = STREAM_TRIGGER_AT_ONCE, int64_t maxDelay = 0, int64_t watermark = 0) { snprintf(expect.name, sizeof(expect.name), "0.%s", pStream); @@ -509,25 +558,31 @@ TEST_F(ParserInitialCTest, createStream) { tFreeSCMCreateStreamReq(&req); }); - setCreateStbReqFunc("s1", "test", "create stream s1 as select * from t1"); + setCreateStreamReqFunc("s1", "test", "create stream s1 as select * from t1"); run("CREATE STREAM s1 AS SELECT * FROM t1"); clearCreateStreamReq(); - setCreateStbReqFunc("s1", "test", "create stream if not exists s1 as select * from t1", nullptr, 1); + setCreateStreamReqFunc("s1", "test", "create stream if not exists s1 as select * from t1", nullptr, 1); run("CREATE STREAM IF NOT EXISTS s1 AS SELECT * FROM t1"); clearCreateStreamReq(); - setCreateStbReqFunc("s1", "test", "create stream s1 into st1 as select * from t1", "st1"); + setCreateStreamReqFunc("s1", "test", "create stream s1 into st1 as select * from t1", "st1"); run("CREATE STREAM s1 INTO st1 AS SELECT * FROM t1"); clearCreateStreamReq(); - setCreateStbReqFunc("s1", "test", - "create stream if not exists s1 trigger max_delay 20s watermark 10s into st1 as select * from t1", - "st1", 1, STREAM_TRIGGER_MAX_DELAY, 20 * MILLISECOND_PER_SECOND, 10 * MILLISECOND_PER_SECOND); + setCreateStreamReqFunc( + "s1", "test", "create stream if not exists s1 trigger max_delay 20s watermark 10s into st1 as select * from t1", + "st1", 1, STREAM_TRIGGER_MAX_DELAY, 20 * MILLISECOND_PER_SECOND, 10 * MILLISECOND_PER_SECOND); run("CREATE STREAM IF NOT EXISTS s1 TRIGGER MAX_DELAY 20s WATERMARK 10s INTO st1 AS SELECT * FROM t1"); clearCreateStreamReq(); } +TEST_F(ParserInitialCTest, createStreamSemanticCheck) { + useDb("root", "test"); + + run("CREATE STREAM s1 AS SELECT PERCENTILE(c1, 30) FROM t1", TSDB_CODE_PAR_STREAM_NOT_ALLOWED_FUNC); +} + TEST_F(ParserInitialCTest, createTable) { useDb("root", "test"); @@ -546,7 +601,7 @@ TEST_F(ParserInitialCTest, createTable) { "TAGS (a1 TIMESTAMP, a2 INT, a3 INT UNSIGNED, a4 BIGINT, a5 BIGINT UNSIGNED, a6 FLOAT, a7 DOUBLE, a8 BINARY(20), " "a9 SMALLINT, a10 SMALLINT UNSIGNED COMMENT 'test column comment', a11 TINYINT, a12 TINYINT UNSIGNED, a13 BOOL, " "a14 NCHAR(30), a15 VARCHAR(50)) " - "TTL 100 COMMENT 'test create table' SMA(c1, c2, c3) ROLLUP (MIN) FILE_FACTOR 0.1"); + "TTL 100 COMMENT 'test create table' SMA(c1, c2, c3) ROLLUP (MIN)"); run("CREATE TABLE IF NOT EXISTS t1 USING st1 TAGS(1, 'wxy', NOW)"); diff --git a/source/libs/parser/test/parInitialDTest.cpp b/source/libs/parser/test/parInitialDTest.cpp index 8562926789a8f4a8307547d30688ab519d1f52ce..4ecbb7b6d80990c38b37eb1106a5e8d0bf140b28 100644 --- a/source/libs/parser/test/parInitialDTest.cpp +++ b/source/libs/parser/test/parInitialDTest.cpp @@ -64,8 +64,9 @@ TEST_F(ParserInitialDTest, dropConsumerGroup) { SMDropCgroupReq expect = {0}; - auto setDropCgroupReqFunc = [&](const char* pTopicName, const char* pCGroupName, int8_t igNotExists = 0) { - memset(&expect, 0, sizeof(SMDropCgroupReq)); + auto clearDropCgroupReq = [&]() { memset(&expect, 0, sizeof(SMDropCgroupReq)); }; + + auto setDropCgroupReq = [&](const char* pTopicName, const char* pCGroupName, int8_t igNotExists = 0) { snprintf(expect.topic, sizeof(expect.topic), "0.%s", pTopicName); strcpy(expect.cgroup, pCGroupName); expect.igNotExists = igNotExists; @@ -81,15 +82,51 @@ TEST_F(ParserInitialDTest, dropConsumerGroup) { ASSERT_EQ(req.igNotExists, expect.igNotExists); }); - setDropCgroupReqFunc("tp1", "cg1"); + setDropCgroupReq("tp1", "cg1"); run("DROP CONSUMER GROUP cg1 ON tp1"); + clearDropCgroupReq(); - setDropCgroupReqFunc("tp1", "cg1", 1); + setDropCgroupReq("tp1", "cg1", 1); run("DROP CONSUMER GROUP IF EXISTS cg1 ON tp1"); + clearDropCgroupReq(); } // todo DROP database + // todo DROP dnode +TEST_F(ParserInitialDTest, dropDnode) { + useDb("root", "test"); + + SDropDnodeReq expect = {0}; + + auto clearDropDnodeReq = [&]() { memset(&expect, 0, sizeof(SDropDnodeReq)); }; + + auto setDropDnodeReqById = [&](int32_t dnodeId) { expect.dnodeId = dnodeId; }; + + auto setDropDnodeReqByEndpoint = [&](const char* pFqdn, int32_t port) { + strcpy(expect.fqdn, pFqdn); + expect.port = port; + }; + + setCheckDdlFunc([&](const SQuery* pQuery, ParserStage stage) { + ASSERT_EQ(nodeType(pQuery->pRoot), QUERY_NODE_DROP_DNODE_STMT); + SDropDnodeReq req = {0}; + ASSERT_TRUE(TSDB_CODE_SUCCESS == tDeserializeSDropDnodeReq(pQuery->pCmdMsg->pMsg, pQuery->pCmdMsg->msgLen, &req)); + + ASSERT_EQ(req.dnodeId, expect.dnodeId); + ASSERT_EQ(std::string(req.fqdn), std::string(expect.fqdn)); + ASSERT_EQ(req.port, expect.port); + }); + + setDropDnodeReqById(1); + run("DROP DNODE 1"); + clearDropDnodeReq(); + + setDropDnodeReqByEndpoint("host1", 7030); + run("DROP DNODE 'host1:7030'"); + clearDropDnodeReq(); +} + // todo DROP function TEST_F(ParserInitialDTest, dropIndex) { diff --git a/source/libs/parser/test/parSelectTest.cpp b/source/libs/parser/test/parSelectTest.cpp index c87520e26206cd4e2c35aacc8fd428849e9b77a2..a69b01257461849b8922ff8651128b80bb8be921 100644 --- a/source/libs/parser/test/parSelectTest.cpp +++ b/source/libs/parser/test/parSelectTest.cpp @@ -161,6 +161,40 @@ TEST_F(ParserSelectTest, useDefinedFunc) { run("SELECT udf2(c1) FROM t1 GROUP BY c2"); } +TEST_F(ParserSelectTest, uniqueFunc) { + useDb("root", "test"); + + run("SELECT UNIQUE(c1) FROM t1"); + + run("SELECT UNIQUE(c2 + 10) FROM t1 WHERE c1 > 10"); + + run("SELECT UNIQUE(c2 + 10), ts, c2 FROM t1 WHERE c1 > 10"); +} + +TEST_F(ParserSelectTest, uniqueFuncSemanticCheck) { + useDb("root", "test"); + + run("SELECT UNIQUE(c1) FROM t1 INTERVAL(10S)", TSDB_CODE_PAR_WINDOW_NOT_ALLOWED_FUNC); + + run("SELECT UNIQUE(c1) FROM t1 GROUP BY c2", TSDB_CODE_PAR_GROUP_BY_NOT_ALLOWED_FUNC); +} + +TEST_F(ParserSelectTest, tailFunc) { + useDb("root", "test"); + + run("SELECT TAIL(c1, 10) FROM t1"); + + run("SELECT TAIL(c2 + 10, 10, 80) FROM t1 WHERE c1 > 10"); +} + +TEST_F(ParserSelectTest, tailFuncSemanticCheck) { + useDb("root", "test"); + + run("SELECT TAIL(c1, 10) FROM t1 INTERVAL(10S)", TSDB_CODE_PAR_WINDOW_NOT_ALLOWED_FUNC); + + run("SELECT TAIL(c1, 10) FROM t1 GROUP BY c2", TSDB_CODE_PAR_GROUP_BY_NOT_ALLOWED_FUNC); +} + TEST_F(ParserSelectTest, groupBy) { useDb("root", "test"); @@ -328,6 +362,8 @@ TEST_F(ParserSelectTest, setOperator) { run("(SELECT * FROM t1) UNION ALL (SELECT * FROM t1)"); run("SELECT c1 FROM (SELECT c1 FROM t1 UNION ALL SELECT c1 FROM t1)"); + + run("SELECT c1, c2 FROM t1 UNION ALL SELECT c1 as a, c2 as b FROM t1 ORDER BY c1"); } TEST_F(ParserSelectTest, informationSchema) { diff --git a/source/libs/parser/test/parShowToUse.cpp b/source/libs/parser/test/parShowToUse.cpp index d468ff25e330236da3f37e3c4f4472e4d8eb4910..940b6ea8ac101bff71416d0abd4e20e2d2cd908b 100644 --- a/source/libs/parser/test/parShowToUse.cpp +++ b/source/libs/parser/test/parShowToUse.cpp @@ -24,9 +24,45 @@ class ParserShowToUseTest : public ParserDdlTest {}; // todo SHOW accounts // todo SHOW apps // todo SHOW connections -// todo SHOW create database -// todo SHOW create stable -// todo SHOW create table + +TEST_F(ParserShowToUseTest, showCreateDatabase) { + useDb("root", "test"); + + setCheckDdlFunc([&](const SQuery* pQuery, ParserStage stage) { + ASSERT_EQ(nodeType(pQuery->pRoot), QUERY_NODE_SHOW_CREATE_DATABASE_STMT); + ASSERT_EQ(pQuery->execMode, QUERY_EXEC_MODE_LOCAL); + ASSERT_TRUE(pQuery->haveResultSet); + ASSERT_NE(((SShowCreateDatabaseStmt*)pQuery->pRoot)->pCfg, nullptr); + }); + + run("SHOW CREATE DATABASE test"); +} + +TEST_F(ParserShowToUseTest, showCreateSTable) { + useDb("root", "test"); + + setCheckDdlFunc([&](const SQuery* pQuery, ParserStage stage) { + ASSERT_EQ(nodeType(pQuery->pRoot), QUERY_NODE_SHOW_CREATE_STABLE_STMT); + ASSERT_EQ(pQuery->execMode, QUERY_EXEC_MODE_LOCAL); + ASSERT_TRUE(pQuery->haveResultSet); + ASSERT_NE(((SShowCreateTableStmt*)pQuery->pRoot)->pMeta, nullptr); + }); + + run("SHOW CREATE STABLE st1"); +} + +TEST_F(ParserShowToUseTest, showCreateTable) { + useDb("root", "test"); + + setCheckDdlFunc([&](const SQuery* pQuery, ParserStage stage) { + ASSERT_EQ(nodeType(pQuery->pRoot), QUERY_NODE_SHOW_CREATE_TABLE_STMT); + ASSERT_EQ(pQuery->execMode, QUERY_EXEC_MODE_LOCAL); + ASSERT_TRUE(pQuery->haveResultSet); + ASSERT_NE(((SShowCreateTableStmt*)pQuery->pRoot)->pMeta, nullptr); + }); + + run("SHOW CREATE TABLE t1"); +} TEST_F(ParserShowToUseTest, showDatabases) { useDb("root", "test"); diff --git a/source/libs/parser/test/parTestUtil.h b/source/libs/parser/test/parTestUtil.h index afdb76344837781c14145eb565187d482b161399..16c3d05b38cd66ba148696fa629728b04ccf4afa 100644 --- a/source/libs/parser/test/parTestUtil.h +++ b/source/libs/parser/test/parTestUtil.h @@ -20,8 +20,11 @@ #define ALLOW_FORBID_FUNC +#include "cmdnodes.h" #include "querynodes.h" #include "taoserror.h" +#include "tglobal.h" +#include "ttime.h" namespace ParserTest { diff --git a/source/libs/planner/src/planLogicCreater.c b/source/libs/planner/src/planLogicCreater.c index cbc74a2711d5ef4621041dcff86853b86888b55e..3cd5eeb6555733834afa79be7dff70f35592b64f 100644 --- a/source/libs/planner/src/planLogicCreater.c +++ b/source/libs/planner/src/planLogicCreater.c @@ -548,6 +548,7 @@ static int32_t createWindowLogicNodeBySession(SLogicPlanContext* pCxt, SSessionW pWindow->winType = WINDOW_TYPE_SESSION; pWindow->sessionGap = ((SValueNode*)pSession->pGap)->datum.i; + pWindow->windowAlgo = pCxt->pPlanCxt->streamQuery ? SESSION_ALGO_STREAM_SINGLE : SESSION_ALGO_MERGE; pWindow->pTspk = nodesCloneNode((SNode*)pSession->pCol); if (NULL == pWindow->pTspk) { @@ -572,7 +573,7 @@ static int32_t createWindowLogicNodeByInterval(SLogicPlanContext* pCxt, SInterva pWindow->sliding = (NULL != pInterval->pSliding ? ((SValueNode*)pInterval->pSliding)->datum.i : pWindow->interval); pWindow->slidingUnit = (NULL != pInterval->pSliding ? ((SValueNode*)pInterval->pSliding)->unit : pWindow->intervalUnit); - pWindow->intervalAlgo = pCxt->pPlanCxt->streamQuery ? INTERVAL_ALGO_STREAM_SINGLE : INTERVAL_ALGO_HASH; + pWindow->windowAlgo = pCxt->pPlanCxt->streamQuery ? INTERVAL_ALGO_STREAM_SINGLE : INTERVAL_ALGO_HASH; pWindow->pTspk = nodesCloneNode(pInterval->pCol); if (NULL == pWindow->pTspk) { diff --git a/source/libs/planner/src/planOptimizer.c b/source/libs/planner/src/planOptimizer.c index 9d7cd0cf27d1674a2555c30dbc3cd9f4d3d61f5e..a7c25162b7834a941cb71499229daa644df9b3e4 100644 --- a/source/libs/planner/src/planOptimizer.c +++ b/source/libs/planner/src/planOptimizer.c @@ -731,6 +731,7 @@ static int32_t opkDoOptimized(SOptimizeContext* pCxt, SSortLogicNode* pSort, SNo FOREACH(pNode, pSort->node.pParent->pChildren) { if (nodesEqualNode(pNode, (SNode*)pSort)) { REPLACE_NODE(pDownNode); + ((SLogicNode*)pDownNode)->pParent = pSort->node.pParent; break; } } diff --git a/source/libs/planner/src/planPhysiCreater.c b/source/libs/planner/src/planPhysiCreater.c index 2974b3ef8cb7517f1de34ca1e81b26bb4df6130c..656976b3cac2974fbd59e68ec87ad5517dbbb638 100644 --- a/source/libs/planner/src/planPhysiCreater.c +++ b/source/libs/planner/src/planPhysiCreater.c @@ -35,7 +35,7 @@ typedef struct SPhysiPlanContext { int32_t errCode; int16_t nextDataBlockId; SArray* pLocationHelper; - SArray* pExecNodeList; + SArray* pExecNodeList; // SArray } SPhysiPlanContext; static int32_t getSlotKey(SNode* pNode, const char* pStmtName, char* pKey) { @@ -459,7 +459,7 @@ static int32_t createTagScanPhysiNode(SPhysiPlanContext* pCxt, SSubplan* pSubpla } vgroupInfoToNodeAddr(pScanLogicNode->pVgroupList->vgroups, &pSubplan->execNode); SQueryNodeLoad node = {.addr = pSubplan->execNode, .load = 0}; - taosArrayPush(pCxt->pExecNodeList, &pSubplan->execNode); + taosArrayPush(pCxt->pExecNodeList, &node); return createScanPhysiNodeFinalize(pCxt, pSubplan, pScanLogicNode, (SScanPhysiNode*)pTagScan, pPhyNode); } @@ -529,10 +529,11 @@ static int32_t createSystemTableScanPhysiNode(SPhysiPlanContext* pCxt, SSubplan* pScan->showRewrite = pScanLogicNode->showRewrite; pScan->accountId = pCxt->pPlanCxt->acctId; - if (0 == strcmp(pScanLogicNode->tableName.tname, TSDB_INS_TABLE_USER_TABLES)) { + if (0 == strcmp(pScanLogicNode->tableName.tname, TSDB_INS_TABLE_USER_TABLES) || + 0 == strcmp(pScanLogicNode->tableName.tname, TSDB_INS_TABLE_USER_TABLE_DISTRIBUTED)) { vgroupInfoToNodeAddr(pScanLogicNode->pVgroupList->vgroups, &pSubplan->execNode); SQueryNodeLoad node = {.addr = pSubplan->execNode, .load = 0}; - taosArrayPush(pCxt->pExecNodeList, &pSubplan->execNode); + taosArrayPush(pCxt->pExecNodeList, &node); } else { SQueryNodeLoad node = {.addr = {.nodeId = MNODE_HANDLE, .epSet = pCxt->pPlanCxt->mgmtEpSet}, .load = 0}; taosArrayPush(pCxt->pExecNodeList, &node); @@ -977,8 +978,8 @@ static int32_t createWindowPhysiNodeFinalize(SPhysiPlanContext* pCxt, SNodeList* return code; } -static ENodeType getIntervalOperatorType(EIntervalAlgorithm intervalAlgo) { - switch (intervalAlgo) { +static ENodeType getIntervalOperatorType(EWindowAlgorithm windowAlgo) { + switch (windowAlgo) { case INTERVAL_ALGO_HASH: return QUERY_NODE_PHYSICAL_PLAN_HASH_INTERVAL; case INTERVAL_ALGO_MERGE: @@ -989,6 +990,14 @@ static ENodeType getIntervalOperatorType(EIntervalAlgorithm intervalAlgo) { return QUERY_NODE_PHYSICAL_PLAN_STREAM_SEMI_INTERVAL; case INTERVAL_ALGO_STREAM_SINGLE: return QUERY_NODE_PHYSICAL_PLAN_STREAM_INTERVAL; + case SESSION_ALGO_STREAM_FINAL: + return QUERY_NODE_PHYSICAL_PLAN_STREAM_FINAL_SESSION; + case SESSION_ALGO_STREAM_SEMI: + return QUERY_NODE_PHYSICAL_PLAN_STREAM_SEMI_SESSION; + case SESSION_ALGO_STREAM_SINGLE: + return QUERY_NODE_PHYSICAL_PLAN_STREAM_SESSION; + case SESSION_ALGO_MERGE: + return QUERY_NODE_PHYSICAL_PLAN_MERGE_SESSION; default: break; } @@ -998,7 +1007,7 @@ static ENodeType getIntervalOperatorType(EIntervalAlgorithm intervalAlgo) { static int32_t createIntervalPhysiNode(SPhysiPlanContext* pCxt, SNodeList* pChildren, SWindowLogicNode* pWindowLogicNode, SPhysiNode** pPhyNode) { SIntervalPhysiNode* pInterval = (SIntervalPhysiNode*)makePhysiNode( - pCxt, (SLogicNode*)pWindowLogicNode, getIntervalOperatorType(pWindowLogicNode->intervalAlgo)); + pCxt, (SLogicNode*)pWindowLogicNode, getIntervalOperatorType(pWindowLogicNode->windowAlgo)); if (NULL == pInterval) { return TSDB_CODE_OUT_OF_MEMORY; } @@ -1015,8 +1024,7 @@ static int32_t createIntervalPhysiNode(SPhysiPlanContext* pCxt, SNodeList* pChil static int32_t createSessionWindowPhysiNode(SPhysiPlanContext* pCxt, SNodeList* pChildren, SWindowLogicNode* pWindowLogicNode, SPhysiNode** pPhyNode) { SSessionWinodwPhysiNode* pSession = (SSessionWinodwPhysiNode*)makePhysiNode( - pCxt, (SLogicNode*)pWindowLogicNode, - (pCxt->pPlanCxt->streamQuery ? QUERY_NODE_PHYSICAL_PLAN_STREAM_SESSION : QUERY_NODE_PHYSICAL_PLAN_MERGE_SESSION)); + pCxt, (SLogicNode*)pWindowLogicNode, getIntervalOperatorType(pWindowLogicNode->windowAlgo)); if (NULL == pSession) { return TSDB_CODE_OUT_OF_MEMORY; } diff --git a/source/libs/planner/src/planSpliter.c b/source/libs/planner/src/planSpliter.c index 3a9e1d74056a01d182bee21529138ff72a7e0d67..fe02dd6425fdf8ed4190778ed85abddbbde14b6b 100644 --- a/source/libs/planner/src/planSpliter.c +++ b/source/libs/planner/src/planSpliter.c @@ -376,8 +376,8 @@ static int32_t stbSplSplitIntervalForBatch(SSplitContext* pCxt, SStableSplitInfo SLogicNode* pPartWindow = NULL; int32_t code = stbSplCreatePartWindowNode((SWindowLogicNode*)pInfo->pSplitNode, &pPartWindow); if (TSDB_CODE_SUCCESS == code) { - ((SWindowLogicNode*)pPartWindow)->intervalAlgo = INTERVAL_ALGO_HASH; - ((SWindowLogicNode*)pInfo->pSplitNode)->intervalAlgo = INTERVAL_ALGO_MERGE; + ((SWindowLogicNode*)pPartWindow)->windowAlgo = INTERVAL_ALGO_HASH; + ((SWindowLogicNode*)pInfo->pSplitNode)->windowAlgo = INTERVAL_ALGO_MERGE; SNodeList* pMergeKeys = NULL; code = stbSplCreateMergeKeysByPrimaryKey(((SWindowLogicNode*)pInfo->pSplitNode)->pTspk, &pMergeKeys); if (TSDB_CODE_SUCCESS == code) { @@ -400,8 +400,8 @@ static int32_t stbSplSplitIntervalForStream(SSplitContext* pCxt, SStableSplitInf SLogicNode* pPartWindow = NULL; int32_t code = stbSplCreatePartWindowNode((SWindowLogicNode*)pInfo->pSplitNode, &pPartWindow); if (TSDB_CODE_SUCCESS == code) { - ((SWindowLogicNode*)pPartWindow)->intervalAlgo = INTERVAL_ALGO_STREAM_SEMI; - ((SWindowLogicNode*)pInfo->pSplitNode)->intervalAlgo = INTERVAL_ALGO_STREAM_FINAL; + ((SWindowLogicNode*)pPartWindow)->windowAlgo = INTERVAL_ALGO_STREAM_SEMI; + ((SWindowLogicNode*)pInfo->pSplitNode)->windowAlgo = INTERVAL_ALGO_STREAM_FINAL; code = stbSplCreateExchangeNode(pCxt, pInfo->pSplitNode, pPartWindow); } if (TSDB_CODE_SUCCESS == code) { @@ -421,8 +421,29 @@ static int32_t stbSplSplitInterval(SSplitContext* pCxt, SStableSplitInfo* pInfo) } } +static int32_t stbSplSplitSessionForStream(SSplitContext* pCxt, SStableSplitInfo* pInfo) { + SLogicNode* pPartWindow = NULL; + int32_t code = stbSplCreatePartWindowNode((SWindowLogicNode*)pInfo->pSplitNode, &pPartWindow); + if (TSDB_CODE_SUCCESS == code) { + ((SWindowLogicNode*)pPartWindow)->windowAlgo = SESSION_ALGO_STREAM_SEMI; + ((SWindowLogicNode*)pInfo->pSplitNode)->windowAlgo = SESSION_ALGO_STREAM_FINAL; + code = stbSplCreateExchangeNode(pCxt, pInfo->pSplitNode, pPartWindow); + } + if (TSDB_CODE_SUCCESS == code) { + code = nodesListMakeStrictAppend(&pInfo->pSubplan->pChildren, + (SNode*)splCreateScanSubplan(pCxt, pPartWindow, SPLIT_FLAG_STABLE_SPLIT)); + } + pInfo->pSubplan->subplanType = SUBPLAN_TYPE_MERGE; + ++(pCxt->groupId); + return code; +} + static int32_t stbSplSplitSession(SSplitContext* pCxt, SStableSplitInfo* pInfo) { - return TSDB_CODE_PLAN_INTERNAL_ERROR; + if (pCxt->pPlanCxt->streamQuery) { + return stbSplSplitSessionForStream(pCxt, pInfo); + } else { + return TSDB_CODE_PLAN_INTERNAL_ERROR; + } } static int32_t stbSplSplitWindowNode(SSplitContext* pCxt, SStableSplitInfo* pInfo) { @@ -537,10 +558,12 @@ static int32_t stbSplCreateMergeKeys(SNodeList* pSortKeys, SNodeList* pTargets, SNode* pNode = NULL; FOREACH(pNode, pSortKeys) { SOrderByExprNode* pSortKey = (SOrderByExprNode*)pNode; + SExprNode* pSortExpr = (SExprNode*)pSortKey->pExpr; SNode* pTarget = NULL; bool found = false; FOREACH(pTarget, pTargets) { - if (0 == strcmp(((SExprNode*)pSortKey->pExpr)->aliasName, ((SColumnNode*)pTarget)->colName)) { + if ((QUERY_NODE_COLUMN == nodeType(pSortExpr) && nodesEqualNode((SNode*)pSortExpr, pTarget)) || + (0 == strcmp(pSortExpr->aliasName, ((SColumnNode*)pTarget)->colName))) { code = nodesListMakeStrictAppend(&pMergeKeys, stbSplCreateOrderByExpr(pSortKey, pTarget)); if (TSDB_CODE_SUCCESS != code) { break; @@ -549,7 +572,7 @@ static int32_t stbSplCreateMergeKeys(SNodeList* pSortKeys, SNodeList* pTargets, } } if (TSDB_CODE_SUCCESS == code && !found) { - SNode* pCol = stbSplCreateColumnNode((SExprNode*)pSortKey->pExpr); + SNode* pCol = stbSplCreateColumnNode(pSortExpr); code = nodesListMakeStrictAppend(&pMergeKeys, stbSplCreateOrderByExpr(pSortKey, pCol)); if (TSDB_CODE_SUCCESS == code) { code = nodesListStrictAppend(pTargets, pCol); diff --git a/source/libs/planner/src/planner.c b/source/libs/planner/src/planner.c index c4385d576eef29e2975ae419a53e3186704628b5..e75c8375fb766a662b87c8e5c7e43af28a45b097 100644 --- a/source/libs/planner/src/planner.c +++ b/source/libs/planner/src/planner.c @@ -21,7 +21,7 @@ static void dumpQueryPlan(SQueryPlan* pPlan) { char* pStr = NULL; nodesNodeToString((SNode*)pPlan, false, &pStr, NULL); - planDebugL("Query Plan: %s", pStr); + planDebugL("QID:0x%" PRIx64 " Query Plan: %s", pPlan->queryId, pStr); taosMemoryFree(pStr); } diff --git a/source/libs/planner/test/planBasicTest.cpp b/source/libs/planner/test/planBasicTest.cpp index 4b84079f7bd417b538a789ff964675208d9ea0bf..c075bfeee85a45893fbf09ab4677302cd0d36cc0 100644 --- a/source/libs/planner/test/planBasicTest.cpp +++ b/source/libs/planner/test/planBasicTest.cpp @@ -53,3 +53,23 @@ TEST_F(PlanBasicTest, func) { run("SELECT TOP(c1, 60) FROM t1"); } + +TEST_F(PlanBasicTest, uniqueFunc) { + useDb("root", "test"); + + run("SELECT UNIQUE(c1) FROM t1"); + + run("SELECT UNIQUE(c2 + 10) FROM t1 WHERE c1 > 10"); + + run("SELECT UNIQUE(c2 + 10), ts, c2 FROM t1 WHERE c1 > 10"); + + run("SELECT UNIQUE(c1) a FROM t1 ORDER BY a"); +} + +TEST_F(PlanBasicTest, tailFunc) { + useDb("root", "test"); + + run("SELECT TAIL(c1, 10) FROM t1"); + + run("SELECT TAIL(c2 + 10, 10, 80) FROM t1 WHERE c1 > 10"); +} diff --git a/source/libs/planner/test/planOrderByTest.cpp b/source/libs/planner/test/planOrderByTest.cpp index 39e93fcff929ddb6a43122c81a47f7f13a7ff4f6..2ca662cf8665946faef361f9b557d4e32403b983 100644 --- a/source/libs/planner/test/planOrderByTest.cpp +++ b/source/libs/planner/test/planOrderByTest.cpp @@ -27,6 +27,10 @@ TEST_F(PlanOrderByTest, basic) { run("SELECT c1 FROM t1 ORDER BY c1"); // ORDER BY key is not in the projection list run("SELECT c1 FROM t1 ORDER BY c2"); + + run("SELECT c1 AS a FROM t1 ORDER BY a"); + + run("SELECT c1 + 10 AS a FROM t1 ORDER BY a"); } TEST_F(PlanOrderByTest, expr) { @@ -41,6 +45,12 @@ TEST_F(PlanOrderByTest, nullsOrder) { run("SELECT * FROM t1 ORDER BY c1 DESC NULLS FIRST"); } +TEST_F(PlanOrderByTest, withGroupBy) { + useDb("root", "test"); + + run("SELECT SUM(c1) AS a FROM t1 GROUP BY c2 ORDER BY a"); +} + TEST_F(PlanOrderByTest, stable) { useDb("root", "test"); @@ -51,4 +61,6 @@ TEST_F(PlanOrderByTest, stable) { run("SELECT c2 FROM st1 ORDER BY c1"); run("SELECT c2 FROM st1 PARTITION BY c2 ORDER BY c1"); + + run("SELECT c1 AS a FROM st1 ORDER BY a"); } diff --git a/source/libs/planner/test/planSetOpTest.cpp b/source/libs/planner/test/planSetOpTest.cpp index 5568d3cfbb994b83c3531957a33fc7ae1ff20eb3..bf26f55c02b1c146c0ff8aad723a593956e4945f 100644 --- a/source/libs/planner/test/planSetOpTest.cpp +++ b/source/libs/planner/test/planSetOpTest.cpp @@ -23,9 +23,9 @@ class PlanSetOpTest : public PlannerTestBase {}; TEST_F(PlanSetOpTest, unionAll) { useDb("root", "test"); - // sql 1: single UNION ALL operator + // single UNION ALL operator run("SELECT c1, c2 FROM t1 WHERE c1 > 10 UNION ALL SELECT c1, c2 FROM t1 WHERE c1 > 20"); - // sql 2: multi UNION ALL operator + // multi UNION ALL operator run("SELECT c1, c2 FROM t1 WHERE c1 > 10 " "UNION ALL SELECT c1, c2 FROM t1 WHERE c1 > 20 " "UNION ALL SELECT c1, c2 FROM t1 WHERE c1 > 30"); @@ -46,6 +46,14 @@ TEST_F(PlanSetOpTest, unionAllWithSubquery) { run("SELECT ts FROM (SELECT ts FROM st1) UNION ALL SELECT ts FROM (SELECT ts FROM st1)"); } +TEST_F(PlanSetOpTest, unionAllWithOrderBy) { + useDb("root", "test"); + + run("SELECT c1, c2 FROM t1 WHERE c1 > 10 UNION ALL SELECT c1, c2 FROM t1 WHERE c1 > 20 ORDER BY c1"); + + run("SELECT c1, c2 FROM t1 WHERE c1 > 10 UNION ALL SELECT c1, c2 FROM t1 WHERE c1 > 20 ORDER BY 1"); +} + TEST_F(PlanSetOpTest, union) { useDb("root", "test"); diff --git a/source/libs/qcom/src/queryUtil.c b/source/libs/qcom/src/queryUtil.c index c945ca00fd03ba324d039ad6c2c3e4b34c9e94bb..2120d24d2612e804a16827125cc7356449dd0d70 100644 --- a/source/libs/qcom/src/queryUtil.c +++ b/source/libs/qcom/src/queryUtil.c @@ -177,10 +177,6 @@ char* jobTaskStatusStr(int32_t status) { return "SUCCEED"; case JOB_TASK_STATUS_FAILED: return "FAILED"; - case JOB_TASK_STATUS_CANCELLING: - return "CANCELLING"; - case JOB_TASK_STATUS_CANCELLED: - return "CANCELLED"; case JOB_TASK_STATUS_DROPPING: return "DROPPING"; default: diff --git a/source/libs/qworker/src/qwDbg.c b/source/libs/qworker/src/qwDbg.c index 368c3bb517e18c8b9254f4d90bdd4b8aa5bb0d2d..add9700a3a68ea8b0b435be54e3df3c4d0d9d7d0 100644 --- a/source/libs/qworker/src/qwDbg.c +++ b/source/libs/qworker/src/qwDbg.c @@ -44,40 +44,30 @@ int32_t qwDbgValidateStatus(QW_FPARAMS_DEF, int8_t oriStatus, int8_t newStatus, break; case JOB_TASK_STATUS_EXECUTING: if (newStatus != JOB_TASK_STATUS_PARTIAL_SUCCEED && newStatus != JOB_TASK_STATUS_SUCCEED && - newStatus != JOB_TASK_STATUS_FAILED && newStatus != JOB_TASK_STATUS_CANCELLING && - newStatus != JOB_TASK_STATUS_CANCELLED && newStatus != JOB_TASK_STATUS_DROPPING) { + newStatus != JOB_TASK_STATUS_FAILED && newStatus != JOB_TASK_STATUS_DROPPING) { QW_ERR_JRET(TSDB_CODE_QRY_APP_ERROR); } break; case JOB_TASK_STATUS_PARTIAL_SUCCEED: if (newStatus != JOB_TASK_STATUS_EXECUTING && newStatus != JOB_TASK_STATUS_SUCCEED && - newStatus != JOB_TASK_STATUS_CANCELLED && newStatus != JOB_TASK_STATUS_FAILED && - newStatus != JOB_TASK_STATUS_DROPPING) { + newStatus != JOB_TASK_STATUS_FAILED && newStatus != JOB_TASK_STATUS_DROPPING) { QW_ERR_JRET(TSDB_CODE_QRY_APP_ERROR); } break; case JOB_TASK_STATUS_SUCCEED: - if (newStatus != JOB_TASK_STATUS_CANCELLED && newStatus != JOB_TASK_STATUS_DROPPING && - newStatus != JOB_TASK_STATUS_FAILED) { + if (newStatus != JOB_TASK_STATUS_DROPPING && newStatus != JOB_TASK_STATUS_FAILED) { QW_ERR_JRET(TSDB_CODE_QRY_APP_ERROR); } break; case JOB_TASK_STATUS_FAILED: - if (newStatus != JOB_TASK_STATUS_CANCELLED && newStatus != JOB_TASK_STATUS_DROPPING) { + if (newStatus != JOB_TASK_STATUS_DROPPING) { QW_ERR_JRET(TSDB_CODE_QRY_APP_ERROR); } break; - case JOB_TASK_STATUS_CANCELLING: - if (newStatus != JOB_TASK_STATUS_CANCELLED) { - QW_ERR_JRET(TSDB_CODE_QRY_APP_ERROR); - } - - break; - case JOB_TASK_STATUS_CANCELLED: case JOB_TASK_STATUS_DROPPING: if (newStatus != JOB_TASK_STATUS_FAILED && newStatus != JOB_TASK_STATUS_PARTIAL_SUCCEED) { QW_ERR_JRET(TSDB_CODE_QRY_APP_ERROR); diff --git a/source/libs/qworker/src/qworker.c b/source/libs/qworker/src/qworker.c index 57ebb89ed22c86c01b911a1e7a390e2bc485d07a..800cc4c6e59ff7da06f699ce6fb8294819873c14 100644 --- a/source/libs/qworker/src/qworker.c +++ b/source/libs/qworker/src/qworker.c @@ -614,6 +614,8 @@ int32_t qwProcessCQuery(QW_FPARAMS_DEF, SQWMsg *qwMsg) { QW_SET_EVENT_PROCESSED(ctx, QW_EVENT_FETCH); qwBuildAndSendFetchRsp(&qwMsg->connInfo, rsp, dataLen, code); + rsp = NULL; + QW_TASK_DLOG("fetch rsp send, handle:%p, code:%x - %s, dataLen:%d", qwMsg->connInfo.handle, code, tstrerror(code), dataLen); } else { @@ -633,7 +635,7 @@ int32_t qwProcessCQuery(QW_FPARAMS_DEF, SQWMsg *qwMsg) { rsp = NULL; qwMsg->connInfo = ctx->dataConnInfo; - qwBuildAndSendFetchRsp(&qwMsg->connInfo, rsp, 0, code); + qwBuildAndSendFetchRsp(&qwMsg->connInfo, NULL, 0, code); QW_TASK_DLOG("fetch rsp send, handle:%p, code:%x - %s, dataLen:%d", qwMsg->connInfo.handle, code, tstrerror(code), 0); } diff --git a/source/libs/scheduler/inc/schedulerInt.h b/source/libs/scheduler/inc/schedulerInt.h index 8875ebdf736662af357084eed98137791687c5b4..6b2570c5b70132032c7df84107aa2b294b79233a 100644 --- a/source/libs/scheduler/inc/schedulerInt.h +++ b/source/libs/scheduler/inc/schedulerInt.h @@ -48,6 +48,12 @@ enum { SCH_FETCH_CB, }; +typedef enum { + SCH_OP_NULL = 0, + SCH_OP_EXEC, + SCH_OP_FETCH, +} SCH_OP_TYPE; + typedef struct SSchTrans { void *pTrans; void *pHandle; @@ -188,11 +194,15 @@ typedef struct SSchTask { typedef struct SSchJobAttr { EExplainMode explainMode; - bool syncSchedule; bool queryJob; bool needFlowCtrl; } SSchJobAttr; +typedef struct { + int32_t op; + bool sync; +} SSchOpStatus; + typedef struct SSchJob { int64_t refId; uint64_t queryId; @@ -200,7 +210,7 @@ typedef struct SSchJob { int32_t levelNum; int32_t taskNum; SRequestConnInfo conn; - SArray *nodeList; // qnode/vnode list, SArray + SArray *nodeList; // qnode/vnode list, SArray SArray *levels; // starting from 0. SArray SNodeList *subPlans; // subplan pointer copied from DAG, no need to free it in scheduler @@ -217,8 +227,8 @@ typedef struct SSchJob { int8_t status; SQueryNodeAddr resNode; tsem_t rspSem; - int8_t userFetch; - int32_t remoteFetch; + SSchOpStatus opStatus; + bool *reqKilled; SSchTask *fetchTask; int32_t errCode; SRWLatch resLock; @@ -227,7 +237,6 @@ typedef struct SSchJob { int32_t resNumOfRows; SSchResInfo userRes; const char *sql; - int32_t userCb; SQueryProfileSummary summary; } SSchJob; @@ -285,6 +294,10 @@ extern SSchedulerMgmt schMgmt; #define SCH_GET_JOB_STATUS(job) atomic_load_8(&(job)->status) #define SCH_GET_JOB_STATUS_STR(job) jobTaskStatusStr(SCH_GET_JOB_STATUS(job)) +#define SCH_JOB_IN_SYNC_OP(job) ((job)->opStatus.op && (job)->opStatus.sync) +#define SCH_JOB_IN_ASYNC_EXEC_OP(job) (((job)->opStatus.op == SCH_OP_EXEC) && (!(job)->opStatus.sync)) +#define SCH_JOB_IN_ASYNC_FETCH_OP(job) (((job)->opStatus.op == SCH_OP_FETCH) && (!(job)->opStatus.sync)) + #define SCH_SET_JOB_NEED_FLOW_CTRL(_job) (_job)->attr.needFlowCtrl = true #define SCH_JOB_NEED_FLOW_CTRL(_job) ((_job)->attr.needFlowCtrl) #define SCH_TASK_NEED_FLOW_CTRL(_job, _task) (SCH_IS_DATA_SRC_QRY_TASK(_task) && SCH_JOB_NEED_FLOW_CTRL(_job) && SCH_IS_LEVEL_UNFINISHED((_task)->level)) @@ -356,7 +369,7 @@ int32_t schMakeBrokenLinkVal(SSchJob *pJob, SSchTask *pTask, SRpcBrokenlinkVal * int32_t schAppendTaskExecNode(SSchJob *pJob, SSchTask *pTask, SQueryNodeAddr *addr, int32_t execIdx); int32_t schExecStaticExplainJob(SSchedulerReq *pReq, int64_t *job, bool sync); int32_t schExecJobImpl(SSchedulerReq *pReq, int64_t *job, SQueryResult* pRes, bool sync); -int32_t schChkUpdateJobStatus(SSchJob *pJob, int8_t newStatus); +int32_t schUpdateJobStatus(SSchJob *pJob, int8_t newStatus); int32_t schCancelJob(SSchJob *pJob); int32_t schProcessOnJobDropped(SSchJob *pJob, int32_t errCode); uint64_t schGenTaskId(void); @@ -368,6 +381,8 @@ int32_t schAsyncFetchRows(SSchJob *pJob); int32_t schUpdateTaskHandle(SSchJob *pJob, SSchTask *pTask, bool dropExecNode, void *handle, int32_t execIdx); int32_t schProcessOnTaskStatusRsp(SQueryNodeEpId* pEpId, SArray* pStatusList); void schFreeSMsgSendInfo(SMsgSendInfo *msgSendInfo); +char* schGetOpStr(SCH_OP_TYPE type); +int32_t schBeginOperation(SSchJob *pJob, SCH_OP_TYPE type, bool sync); #ifdef __cplusplus diff --git a/source/libs/scheduler/src/schJob.c b/source/libs/scheduler/src/schJob.c index 31536f413dedc73299b4370adf62b92a196a23a2..679a244f31f4098fadd82ba203b17af955d6e3a5 100644 --- a/source/libs/scheduler/src/schJob.c +++ b/source/libs/scheduler/src/schJob.c @@ -21,9 +21,9 @@ #include "tref.h" #include "trpc.h" -FORCE_INLINE SSchJob *schAcquireJob(int64_t refId) { qDebug("acquire jobId:0x%"PRIx64, refId); return (SSchJob *)taosAcquireRef(schMgmt.jobRef, refId); } +FORCE_INLINE SSchJob *schAcquireJob(int64_t refId) { qDebug("sch acquire jobId:0x%"PRIx64, refId); return (SSchJob *)taosAcquireRef(schMgmt.jobRef, refId); } -FORCE_INLINE int32_t schReleaseJob(int64_t refId) { qDebug("release jobId:0x%"PRIx64, refId); return taosReleaseRef(schMgmt.jobRef, refId); } +FORCE_INLINE int32_t schReleaseJob(int64_t refId) { qDebug("sch release jobId:0x%"PRIx64, refId); return taosReleaseRef(schMgmt.jobRef, refId); } int32_t schInitTask(SSchJob *pJob, SSchTask *pTask, SSubplan *pPlan, SSchLevel *pLevel) { pTask->plan = pPlan; @@ -47,14 +47,14 @@ int32_t schInitJob(SSchedulerReq *pReq, SSchJob **pSchJob, SQueryResult* pRes, b int64_t refId = -1; SSchJob *pJob = taosMemoryCalloc(1, sizeof(SSchJob)); if (NULL == pJob) { - qError("QID:%" PRIx64 " calloc %d failed", pReq->pDag->queryId, (int32_t)sizeof(SSchJob)); + qError("QID:0x%" PRIx64 " calloc %d failed", pReq->pDag->queryId, (int32_t)sizeof(SSchJob)); SCH_ERR_RET(TSDB_CODE_QRY_OUT_OF_MEMORY); } pJob->attr.explainMode = pReq->pDag->explainInfo.mode; - pJob->attr.syncSchedule = syncSchedule; pJob->conn = *pReq->pConn; pJob->sql = pReq->sql; + pJob->reqKilled = pReq->reqKilled; pJob->userRes.queryRes = pRes; pJob->userRes.execFp = pReq->fp; pJob->userRes.userParam = pReq->cbParam; @@ -108,7 +108,7 @@ int32_t schInitJob(SSchedulerReq *pReq, SSchJob **pSchJob, SQueryResult* pRes, b atomic_add_fetch_32(&schMgmt.jobNum, 1); if (NULL == schAcquireJob(refId)) { - SCH_JOB_ELOG("schAcquireJob job failed, refId:%" PRIx64, refId); + SCH_JOB_ELOG("schAcquireJob job failed, refId:0x%" PRIx64, refId); SCH_ERR_JRET(TSDB_CODE_SCH_STATUS_ERROR); } @@ -116,7 +116,7 @@ int32_t schInitJob(SSchedulerReq *pReq, SSchJob **pSchJob, SQueryResult* pRes, b SCH_JOB_DLOG("job refId:0x%" PRIx64" created", pJob->refId); - pJob->status = JOB_TASK_STATUS_NOT_START; + schUpdateJobStatus(pJob, JOB_TASK_STATUS_NOT_START); *pSchJob = pJob; @@ -155,18 +155,57 @@ void schFreeTask(SSchJob *pJob, SSchTask *pTask) { } } + +void schUpdateJobErrCode(SSchJob *pJob, int32_t errCode) { + if (TSDB_CODE_SUCCESS == errCode) { + return; + } + + int32_t origCode = atomic_load_32(&pJob->errCode); + if (TSDB_CODE_SUCCESS == origCode) { + if (origCode == atomic_val_compare_exchange_32(&pJob->errCode, origCode, errCode)) { + goto _return; + } + + origCode = atomic_load_32(&pJob->errCode); + } + + if (NEED_CLIENT_HANDLE_ERROR(origCode)) { + return; + } + + if (NEED_CLIENT_HANDLE_ERROR(errCode)) { + atomic_store_32(&pJob->errCode, errCode); + goto _return; + } + + return; + +_return: + + SCH_JOB_DLOG("job errCode updated to %x - %s", errCode, tstrerror(errCode)); +} + + + FORCE_INLINE bool schJobNeedToStop(SSchJob *pJob, int8_t *pStatus) { int8_t status = SCH_GET_JOB_STATUS(pJob); if (pStatus) { *pStatus = status; } - return (status == JOB_TASK_STATUS_FAILED || status == JOB_TASK_STATUS_CANCELLED || - status == JOB_TASK_STATUS_CANCELLING || status == JOB_TASK_STATUS_DROPPING || + if (*pJob->reqKilled) { + schUpdateJobStatus(pJob, JOB_TASK_STATUS_DROPPING); + schUpdateJobErrCode(pJob, TSDB_CODE_TSC_QUERY_KILLED); + + return true; + } + + return (status == JOB_TASK_STATUS_FAILED || status == JOB_TASK_STATUS_DROPPING || status == JOB_TASK_STATUS_SUCCEED); } -int32_t schChkUpdateJobStatus(SSchJob *pJob, int8_t newStatus) { +int32_t schUpdateJobStatus(SSchJob *pJob, int8_t newStatus) { int32_t code = 0; int8_t oriStatus = 0; @@ -175,7 +214,11 @@ int32_t schChkUpdateJobStatus(SSchJob *pJob, int8_t newStatus) { oriStatus = SCH_GET_JOB_STATUS(pJob); if (oriStatus == newStatus) { - SCH_ERR_JRET(TSDB_CODE_QRY_APP_ERROR); + if (newStatus == JOB_TASK_STATUS_DROPPING) { + SCH_ERR_JRET(TSDB_CODE_SCH_JOB_IS_DROPPING); + } + + SCH_ERR_JRET(TSDB_CODE_SCH_IGNORE_ERROR); } switch (oriStatus) { @@ -186,14 +229,13 @@ int32_t schChkUpdateJobStatus(SSchJob *pJob, int8_t newStatus) { break; case JOB_TASK_STATUS_NOT_START: - if (newStatus != JOB_TASK_STATUS_EXECUTING) { + if (newStatus != JOB_TASK_STATUS_EXECUTING && newStatus != JOB_TASK_STATUS_DROPPING) { SCH_ERR_JRET(TSDB_CODE_QRY_APP_ERROR); } break; case JOB_TASK_STATUS_EXECUTING: if (newStatus != JOB_TASK_STATUS_PARTIAL_SUCCEED && newStatus != JOB_TASK_STATUS_FAILED && - newStatus != JOB_TASK_STATUS_CANCELLING && newStatus != JOB_TASK_STATUS_CANCELLED && newStatus != JOB_TASK_STATUS_DROPPING) { SCH_ERR_JRET(TSDB_CODE_QRY_APP_ERROR); } @@ -208,13 +250,11 @@ int32_t schChkUpdateJobStatus(SSchJob *pJob, int8_t newStatus) { break; case JOB_TASK_STATUS_SUCCEED: case JOB_TASK_STATUS_FAILED: - case JOB_TASK_STATUS_CANCELLING: if (newStatus != JOB_TASK_STATUS_DROPPING) { SCH_ERR_JRET(TSDB_CODE_QRY_APP_ERROR); } break; - case JOB_TASK_STATUS_CANCELLED: case JOB_TASK_STATUS_DROPPING: SCH_ERR_JRET(TSDB_CODE_QRY_JOB_FREED); break; @@ -238,8 +278,67 @@ int32_t schChkUpdateJobStatus(SSchJob *pJob, int8_t newStatus) { _return: SCH_JOB_ELOG("invalid job status update, from %s to %s", jobTaskStatusStr(oriStatus), jobTaskStatusStr(newStatus)); - SCH_ERR_RET(code); + SCH_RET(code); +} + + +void schEndOperation(SSchJob *pJob) { + int32_t op = atomic_load_32(&pJob->opStatus.op); + if (SCH_OP_NULL == op) { + SCH_JOB_DLOG("job already not in any operation, status:%s", jobTaskStatusStr(pJob->status)); + return; + } + + atomic_store_32(&pJob->opStatus.op, SCH_OP_NULL); + + SCH_JOB_DLOG("job end %s operation", schGetOpStr(op)); +} + +int32_t schBeginOperation(SSchJob *pJob, SCH_OP_TYPE type, bool sync) { + int32_t code = 0; + int8_t status = 0; + + if (schJobNeedToStop(pJob, &status)) { + SCH_JOB_ELOG("abort op %s cause of job need to stop", schGetOpStr(type)); + SCH_ERR_JRET(pJob->errCode); + } + + if (SCH_OP_NULL != atomic_val_compare_exchange_32(&pJob->opStatus.op, SCH_OP_NULL, type)) { + SCH_JOB_ELOG("job already in %s operation", schGetOpStr(pJob->opStatus.op)); + SCH_ERR_JRET(TSDB_CODE_TSC_APP_ERROR); + } + + SCH_JOB_DLOG("job start %s operation", schGetOpStr(pJob->opStatus.op)); + + pJob->opStatus.sync = sync; + + switch (type) { + case SCH_OP_EXEC: + SCH_ERR_JRET(schUpdateJobStatus(pJob, JOB_TASK_STATUS_EXECUTING)); + break; + case SCH_OP_FETCH: + if (!SCH_JOB_NEED_FETCH(pJob)) { + SCH_JOB_ELOG("no need to fetch data, status:%s", SCH_GET_JOB_STATUS_STR(pJob)); + SCH_ERR_JRET(TSDB_CODE_QRY_APP_ERROR); + } + + if (status != JOB_TASK_STATUS_PARTIAL_SUCCEED) { + SCH_JOB_ELOG("job status error for fetch, status:%s", jobTaskStatusStr(status)); + SCH_ERR_JRET(TSDB_CODE_SCH_STATUS_ERROR); + } + break; + default: + SCH_JOB_ELOG("unknown operation type %d", type); + SCH_ERR_JRET(TSDB_CODE_TSC_APP_ERROR); + } + return TSDB_CODE_SUCCESS; + +_return: + + schEndOperation(pJob); + + SCH_RET(code); } int32_t schBuildTaskRalation(SSchJob *pJob, SHashObj *planToTask) { @@ -278,7 +377,7 @@ int32_t schBuildTaskRalation(SSchJob *pJob, SHashObj *planToTask) { SCH_ERR_RET(TSDB_CODE_QRY_OUT_OF_MEMORY); } - SCH_TASK_DLOG("children info, the %d child TID %" PRIx64, n, (*childTask)->taskId); + SCH_TASK_DLOG("children info, the %d child TID 0x%" PRIx64, n, (*childTask)->taskId); } if (parentNum > 0) { @@ -312,7 +411,7 @@ int32_t schBuildTaskRalation(SSchJob *pJob, SHashObj *planToTask) { SCH_ERR_RET(TSDB_CODE_QRY_OUT_OF_MEMORY); } - SCH_TASK_DLOG("parents info, the %d parent TID %" PRIx64, n, (*parentTask)->taskId); + SCH_TASK_DLOG("parents info, the %d parent TID 0x%" PRIx64, n, (*parentTask)->taskId); } SCH_TASK_DLOG("level:%d, parentNum:%d, childNum:%d", i, parentNum, childNum); @@ -515,7 +614,7 @@ int32_t schValidateAndBuildJob(SQueryPlan *pDag, SSchJob *pJob) { ++pJob->taskNum; } - SCH_JOB_DLOG("level initialized, taskNum:%d", taskNum); + SCH_JOB_DLOG("level %d initialized, taskNum:%d", i, taskNum); } SCH_ERR_JRET(schBuildTaskRalation(pJob, planToTask)); @@ -537,8 +636,9 @@ int32_t schSetAddrsFromNodeList(SSchJob *pJob, SSchTask *pTask) { nodeNum = taosArrayGetSize(pJob->nodeList); for (int32_t i = 0; i < nodeNum && addNum < SCH_MAX_CANDIDATE_EP_NUM; ++i) { - SQueryNodeAddr *naddr = taosArrayGet(pJob->nodeList, i); - + SQueryNodeLoad *nload = taosArrayGet(pJob->nodeList, i); + SQueryNodeAddr *naddr = &nload->addr; + if (NULL == taosArrayPush(pTask->candidateAddrs, naddr)) { SCH_TASK_ELOG("taosArrayPush execNode to candidate addrs failed, addNum:%d, errno:%d", addNum, errno); SCH_ERR_RET(TSDB_CODE_QRY_OUT_OF_MEMORY); @@ -785,37 +885,6 @@ int32_t schHandleTaskRetry(SSchJob *pJob, SSchTask *pTask) { return TSDB_CODE_SUCCESS; } -void schUpdateJobErrCode(SSchJob *pJob, int32_t errCode) { - if (TSDB_CODE_SUCCESS == errCode) { - return; - } - - int32_t origCode = atomic_load_32(&pJob->errCode); - if (TSDB_CODE_SUCCESS == origCode) { - if (origCode == atomic_val_compare_exchange_32(&pJob->errCode, origCode, errCode)) { - goto _return; - } - - origCode = atomic_load_32(&pJob->errCode); - } - - if (NEED_CLIENT_HANDLE_ERROR(origCode)) { - return; - } - - if (NEED_CLIENT_HANDLE_ERROR(errCode)) { - atomic_store_32(&pJob->errCode, errCode); - goto _return; - } - - return; - -_return: - - SCH_JOB_DLOG("job errCode updated to %x - %s", errCode, tstrerror(errCode)); -} - - int32_t schSetJobQueryRes(SSchJob* pJob, SQueryResult* pRes) { pRes->code = atomic_load_32(&pJob->errCode); pRes->numOfRows = pJob->resNumOfRows; @@ -828,7 +897,7 @@ int32_t schSetJobQueryRes(SSchJob* pJob, SQueryResult* pRes) { int32_t schSetJobFetchRes(SSchJob* pJob, void** pData) { int32_t code = 0; if (pJob->resData && ((SRetrieveTableRsp *)pJob->resData)->completed) { - SCH_ERR_RET(schChkUpdateJobStatus(pJob, JOB_TASK_STATUS_SUCCEED)); + SCH_ERR_RET(schUpdateJobStatus(pJob, JOB_TASK_STATUS_SUCCEED)); } while (true) { @@ -855,15 +924,17 @@ int32_t schSetJobFetchRes(SSchJob* pJob, void** pData) { return TSDB_CODE_SUCCESS; } -int32_t schNotifyUserQueryRes(SSchJob* pJob) { - pJob->userRes.queryRes = taosMemoryCalloc(1, sizeof(*pJob->userRes.queryRes)); - if (pJob->userRes.queryRes) { - schSetJobQueryRes(pJob, pJob->userRes.queryRes); +int32_t schNotifyUserExecRes(SSchJob* pJob) { + SQueryResult* pRes = taosMemoryCalloc(1, sizeof(SQueryResult)); + if (pRes) { + schSetJobQueryRes(pJob, pRes); } - (*pJob->userRes.execFp)(pJob->userRes.queryRes, pJob->userRes.userParam, atomic_load_32(&pJob->errCode)); + schEndOperation(pJob); - pJob->userRes.queryRes = NULL; + SCH_JOB_DLOG("sch start to invoke exec cb, code: %s", tstrerror(pJob->errCode)); + (*pJob->userRes.execFp)(pRes, pJob->userRes.userParam, atomic_load_32(&pJob->errCode)); + SCH_JOB_DLOG("sch end from query cb, code: %s", tstrerror(pJob->errCode)); return TSDB_CODE_SUCCESS; } @@ -871,36 +942,52 @@ int32_t schNotifyUserQueryRes(SSchJob* pJob) { int32_t schNotifyUserFetchRes(SSchJob* pJob) { void* pRes = NULL; - SCH_ERR_RET(schSetJobFetchRes(pJob, &pRes)); + schSetJobFetchRes(pJob, &pRes); + + schEndOperation(pJob); + SCH_JOB_DLOG("sch start to invoke fetch cb, code: %s", tstrerror(pJob->errCode)); (*pJob->userRes.fetchFp)(pRes, pJob->userRes.userParam, atomic_load_32(&pJob->errCode)); + SCH_JOB_DLOG("sch end from fetch cb, code: %s", tstrerror(pJob->errCode)); return TSDB_CODE_SUCCESS; } +void schPostJobRes(SSchJob *pJob, SCH_OP_TYPE op) { + if (SCH_OP_NULL == pJob->opStatus.op) { + SCH_JOB_DLOG("job not in any op, no need to post job res, status:%s", jobTaskStatusStr(pJob->status)); + return; + } + + if (op && pJob->opStatus.op != op) { + SCH_JOB_ELOG("job in op %s mis-match with expected %s", schGetOpStr(pJob->opStatus.op), schGetOpStr(op)); + return; + } + + if (SCH_JOB_IN_SYNC_OP(pJob)) { + tsem_post(&pJob->rspSem); + } else if (SCH_JOB_IN_ASYNC_EXEC_OP(pJob)) { + schNotifyUserExecRes(pJob); + } else if (SCH_JOB_IN_ASYNC_FETCH_OP(pJob)) { + schNotifyUserFetchRes(pJob); + } else { + SCH_JOB_ELOG("job not in any operation, status:%s", jobTaskStatusStr(pJob->status)); + } +} + int32_t schProcessOnJobFailureImpl(SSchJob *pJob, int32_t status, int32_t errCode) { // if already FAILED, no more processing - SCH_ERR_RET(schChkUpdateJobStatus(pJob, status)); + SCH_ERR_RET(schUpdateJobStatus(pJob, status)); schUpdateJobErrCode(pJob, errCode); - - if (atomic_load_8(&pJob->userFetch) || pJob->attr.syncSchedule) { - tsem_post(&pJob->rspSem); - } - + int32_t code = atomic_load_32(&pJob->errCode); - - SCH_JOB_DLOG("job failed with error: %s", tstrerror(code)); - - if (!pJob->attr.syncSchedule) { - if (SCH_EXEC_CB == atomic_val_compare_exchange_32(&pJob->userCb, SCH_EXEC_CB, 0)) { - schNotifyUserQueryRes(pJob); - } else if (SCH_FETCH_CB == atomic_val_compare_exchange_32(&pJob->userCb, SCH_FETCH_CB, 0)) { - atomic_val_compare_exchange_8(&pJob->userFetch, 1, 0); - schNotifyUserFetchRes(pJob); - } + if (code) { + SCH_JOB_DLOG("job failed with error: %s", tstrerror(code)); } + schPostJobRes(pJob, 0); + SCH_RET(code); } @@ -918,20 +1005,9 @@ int32_t schProcessOnJobDropped(SSchJob *pJob, int32_t errCode) { int32_t schProcessOnJobPartialSuccess(SSchJob *pJob) { int32_t code = 0; - SCH_ERR_RET(schChkUpdateJobStatus(pJob, JOB_TASK_STATUS_PARTIAL_SUCCEED)); - - if (pJob->attr.syncSchedule) { - tsem_post(&pJob->rspSem); - } else if (SCH_EXEC_CB == atomic_val_compare_exchange_32(&pJob->userCb, SCH_EXEC_CB, 0)) { - schNotifyUserQueryRes(pJob); - } else if (SCH_FETCH_CB == atomic_val_compare_exchange_32(&pJob->userCb, SCH_FETCH_CB, 0)) { - atomic_val_compare_exchange_8(&pJob->userFetch, 1, 0); - schNotifyUserFetchRes(pJob); - } + SCH_ERR_RET(schUpdateJobStatus(pJob, JOB_TASK_STATUS_PARTIAL_SUCCEED)); - if (atomic_load_8(&pJob->userFetch)) { - SCH_ERR_JRET(schFetchFromRemote(pJob)); - } + schPostJobRes(pJob, SCH_OP_EXEC); return TSDB_CODE_SUCCESS; @@ -940,16 +1016,8 @@ _return: SCH_RET(schProcessOnJobFailure(pJob, code)); } -void schProcessOnDataFetched(SSchJob *job) { - atomic_val_compare_exchange_32(&job->remoteFetch, 1, 0); - - if (job->attr.syncSchedule) { - tsem_post(&job->rspSem); - } else if (SCH_FETCH_CB == atomic_val_compare_exchange_32(&job->userCb, SCH_FETCH_CB, 0)) { - atomic_val_compare_exchange_8(&job->userFetch, 1, 0); - - schNotifyUserFetchRes(job); - } +void schProcessOnDataFetched(SSchJob *pJob) { + schPostJobRes(pJob, SCH_OP_FETCH); } // Note: no more task error processing, handled in function internal @@ -1109,7 +1177,7 @@ int32_t schProcessOnTaskSuccess(SSchJob *pJob, SSchTask *pTask) { SCH_UNLOCK(SCH_WRITE, &parent->lock); if (SCH_TASK_READY_FOR_LAUNCH(readyNum, parent)) { - SCH_TASK_DLOG("all %d children task done, start to launch parent task %" PRIx64, readyNum, parent->taskId); + SCH_TASK_DLOG("all %d children task done, start to launch parent task 0x%" PRIx64, readyNum, parent->taskId); SCH_ERR_RET(schLaunchTask(pJob, parent)); } } @@ -1127,15 +1195,8 @@ _return: int32_t schFetchFromRemote(SSchJob *pJob) { int32_t code = 0; - if (atomic_val_compare_exchange_32(&pJob->remoteFetch, 0, 1) != 0) { - SCH_JOB_ELOG("prior fetching not finished, remoteFetch:%d", atomic_load_32(&pJob->remoteFetch)); - return TSDB_CODE_SUCCESS; - } - void *resData = atomic_load_ptr(&pJob->resData); if (resData) { - atomic_val_compare_exchange_32(&pJob->remoteFetch, 1, 0); - SCH_JOB_DLOG("res already fetched, res:%p", resData); return TSDB_CODE_SUCCESS; } @@ -1146,8 +1207,6 @@ int32_t schFetchFromRemote(SSchJob *pJob) { _return: - atomic_val_compare_exchange_32(&pJob->remoteFetch, 1, 0); - SCH_RET(schProcessOnTaskFailure(pJob, pJob->fetchTask, code)); } @@ -1291,7 +1350,7 @@ int32_t schGetTaskFromList(SHashObj *pTaskList, uint64_t taskId, SSchTask **pTas int32_t schGetTaskInJob(SSchJob *pJob, uint64_t taskId, SSchTask **pTask) { schGetTaskFromList(pJob->taskList, taskId, pTask); if (NULL == *pTask) { - SCH_JOB_ELOG("task not found in job task list, taskId:%" PRIx64, taskId); + SCH_JOB_ELOG("task not found in job task list, taskId:0x%" PRIx64, taskId); SCH_ERR_RET(TSDB_CODE_SCH_INTERNAL_ERROR); } @@ -1382,8 +1441,6 @@ int32_t schLaunchLevelTasks(SSchJob *pJob, SSchLevel *level) { int32_t schLaunchJob(SSchJob *pJob) { SSchLevel *level = taosArrayGet(pJob->levels, pJob->levelIdx); - SCH_ERR_RET(schChkUpdateJobStatus(pJob, JOB_TASK_STATUS_EXECUTING)); - SCH_ERR_RET(schChkJobNeedFlowCtrl(pJob, level)); SCH_ERR_RET(schLaunchLevelTasks(pJob, level)); @@ -1466,9 +1523,9 @@ void schFreeJobImpl(void *job) { taosMemoryFreeClear(pJob->userRes.queryRes); taosMemoryFreeClear(pJob->resData); - taosMemoryFreeClear(pJob); + taosMemoryFree(pJob); - qDebug("QID:0x%" PRIx64 " job freed, refId:%" PRIx64 ", pointer:%p", queryId, refId, pJob); + qDebug("QID:0x%" PRIx64 " sch job freed, refId:0x%" PRIx64 ", pointer:%p", queryId, refId, pJob); int32_t jobNum = atomic_sub_fetch_32(&schMgmt.jobNum, 1); if (jobNum == 0) { @@ -1483,26 +1540,36 @@ int32_t schExecJobImpl(SSchedulerReq *pReq, int64_t *job, SQueryResult* pRes, bo int32_t code = 0; SSchJob *pJob = NULL; - SCH_ERR_RET(schInitJob(pReq, &pJob, pRes, sync)); + SCH_ERR_JRET(schInitJob(pReq, &pJob, pRes, sync)); - qDebug("QID:0x%" PRIx64 " job refId 0x%"PRIx64 " started", pReq->pDag->queryId, pJob->refId); + qDebug("QID:0x%" PRIx64 " sch job refId 0x%"PRIx64 " started", pReq->pDag->queryId, pJob->refId); *job = pJob->refId; - if (!sync) { - pJob->userCb = SCH_EXEC_CB; - } + SCH_ERR_JRET(schBeginOperation(pJob, SCH_OP_EXEC, sync)); - SCH_ERR_JRET(schLaunchJob(pJob)); + code = schLaunchJob(pJob); if (sync) { SCH_JOB_DLOG("will wait for rsp now, job status:%s", SCH_GET_JOB_STATUS_STR(pJob)); tsem_wait(&pJob->rspSem); + + schEndOperation(pJob); + } else if (code) { + schPostJobRes(pJob, SCH_OP_EXEC); } - SCH_JOB_DLOG("job exec done, job status:%s, jobId:0x%"PRIx64, SCH_GET_JOB_STATUS_STR(pJob), pJob->refId); + SCH_JOB_DLOG("job exec done, job status:%s, jobId:0x%" PRIx64, SCH_GET_JOB_STATUS_STR(pJob), pJob->refId); + + schReleaseJob(pJob->refId); + + SCH_RET(code); _return: + if (!sync) { + pReq->fp(NULL, pReq->cbParam, code); + } + schReleaseJob(pJob->refId); SCH_RET(code); @@ -1536,10 +1603,10 @@ int32_t schAsyncExecJob(SSchedulerReq *pReq, int64_t *pJob) { *pJob = 0; if (EXPLAIN_MODE_STATIC == pReq->pDag->explainInfo.mode) { - SCH_ERR_RET(schExecStaticExplainJob(pReq, pJob, false)); - } else { - SCH_ERR_RET(schExecJobImpl(pReq, pJob, NULL, false)); + SCH_RET(schExecStaticExplainJob(pReq, pJob, false)); } + + SCH_ERR_RET(schExecJobImpl(pReq, pJob, NULL, false)); return code; } @@ -1550,19 +1617,29 @@ int32_t schExecStaticExplainJob(SSchedulerReq *pReq, int64_t *job, bool sync) { int32_t code = 0; SSchJob *pJob = taosMemoryCalloc(1, sizeof(SSchJob)); if (NULL == pJob) { - qError("QID:%" PRIx64 " calloc %d failed", pReq->pDag->queryId, (int32_t)sizeof(SSchJob)); - SCH_ERR_RET(TSDB_CODE_QRY_OUT_OF_MEMORY); + qError("QID:0x%" PRIx64 " calloc %d failed", pReq->pDag->queryId, (int32_t)sizeof(SSchJob)); + code = TSDB_CODE_QRY_OUT_OF_MEMORY; + pReq->fp(NULL, pReq->cbParam, code); + SCH_ERR_RET(code); } pJob->sql = pReq->sql; + pJob->reqKilled = pReq->reqKilled; pJob->attr.queryJob = true; - pJob->attr.syncSchedule = sync; pJob->attr.explainMode = pReq->pDag->explainInfo.mode; pJob->queryId = pReq->pDag->queryId; pJob->subPlans = pReq->pDag->pSubplans; pJob->userRes.execFp = pReq->fp; pJob->userRes.userParam = pReq->cbParam; + schUpdateJobStatus(pJob, JOB_TASK_STATUS_NOT_START); + + code = schBeginOperation(pJob, SCH_OP_EXEC, sync); + if (code) { + pReq->fp(NULL, pReq->cbParam, code); + SCH_ERR_RET(code); + } + SCH_ERR_JRET(qExecStaticExplain(pReq->pDag, (SRetrieveTableRsp **)&pJob->resData)); int64_t refId = taosAddRef(schMgmt.jobRef, pJob); @@ -1572,21 +1649,23 @@ int32_t schExecStaticExplainJob(SSchedulerReq *pReq, int64_t *job, bool sync) { } if (NULL == schAcquireJob(refId)) { - SCH_JOB_ELOG("schAcquireJob job failed, refId:%" PRIx64, refId); - SCH_ERR_RET(TSDB_CODE_SCH_STATUS_ERROR); + SCH_JOB_ELOG("schAcquireJob job failed, refId:0x%" PRIx64, refId); + SCH_ERR_JRET(TSDB_CODE_SCH_STATUS_ERROR); } pJob->refId = refId; - SCH_JOB_DLOG("job refId:%" PRIx64, pJob->refId); + SCH_JOB_DLOG("job refId:0x%" PRIx64, pJob->refId); pJob->status = JOB_TASK_STATUS_PARTIAL_SUCCEED; *job = pJob->refId; SCH_JOB_DLOG("job exec done, job status:%s", SCH_GET_JOB_STATUS_STR(pJob)); - - if (!pJob->attr.syncSchedule) { - code = schNotifyUserQueryRes(pJob); + + if (!sync) { + schPostJobRes(pJob, SCH_OP_EXEC); + } else { + schEndOperation(pJob); } schReleaseJob(pJob->refId); @@ -1595,56 +1674,29 @@ int32_t schExecStaticExplainJob(SSchedulerReq *pReq, int64_t *job, bool sync) { _return: + schEndOperation(pJob); + if (!sync) { + pReq->fp(NULL, pReq->cbParam, code); + } + schFreeJobImpl(pJob); + SCH_RET(code); } int32_t schFetchRows(SSchJob *pJob) { int32_t code = 0; - int8_t status = SCH_GET_JOB_STATUS(pJob); - if (status == JOB_TASK_STATUS_DROPPING) { - SCH_JOB_ELOG("job is dropping, status:%s", jobTaskStatusStr(status)); - SCH_ERR_RET(TSDB_CODE_SCH_STATUS_ERROR); - } - - if (!SCH_JOB_NEED_FETCH(pJob)) { - SCH_JOB_ELOG("no need to fetch data, status:%s", SCH_GET_JOB_STATUS_STR(pJob)); - SCH_ERR_RET(TSDB_CODE_QRY_APP_ERROR); - } - - if (atomic_val_compare_exchange_8(&pJob->userFetch, 0, 1) != 0) { - SCH_JOB_ELOG("prior fetching not finished, userFetch:%d", atomic_load_8(&pJob->userFetch)); - SCH_ERR_RET(TSDB_CODE_QRY_APP_ERROR); - } - - if (JOB_TASK_STATUS_FAILED == status || JOB_TASK_STATUS_DROPPING == status) { - SCH_JOB_ELOG("job failed or dropping, status:%s", jobTaskStatusStr(status)); - SCH_ERR_JRET(atomic_load_32(&pJob->errCode)); - } else if (status == JOB_TASK_STATUS_SUCCEED) { - SCH_JOB_DLOG("job already succeed, status:%s", jobTaskStatusStr(status)); - goto _return; - } else if (status != JOB_TASK_STATUS_PARTIAL_SUCCEED) { - SCH_JOB_ELOG("job status error for fetch, status:%s", jobTaskStatusStr(status)); - SCH_ERR_JRET(TSDB_CODE_SCH_STATUS_ERROR); - } - if (!(pJob->attr.explainMode == EXPLAIN_MODE_STATIC)) { SCH_ERR_JRET(schFetchFromRemote(pJob)); tsem_wait(&pJob->rspSem); - - status = SCH_GET_JOB_STATUS(pJob); - if (JOB_TASK_STATUS_FAILED == status || JOB_TASK_STATUS_DROPPING == status) { - SCH_JOB_ELOG("job failed or dropping, status:%s", jobTaskStatusStr(status)); - SCH_ERR_JRET(atomic_load_32(&pJob->errCode)); - } } SCH_ERR_JRET(schSetJobFetchRes(pJob, pJob->userRes.fetchRes)); _return: - atomic_val_compare_exchange_8(&pJob->userFetch, 1, 0); + schEndOperation(pJob); SCH_RET(code); } @@ -1652,50 +1704,14 @@ _return: int32_t schAsyncFetchRows(SSchJob *pJob) { int32_t code = 0; - int8_t status = SCH_GET_JOB_STATUS(pJob); - if (status == JOB_TASK_STATUS_DROPPING) { - SCH_JOB_ELOG("job is dropping, status:%s", jobTaskStatusStr(status)); - SCH_ERR_RET(TSDB_CODE_SCH_STATUS_ERROR); - } - - if (!SCH_JOB_NEED_FETCH(pJob)) { - SCH_JOB_ELOG("no need to fetch data, status:%s", SCH_GET_JOB_STATUS_STR(pJob)); - SCH_ERR_RET(TSDB_CODE_QRY_APP_ERROR); - } - - if (atomic_val_compare_exchange_8(&pJob->userFetch, 0, 1) != 0) { - SCH_JOB_ELOG("prior fetching not finished, userFetch:%d", atomic_load_8(&pJob->userFetch)); - SCH_ERR_RET(TSDB_CODE_QRY_APP_ERROR); - } - - if (JOB_TASK_STATUS_FAILED == status || JOB_TASK_STATUS_DROPPING == status) { - SCH_JOB_ELOG("job failed or dropping, status:%s", jobTaskStatusStr(status)); - SCH_ERR_JRET(atomic_load_32(&pJob->errCode)); - } else if (status == JOB_TASK_STATUS_SUCCEED) { - SCH_JOB_DLOG("job already succeed, status:%s", jobTaskStatusStr(status)); - goto _return; - } else if (status != JOB_TASK_STATUS_PARTIAL_SUCCEED) { - SCH_JOB_ELOG("job status error for fetch, status:%s", jobTaskStatusStr(status)); - SCH_ERR_JRET(TSDB_CODE_SCH_STATUS_ERROR); - } - if (pJob->attr.explainMode == EXPLAIN_MODE_STATIC) { - atomic_val_compare_exchange_8(&pJob->userFetch, 1, 0); - - SCH_ERR_JRET(schNotifyUserFetchRes(pJob)); - } else { - pJob->userCb = SCH_FETCH_CB; - - SCH_ERR_JRET(schFetchFromRemote(pJob)); + schPostJobRes(pJob, SCH_OP_FETCH); + return TSDB_CODE_SUCCESS; } + + SCH_ERR_RET(schFetchFromRemote(pJob)); return TSDB_CODE_SUCCESS; - -_return: - - atomic_val_compare_exchange_8(&pJob->userFetch, 1, 0); - - SCH_RET(code); } diff --git a/source/libs/scheduler/src/schRemote.c b/source/libs/scheduler/src/schRemote.c index ddcb53d62f9d342d9bc0ef266163b53db1911108..0bd747785cb162ed6ad10b7c7fc6f5f77ab28d9e 100644 --- a/source/libs/scheduler/src/schRemote.c +++ b/source/libs/scheduler/src/schRemote.c @@ -315,8 +315,6 @@ int32_t schHandleResponseMsg(SSchJob *pJob, SSchTask *pTask, int32_t msgType, ch return TSDB_CODE_SUCCESS; } - atomic_val_compare_exchange_32(&pJob->remoteFetch, 1, 0); - SCH_ERR_JRET(schFetchFromRemote(pJob)); taosMemoryFreeClear(msg); @@ -346,7 +344,7 @@ int32_t schHandleResponseMsg(SSchJob *pJob, SSchTask *pTask, int32_t msgType, ch } case TDMT_VND_DROP_TASK_RSP: { // SHOULD NEVER REACH HERE - SCH_TASK_ELOG("invalid status to handle drop task rsp, refId:%" PRIx64, pJob->refId); + SCH_TASK_ELOG("invalid status to handle drop task rsp, refId:0x%" PRIx64, pJob->refId); SCH_ERR_JRET(TSDB_CODE_SCH_INTERNAL_ERROR); break; } @@ -376,7 +374,7 @@ int32_t schHandleCallback(void *param, const SDataBuf *pMsg, int32_t msgType, in SSchJob *pJob = schAcquireJob(pParam->refId); if (NULL == pJob) { - qWarn("QID:0x%" PRIx64 ",TID:0x%" PRIx64 "taosAcquireRef job failed, may be dropped, refId:%" PRIx64, + qWarn("QID:0x%" PRIx64 ",TID:0x%" PRIx64 "taosAcquireRef job failed, may be dropped, refId:0x%" PRIx64, pParam->queryId, pParam->taskId, pParam->refId); SCH_ERR_JRET(TSDB_CODE_QRY_JOB_FREED); } @@ -445,7 +443,7 @@ int32_t schHandleExplainCallback(void *param, const SDataBuf *pMsg, int32_t code int32_t schHandleDropCallback(void *param, const SDataBuf *pMsg, int32_t code) { SSchTaskCallbackParam *pParam = (SSchTaskCallbackParam *)param; - qDebug("QID:%" PRIx64 ",TID:%" PRIx64 " drop task rsp received, code:%x", pParam->queryId, pParam->taskId, code); + qDebug("QID:0x%" PRIx64 ",TID:0x%" PRIx64 " drop task rsp received, code:0x%x", pParam->queryId, pParam->taskId, code); taosMemoryFreeClear(param); return TSDB_CODE_SUCCESS; } diff --git a/source/libs/scheduler/src/schUtil.c b/source/libs/scheduler/src/schUtil.c index 66483187daa52cc28b54fd3a2a1317c705e9d483..73077cbf0f6632e0a7a7b9979e0b6d17baf90fd3 100644 --- a/source/libs/scheduler/src/schUtil.c +++ b/source/libs/scheduler/src/schUtil.c @@ -21,6 +21,18 @@ #include "tref.h" #include "trpc.h" +char* schGetOpStr(SCH_OP_TYPE type) { + switch (type) { + case SCH_OP_NULL: + return "NULL"; + case SCH_OP_EXEC: + return "EXEC"; + case SCH_OP_FETCH: + return "FETCH"; + default: + return "UNKNOWN"; + } +} void schCleanClusterHb(void* pTrans) { SCH_LOCK(SCH_WRITE, &schMgmt.hbLock); @@ -188,7 +200,7 @@ int32_t schUpdateHbConnection(SQueryNodeEpId *epId, SSchTrans *trans) { SCH_UNLOCK(SCH_WRITE, &hb->lock); SCH_UNLOCK(SCH_READ, &schMgmt.hbLock); - qDebug("hb connection updated, sId:%" PRIx64 ", nodeId:%d, fqdn:%s, port:%d, pTrans:%p, pHandle:%p", schMgmt.sId, + qDebug("hb connection updated, sId:0x%" PRIx64 ", nodeId:%d, fqdn:%s, port:%d, pTrans:%p, pHandle:%p", schMgmt.sId, epId->nodeId, epId->ep.fqdn, epId->ep.port, trans->pTrans, trans->pHandle); return TSDB_CODE_SUCCESS; diff --git a/source/libs/scheduler/src/scheduler.c b/source/libs/scheduler/src/scheduler.c index 0eaeeae9cb75edbe5c3b974af4acb61eee6e713d..57a405ffa3ffd524e7203a2935a4135ec3bd5208 100644 --- a/source/libs/scheduler/src/scheduler.c +++ b/source/libs/scheduler/src/scheduler.c @@ -62,12 +62,14 @@ int32_t schedulerInit(SSchedulerCfg *cfg) { SCH_ERR_RET(TSDB_CODE_QRY_SYS_ERROR); } - qInfo("scheduler %" PRIx64 " initizlized, maxJob:%u", schMgmt.sId, schMgmt.cfg.maxJobNum); + qInfo("scheduler 0x%" PRIx64 " initizlized, maxJob:%u", schMgmt.sId, schMgmt.cfg.maxJobNum); return TSDB_CODE_SUCCESS; } int32_t schedulerExecJob(SSchedulerReq *pReq, int64_t *pJob, SQueryResult *pRes) { + qDebug("scheduler sync exec job start"); + if (NULL == pReq || NULL == pJob || NULL == pRes) { SCH_ERR_RET(TSDB_CODE_QRY_INVALID_INPUT); } @@ -76,21 +78,27 @@ int32_t schedulerExecJob(SSchedulerReq *pReq, int64_t *pJob, SQueryResult *pRes) } int32_t schedulerAsyncExecJob(SSchedulerReq *pReq, int64_t *pJob) { + qDebug("scheduler async exec job start"); + int32_t code = 0; if (NULL == pReq || NULL == pJob) { - code = TSDB_CODE_QRY_INVALID_INPUT; - } else { - code = schAsyncExecJob(pReq, pJob); + SCH_ERR_JRET(TSDB_CODE_QRY_INVALID_INPUT); } + + schAsyncExecJob(pReq, pJob); + +_return: if (code != TSDB_CODE_SUCCESS) { pReq->fp(NULL, pReq->cbParam, code); } - return code; + SCH_RET(code); } int32_t schedulerFetchRows(int64_t job, void **pData) { + qDebug("scheduler sync fetch rows start"); + if (NULL == pData) { SCH_ERR_RET(TSDB_CODE_QRY_INVALID_INPUT); } @@ -102,7 +110,8 @@ int32_t schedulerFetchRows(int64_t job, void **pData) { SCH_ERR_RET(TSDB_CODE_SCH_STATUS_ERROR); } - pJob->attr.syncSchedule = true; + SCH_ERR_RET(schBeginOperation(pJob, SCH_OP_FETCH, true)); + pJob->userRes.fetchRes = pData; code = schFetchRows(pJob); @@ -112,24 +121,32 @@ int32_t schedulerFetchRows(int64_t job, void **pData) { } void schedulerAsyncFetchRows(int64_t job, schedulerFetchCallback fp, void* param) { + qDebug("scheduler async fetch rows start"); + + int32_t code = 0; if (NULL == fp || NULL == param) { - fp(NULL, param, TSDB_CODE_QRY_INVALID_INPUT); - return; + SCH_ERR_JRET(TSDB_CODE_QRY_INVALID_INPUT); } SSchJob *pJob = schAcquireJob(job); if (NULL == pJob) { - qError("acquire job from jobRef list failed, may be dropped, jobId:0x%" PRIx64, job); - fp(NULL, param, TSDB_CODE_SCH_STATUS_ERROR); - return; + qError("acquire sch job from job list failed, may be dropped, jobId:0x%" PRIx64, job); + SCH_ERR_JRET(TSDB_CODE_SCH_STATUS_ERROR); } - pJob->attr.syncSchedule = false; + SCH_ERR_JRET(schBeginOperation(pJob, SCH_OP_FETCH, false)); + pJob->userRes.fetchFp = fp; pJob->userRes.userParam = param; - /*code = */schAsyncFetchRows(pJob); + SCH_ERR_JRET(schAsyncFetchRows(pJob)); +_return: + + if (code) { + fp(NULL, param, code); + } + schReleaseJob(job); } @@ -137,12 +154,12 @@ int32_t schedulerGetTasksStatus(int64_t job, SArray *pSub) { int32_t code = 0; SSchJob *pJob = schAcquireJob(job); if (NULL == pJob) { - qDebug("acquire job from jobRef list failed, may not started or dropped, refId:%" PRIx64, job); + qDebug("acquire job from jobRef list failed, may not started or dropped, refId:0x%" PRIx64, job); SCH_ERR_RET(TSDB_CODE_SCH_STATUS_ERROR); } if (pJob->status < JOB_TASK_STATUS_NOT_START || pJob->levelNum <= 0 || NULL == pJob->levels) { - qDebug("job not initialized or not executable job, refId:%" PRIx64, job); + qDebug("job not initialized or not executable job, refId:0x%" PRIx64, job); SCH_ERR_JRET(TSDB_CODE_SCH_STATUS_ERROR); } @@ -188,21 +205,23 @@ void schedulerStopQueryHb(void *pTrans) { schCleanClusterHb(pTrans); } -void schedulerFreeJob(int64_t job) { +void schedulerFreeJob(int64_t job, int32_t errCode) { SSchJob *pJob = schAcquireJob(job); if (NULL == pJob) { qError("acquire job from jobRef list failed, may be dropped, jobId:0x%" PRIx64, job); return; } - if (atomic_load_8(&pJob->userFetch) > 0) { - schProcessOnJobDropped(pJob, TSDB_CODE_QRY_JOB_FREED); + int32_t code = schProcessOnJobDropped(pJob, errCode); + if (TSDB_CODE_SCH_JOB_IS_DROPPING == code) { + SCH_JOB_DLOG("sch job is already dropping, refId:0x%" PRIx64, job); + return; } - SCH_JOB_DLOG("start to remove job from jobRef list, refId:%" PRIx64, job); + SCH_JOB_DLOG("start to remove job from jobRef list, refId:0x%" PRIx64, job); if (taosRemoveRef(schMgmt.jobRef, job)) { - SCH_JOB_ELOG("remove job from job list failed, refId:%" PRIx64, job); + SCH_JOB_ELOG("remove job from job list failed, refId:0x%" PRIx64, job); } schReleaseJob(job); diff --git a/source/libs/scheduler/test/schedulerTests.cpp b/source/libs/scheduler/test/schedulerTests.cpp index 9b624ee5cdecac00694dfdbf857fd106ba7c152d..e5cc3cd48188f51ad9431c3e21027292998213ad 100644 --- a/source/libs/scheduler/test/schedulerTests.cpp +++ b/source/libs/scheduler/test/schedulerTests.cpp @@ -457,7 +457,7 @@ void schtFreeQueryJob(int32_t freeThread) { int64_t job = queryJobRefId; if (job && atomic_val_compare_exchange_64(&queryJobRefId, job, 0)) { - schedulerFreeJob(job); + schedulerFreeJob(job, 0); if (freeThread) { if (++freeNum % schtTestPrintNum == 0) { printf("FreeNum:%d\n", freeNum); @@ -724,7 +724,7 @@ TEST(queryTest, normalCase) { schReleaseJob(job); - schedulerFreeJob(job); + schedulerFreeJob(job, 0); schtFreeQueryDag(&dag); @@ -828,7 +828,7 @@ TEST(queryTest, readyFirstCase) { schReleaseJob(job); - schedulerFreeJob(job); + schedulerFreeJob(job, 0); schtFreeQueryDag(&dag); @@ -940,7 +940,7 @@ TEST(queryTest, flowCtrlCase) { schReleaseJob(job); - schedulerFreeJob(job); + schedulerFreeJob(job, 0); schtFreeQueryDag(&dag); @@ -994,7 +994,7 @@ TEST(insertTest, normalCase) { ASSERT_EQ(code, 0); ASSERT_EQ(res.numOfRows, 20); - schedulerFreeJob(insertJobRefId); + schedulerFreeJob(insertJobRefId, 0); schedulerDestroy(); } diff --git a/source/libs/stream/src/streamDispatch.c b/source/libs/stream/src/streamDispatch.c index 59ec2b5cebcce508cc7b75bcea446ca17f9ddc43..ca10e7d95642c99082dd4ca97ed81a49eaccbad5 100644 --- a/source/libs/stream/src/streamDispatch.c +++ b/source/libs/stream/src/streamDispatch.c @@ -134,6 +134,7 @@ int32_t streamBuildDispatchMsg(SStreamTask* pTask, SStreamDataBlock* data, SRpcM int32_t sz = taosArrayGetSize(vgInfo); for (int32_t i = 0; i < sz; i++) { SVgroupInfo* pVgInfo = taosArrayGet(vgInfo, i); + ASSERT(pVgInfo->vgId > 0); if (hashValue >= pVgInfo->hashBegin && hashValue <= pVgInfo->hashEnd) { vgId = pVgInfo->vgId; downstreamTaskId = pVgInfo->taskId; diff --git a/source/libs/stream/src/streamTask.c b/source/libs/stream/src/streamTask.c index 7a7d9d15ad85c7026381283b265bb72b81f32c75..a35e7679a121c9799a61a7973cbb1b3fc3898a5a 100644 --- a/source/libs/stream/src/streamTask.c +++ b/source/libs/stream/src/streamTask.c @@ -70,7 +70,7 @@ int32_t tEncodeSStreamTask(SEncoder* pEncoder, const SStreamTask* pTask) { if (tEncodeSEpSet(pEncoder, &pTask->fixedEpDispatcher.epSet) < 0) return -1; } else if (pTask->dispatchType == TASK_DISPATCH__SHUFFLE) { if (tSerializeSUseDbRspImp(pEncoder, &pTask->shuffleDispatcher.dbInfo) < 0) return -1; - /*if (tEncodeI8(pEncoder, pTask->shuffleDispatcher.hashMethod) < 0) return -1;*/ + if (tEncodeCStr(pEncoder, pTask->shuffleDispatcher.stbFullName) < 0) return -1; } if (tEncodeI64(pEncoder, pTask->triggerParam) < 0) return -1; @@ -119,8 +119,8 @@ int32_t tDecodeSStreamTask(SDecoder* pDecoder, SStreamTask* pTask) { if (tDecodeI32(pDecoder, &pTask->fixedEpDispatcher.nodeId) < 0) return -1; if (tDecodeSEpSet(pDecoder, &pTask->fixedEpDispatcher.epSet) < 0) return -1; } else if (pTask->dispatchType == TASK_DISPATCH__SHUFFLE) { - /*if (tDecodeI8(pDecoder, &pTask->shuffleDispatcher.hashMethod) < 0) return -1;*/ if (tDeserializeSUseDbRspImp(pDecoder, &pTask->shuffleDispatcher.dbInfo) < 0) return -1; + if (tDecodeCStrTo(pDecoder, pTask->shuffleDispatcher.stbFullName) < 0) return -1; } if (tDecodeI64(pDecoder, &pTask->triggerParam) < 0) return -1; diff --git a/source/libs/sync/inc/syncInt.h b/source/libs/sync/inc/syncInt.h index b00c7cbda1fcc430b4dd7e52b72ac40e739a9158..b910b59f83ee4d83b6b46a87c0439051ca5eea04 100644 --- a/source/libs/sync/inc/syncInt.h +++ b/source/libs/sync/inc/syncInt.h @@ -171,7 +171,8 @@ void syncNodeClose(SSyncNode* pSyncNode); int32_t syncNodePropose(SSyncNode* pSyncNode, const SRpcMsg* pMsg, bool isWeak); // option -bool syncNodeSnapshotEnable(SSyncNode* pSyncNode); +bool syncNodeSnapshotEnable(SSyncNode* pSyncNode); +SyncIndex syncNodeGetSnapshotConfigIndex(SSyncNode* pSyncNode, SyncIndex snapshotLastApplyIndex); // ping -------------- int32_t syncNodePing(SSyncNode* pSyncNode, const SRaftId* destRaftId, SyncPing* pMsg); @@ -194,6 +195,7 @@ int32_t syncNodeSendMsgById(const SRaftId* destRaftId, SSyncNode* pSyncNode, SRp int32_t syncNodeSendMsgByInfo(const SNodeInfo* nodeInfo, SSyncNode* pSyncNode, SRpcMsg* pMsg); cJSON* syncNode2Json(const SSyncNode* pSyncNode); char* syncNode2Str(const SSyncNode* pSyncNode); +void syncNodeEventLog(const SSyncNode* pSyncNode, char* str); char* syncNode2SimpleStr(const SSyncNode* pSyncNode); bool syncNodeInConfig(SSyncNode* pSyncNode, const SSyncCfg* config); void syncNodeUpdateConfig(SSyncNode* pSyncNode, SSyncCfg* newConfig, SyncIndex lastConfigChangeIndex, bool* isDrop); diff --git a/source/libs/sync/inc/syncRaftCfg.h b/source/libs/sync/inc/syncRaftCfg.h index e72e1e7be706523a0aec15b7488e6b61380e4f82..435ad98fb36f0677d8eb4d67585a33de3e2faae9 100644 --- a/source/libs/sync/inc/syncRaftCfg.h +++ b/source/libs/sync/inc/syncRaftCfg.h @@ -29,6 +29,8 @@ extern "C" { #define CONFIG_FILE_LEN 1024 +#define MAX_CONFIG_INDEX_COUNT 512 + typedef struct SRaftCfg { SSyncCfg cfg; TdFilePtr pFile; @@ -36,19 +38,25 @@ typedef struct SRaftCfg { int8_t isStandBy; int8_t snapshotEnable; SyncIndex lastConfigIndex; + + SyncIndex configIndexArr[MAX_CONFIG_INDEX_COUNT]; + int32_t configIndexCount; + } SRaftCfg; SRaftCfg *raftCfgOpen(const char *path); int32_t raftCfgClose(SRaftCfg *pRaftCfg); int32_t raftCfgPersist(SRaftCfg *pRaftCfg); +int32_t raftCfgAddConfigIndex(SRaftCfg *pRaftCfg, SyncIndex configIndex); -cJSON * syncCfg2Json(SSyncCfg *pSyncCfg); -char * syncCfg2Str(SSyncCfg *pSyncCfg); +cJSON *syncCfg2Json(SSyncCfg *pSyncCfg); +char *syncCfg2Str(SSyncCfg *pSyncCfg); +char *syncCfg2SimpleStr(SSyncCfg *pSyncCfg); int32_t syncCfgFromJson(const cJSON *pRoot, SSyncCfg *pSyncCfg); int32_t syncCfgFromStr(const char *s, SSyncCfg *pSyncCfg); -cJSON * raftCfg2Json(SRaftCfg *pRaftCfg); -char * raftCfg2Str(SRaftCfg *pRaftCfg); +cJSON *raftCfg2Json(SRaftCfg *pRaftCfg); +char *raftCfg2Str(SRaftCfg *pRaftCfg); int32_t raftCfgFromJson(const cJSON *pRoot, SRaftCfg *pRaftCfg); int32_t raftCfgFromStr(const char *s, SRaftCfg *pRaftCfg); @@ -65,6 +73,7 @@ void syncCfgPrint(SSyncCfg *pCfg); void syncCfgPrint2(char *s, SSyncCfg *pCfg); void syncCfgLog(SSyncCfg *pCfg); void syncCfgLog2(char *s, SSyncCfg *pCfg); +void syncCfgLog3(char *s, SSyncCfg *pCfg); void raftCfgPrint(SRaftCfg *pCfg); void raftCfgPrint2(char *s, SRaftCfg *pCfg); diff --git a/source/libs/sync/inc/syncSnapshot.h b/source/libs/sync/inc/syncSnapshot.h index a6170a92e3f29b2b420803932fdc9469f46a5f68..f6544dd79db4d508d069b7fc80921a0e55ed804c 100644 --- a/source/libs/sync/inc/syncSnapshot.h +++ b/source/libs/sync/inc/syncSnapshot.h @@ -39,8 +39,8 @@ typedef struct SSyncSnapshotSender { bool start; int32_t seq; int32_t ack; - void * pReader; - void * pCurrentBlock; + void *pReader; + void *pCurrentBlock; int32_t blockLen; SSnapshot snapshot; SSyncCfg lastConfig; @@ -59,14 +59,15 @@ void snapshotSenderStart(SSyncSnapshotSender *pSender); void snapshotSenderStop(SSyncSnapshotSender *pSender); int32_t snapshotSend(SSyncSnapshotSender *pSender); int32_t snapshotReSend(SSyncSnapshotSender *pSender); -cJSON * snapshotSender2Json(SSyncSnapshotSender *pSender); -char * snapshotSender2Str(SSyncSnapshotSender *pSender); +cJSON *snapshotSender2Json(SSyncSnapshotSender *pSender); +char *snapshotSender2Str(SSyncSnapshotSender *pSender); +char *snapshotSender2SimpleStr(SSyncSnapshotSender *pSender, char *event); typedef struct SSyncSnapshotReceiver { bool start; int32_t ack; - void * pWriter; + void *pWriter; SyncTerm term; SyncTerm privateTerm; @@ -80,8 +81,9 @@ void snapshotReceiverDestroy(SSyncSnapshotReceiver *pReceiver) void snapshotReceiverStart(SSyncSnapshotReceiver *pReceiver, SyncTerm privateTerm, SRaftId fromId); bool snapshotReceiverIsStart(SSyncSnapshotReceiver *pReceiver); void snapshotReceiverStop(SSyncSnapshotReceiver *pReceiver, bool apply); -cJSON * snapshotReceiver2Json(SSyncSnapshotReceiver *pReceiver); -char * snapshotReceiver2Str(SSyncSnapshotReceiver *pReceiver); +cJSON *snapshotReceiver2Json(SSyncSnapshotReceiver *pReceiver); +char *snapshotReceiver2Str(SSyncSnapshotReceiver *pReceiver); +char *snapshotReceiver2SimpleStr(SSyncSnapshotReceiver *pReceiver, char *event); int32_t syncNodeOnSnapshotSendCb(SSyncNode *ths, SyncSnapshotSend *pMsg); int32_t syncNodeOnSnapshotRspCb(SSyncNode *ths, SyncSnapshotRsp *pMsg); diff --git a/source/libs/sync/src/syncAppendEntries.c b/source/libs/sync/src/syncAppendEntries.c index 89b4761212bd4ea50451fed0821adb862df1dc98..58e40668a22a7b36903346fdca37217885786c1d 100644 --- a/source/libs/sync/src/syncAppendEntries.c +++ b/source/libs/sync/src/syncAppendEntries.c @@ -208,8 +208,9 @@ int32_t syncNodeOnAppendEntriesCb(SSyncNode* ths, SyncAppendEntries* pMsg) { SRpcMsg rpcMsg; syncEntry2OriginalRpc(pRollBackEntry, &rpcMsg); - SFsmCbMeta cbMeta; + SFsmCbMeta cbMeta = {0}; cbMeta.index = pRollBackEntry->index; + cbMeta.lastConfigIndex = syncNodeGetSnapshotConfigIndex(ths, cbMeta.index); cbMeta.isWeak = pRollBackEntry->isWeak; cbMeta.code = 0; cbMeta.state = ths->state; @@ -234,8 +235,9 @@ int32_t syncNodeOnAppendEntriesCb(SSyncNode* ths, SyncAppendEntries* pMsg) { if (ths->pFsm != NULL) { // if (ths->pFsm->FpPreCommitCb != NULL && pAppendEntry->originalRpcType != TDMT_SYNC_NOOP) { if (ths->pFsm->FpPreCommitCb != NULL && syncUtilUserPreCommit(pAppendEntry->originalRpcType)) { - SFsmCbMeta cbMeta; + SFsmCbMeta cbMeta = {0}; cbMeta.index = pAppendEntry->index; + cbMeta.lastConfigIndex = syncNodeGetSnapshotConfigIndex(ths, cbMeta.index); cbMeta.isWeak = pAppendEntry->isWeak; cbMeta.code = 2; cbMeta.state = ths->state; @@ -266,8 +268,9 @@ int32_t syncNodeOnAppendEntriesCb(SSyncNode* ths, SyncAppendEntries* pMsg) { if (ths->pFsm != NULL) { // if (ths->pFsm->FpPreCommitCb != NULL && pAppendEntry->originalRpcType != TDMT_SYNC_NOOP) { if (ths->pFsm->FpPreCommitCb != NULL && syncUtilUserPreCommit(pAppendEntry->originalRpcType)) { - SFsmCbMeta cbMeta; + SFsmCbMeta cbMeta = {0}; cbMeta.index = pAppendEntry->index; + cbMeta.lastConfigIndex = syncNodeGetSnapshotConfigIndex(ths, cbMeta.index); cbMeta.isWeak = pAppendEntry->isWeak; cbMeta.code = 3; cbMeta.state = ths->state; @@ -326,358 +329,6 @@ int32_t syncNodeOnAppendEntriesCb(SSyncNode* ths, SyncAppendEntries* pMsg) { return ret; } -#if 0 -int32_t syncNodeOnAppendEntriesCb(SSyncNode* ths, SyncAppendEntries* pMsg) { - int32_t ret = 0; - - char logBuf[128] = {0}; - snprintf(logBuf, sizeof(logBuf), "==syncNodeOnAppendEntriesCb== term:%lu", ths->pRaftStore->currentTerm); - syncAppendEntriesLog2(logBuf, pMsg); - - if (pMsg->term > ths->pRaftStore->currentTerm) { - syncNodeUpdateTerm(ths, pMsg->term); - } - assert(pMsg->term <= ths->pRaftStore->currentTerm); - - // reset elect timer - if (pMsg->term == ths->pRaftStore->currentTerm) { - ths->leaderCache = pMsg->srcId; - syncNodeResetElectTimer(ths); - } - assert(pMsg->dataLen >= 0); - - SyncTerm localPreLogTerm = 0; - if (pMsg->prevLogIndex >= SYNC_INDEX_BEGIN && pMsg->prevLogIndex <= ths->pLogStore->getLastIndex(ths->pLogStore)) { - SSyncRaftEntry* pEntry = ths->pLogStore->getEntry(ths->pLogStore, pMsg->prevLogIndex); - assert(pEntry != NULL); - localPreLogTerm = pEntry->term; - syncEntryDestory(pEntry); - } - - bool logOK = - (pMsg->prevLogIndex == SYNC_INDEX_INVALID) || - ((pMsg->prevLogIndex >= SYNC_INDEX_BEGIN) && - (pMsg->prevLogIndex <= ths->pLogStore->getLastIndex(ths->pLogStore)) && (pMsg->prevLogTerm == localPreLogTerm)); - - // reject request - if ((pMsg->term < ths->pRaftStore->currentTerm) || - ((pMsg->term == ths->pRaftStore->currentTerm) && (ths->state == TAOS_SYNC_STATE_FOLLOWER) && !logOK)) { - sTrace( - "syncNodeOnAppendEntriesCb --> reject, pMsg->term:%lu, ths->pRaftStore->currentTerm:%lu, ths->state:%d, " - "logOK:%d", - pMsg->term, ths->pRaftStore->currentTerm, ths->state, logOK); - - SyncAppendEntriesReply* pReply = syncAppendEntriesReplyBuild(ths->vgId); - pReply->srcId = ths->myRaftId; - pReply->destId = pMsg->srcId; - pReply->term = ths->pRaftStore->currentTerm; - pReply->success = false; - pReply->matchIndex = SYNC_INDEX_INVALID; - - SRpcMsg rpcMsg; - syncAppendEntriesReply2RpcMsg(pReply, &rpcMsg); - syncNodeSendMsgById(&pReply->destId, ths, &rpcMsg); - syncAppendEntriesReplyDestroy(pReply); - - return ret; - } - - // return to follower state - if (pMsg->term == ths->pRaftStore->currentTerm && ths->state == TAOS_SYNC_STATE_CANDIDATE) { - sTrace( - "syncNodeOnAppendEntriesCb --> return to follower, pMsg->term:%lu, ths->pRaftStore->currentTerm:%lu, " - "ths->state:%d, logOK:%d", - pMsg->term, ths->pRaftStore->currentTerm, ths->state, logOK); - - syncNodeBecomeFollower(ths, "from candidate by append entries"); - - // ret or reply? - return ret; - } - - // accept request - if (pMsg->term == ths->pRaftStore->currentTerm && ths->state == TAOS_SYNC_STATE_FOLLOWER && logOK) { - // preIndex = -1, or has preIndex entry in local log - assert(pMsg->prevLogIndex <= ths->pLogStore->getLastIndex(ths->pLogStore)); - - // has extra entries (> preIndex) in local log - bool hasExtraEntries = pMsg->prevLogIndex < ths->pLogStore->getLastIndex(ths->pLogStore); - - // has entries in SyncAppendEntries msg - bool hasAppendEntries = pMsg->dataLen > 0; - - sTrace( - "syncNodeOnAppendEntriesCb --> accept, pMsg->term:%lu, ths->pRaftStore->currentTerm:%lu, ths->state:%d, " - "logOK:%d, hasExtraEntries:%d, hasAppendEntries:%d", - pMsg->term, ths->pRaftStore->currentTerm, ths->state, logOK, hasExtraEntries, hasAppendEntries); - - if (hasExtraEntries && hasAppendEntries) { - // not conflict by default - bool conflict = false; - - SyncIndex extraIndex = pMsg->prevLogIndex + 1; - SSyncRaftEntry* pExtraEntry = ths->pLogStore->getEntry(ths->pLogStore, extraIndex); - assert(pExtraEntry != NULL); - - SSyncRaftEntry* pAppendEntry = syncEntryDeserialize(pMsg->data, pMsg->dataLen); - assert(pAppendEntry != NULL); - - // log not match, conflict - assert(extraIndex == pAppendEntry->index); - if (pExtraEntry->term != pAppendEntry->term) { - conflict = true; - } - - if (conflict) { - // roll back - SyncIndex delBegin = ths->pLogStore->getLastIndex(ths->pLogStore); - SyncIndex delEnd = extraIndex; - - sTrace("syncNodeOnAppendEntriesCb --> conflict:%d, delBegin:%ld, delEnd:%ld", conflict, delBegin, delEnd); - - // notice! reverse roll back! - for (SyncIndex index = delEnd; index >= delBegin; --index) { - if (ths->pFsm->FpRollBackCb != NULL) { - SSyncRaftEntry* pRollBackEntry = ths->pLogStore->getEntry(ths->pLogStore, index); - assert(pRollBackEntry != NULL); - - // if (pRollBackEntry->msgType != TDMT_SYNC_NOOP) { - if (syncUtilUserRollback(pRollBackEntry->msgType)) { - SRpcMsg rpcMsg; - syncEntry2OriginalRpc(pRollBackEntry, &rpcMsg); - - SFsmCbMeta cbMeta; - cbMeta.index = pRollBackEntry->index; - cbMeta.isWeak = pRollBackEntry->isWeak; - cbMeta.code = 0; - cbMeta.state = ths->state; - cbMeta.seqNum = pRollBackEntry->seqNum; - ths->pFsm->FpRollBackCb(ths->pFsm, &rpcMsg, cbMeta); - rpcFreeCont(rpcMsg.pCont); - } - - syncEntryDestory(pRollBackEntry); - } - } - - // delete confict entries - ths->pLogStore->truncate(ths->pLogStore, extraIndex); - - // append new entries - ths->pLogStore->appendEntry(ths->pLogStore, pAppendEntry); - - // pre commit - SRpcMsg rpcMsg; - syncEntry2OriginalRpc(pAppendEntry, &rpcMsg); - if (ths->pFsm != NULL) { - // if (ths->pFsm->FpPreCommitCb != NULL && pAppendEntry->originalRpcType != TDMT_SYNC_NOOP) { - if (ths->pFsm->FpPreCommitCb != NULL && syncUtilUserPreCommit(pAppendEntry->originalRpcType)) { - SFsmCbMeta cbMeta; - cbMeta.index = pAppendEntry->index; - cbMeta.isWeak = pAppendEntry->isWeak; - cbMeta.code = 2; - cbMeta.state = ths->state; - cbMeta.seqNum = pAppendEntry->seqNum; - ths->pFsm->FpPreCommitCb(ths->pFsm, &rpcMsg, cbMeta); - } - } - rpcFreeCont(rpcMsg.pCont); - } - - // free memory - syncEntryDestory(pExtraEntry); - syncEntryDestory(pAppendEntry); - - } else if (hasExtraEntries && !hasAppendEntries) { - // do nothing - - } else if (!hasExtraEntries && hasAppendEntries) { - SSyncRaftEntry* pAppendEntry = syncEntryDeserialize(pMsg->data, pMsg->dataLen); - assert(pAppendEntry != NULL); - - // append new entries - ths->pLogStore->appendEntry(ths->pLogStore, pAppendEntry); - - // pre commit - SRpcMsg rpcMsg; - syncEntry2OriginalRpc(pAppendEntry, &rpcMsg); - if (ths->pFsm != NULL) { - // if (ths->pFsm->FpPreCommitCb != NULL && pAppendEntry->originalRpcType != TDMT_SYNC_NOOP) { - if (ths->pFsm->FpPreCommitCb != NULL && syncUtilUserPreCommit(pAppendEntry->originalRpcType)) { - SFsmCbMeta cbMeta; - cbMeta.index = pAppendEntry->index; - cbMeta.isWeak = pAppendEntry->isWeak; - cbMeta.code = 3; - cbMeta.state = ths->state; - cbMeta.seqNum = pAppendEntry->seqNum; - ths->pFsm->FpPreCommitCb(ths->pFsm, &rpcMsg, cbMeta); - } - } - rpcFreeCont(rpcMsg.pCont); - - // free memory - syncEntryDestory(pAppendEntry); - - } else if (!hasExtraEntries && !hasAppendEntries) { - // do nothing - - } else { - assert(0); - } - - SyncAppendEntriesReply* pReply = syncAppendEntriesReplyBuild(ths->vgId); - pReply->srcId = ths->myRaftId; - pReply->destId = pMsg->srcId; - pReply->term = ths->pRaftStore->currentTerm; - pReply->success = true; - - if (hasAppendEntries) { - pReply->matchIndex = pMsg->prevLogIndex + 1; - } else { - pReply->matchIndex = pMsg->prevLogIndex; - } - - SRpcMsg rpcMsg; - syncAppendEntriesReply2RpcMsg(pReply, &rpcMsg); - syncNodeSendMsgById(&pReply->destId, ths, &rpcMsg); - syncAppendEntriesReplyDestroy(pReply); - - // maybe update commit index from leader - if (pMsg->commitIndex > ths->commitIndex) { - // has commit entry in local - if (pMsg->commitIndex <= ths->pLogStore->getLastIndex(ths->pLogStore)) { - SyncIndex beginIndex = ths->commitIndex + 1; - SyncIndex endIndex = pMsg->commitIndex; - - // update commit index - ths->commitIndex = pMsg->commitIndex; - - // call back Wal - ths->pLogStore->updateCommitIndex(ths->pLogStore, ths->commitIndex); - - // execute fsm - if (ths->pFsm != NULL) { - for (SyncIndex i = beginIndex; i <= endIndex; ++i) { - if (i != SYNC_INDEX_INVALID) { - SSyncRaftEntry* pEntry = ths->pLogStore->getEntry(ths->pLogStore, i); - assert(pEntry != NULL); - - SRpcMsg rpcMsg; - syncEntry2OriginalRpc(pEntry, &rpcMsg); - - if (ths->pFsm->FpCommitCb != NULL && syncUtilUserCommit(pEntry->originalRpcType)) { - SFsmCbMeta cbMeta; - cbMeta.index = pEntry->index; - cbMeta.isWeak = pEntry->isWeak; - cbMeta.code = 0; - cbMeta.state = ths->state; - cbMeta.seqNum = pEntry->seqNum; - cbMeta.term = pEntry->term; - cbMeta.currentTerm = ths->pRaftStore->currentTerm; - cbMeta.flag = 0x11; - - SSnapshot snapshot; - ASSERT(ths->pFsm->FpGetSnapshot != NULL); - ths->pFsm->FpGetSnapshot(ths->pFsm, &snapshot); - - bool needExecute = true; - if (cbMeta.index <= snapshot.lastApplyIndex) { - needExecute = false; - } - - if (needExecute) { - ths->pFsm->FpCommitCb(ths->pFsm, &rpcMsg, cbMeta); - } - } - - // config change - if (pEntry->originalRpcType == TDMT_SYNC_CONFIG_CHANGE) { - SSyncCfg oldSyncCfg = ths->pRaftCfg->cfg; - - SSyncCfg newSyncCfg; - int32_t ret = syncCfgFromStr(rpcMsg.pCont, &newSyncCfg); - ASSERT(ret == 0); - - // update new config myIndex - bool hit = false; - for (int i = 0; i < newSyncCfg.replicaNum; ++i) { - if (strcmp(ths->myNodeInfo.nodeFqdn, (newSyncCfg.nodeInfo)[i].nodeFqdn) == 0 && - ths->myNodeInfo.nodePort == (newSyncCfg.nodeInfo)[i].nodePort) { - newSyncCfg.myIndex = i; - hit = true; - break; - } - } - - SReConfigCbMeta cbMeta = {0}; - bool isDrop; - - // I am in newConfig - if (hit) { - syncNodeUpdateConfig(ths, &newSyncCfg, pEntry->index, &isDrop); - - // change isStandBy to normal - if (!isDrop) { - if (ths->state == TAOS_SYNC_STATE_LEADER) { - syncNodeBecomeLeader(ths, "config change"); - } else { - syncNodeBecomeFollower(ths, "config change"); - } - } - - if (gRaftDetailLog) { - char* sOld = syncCfg2Str(&oldSyncCfg); - char* sNew = syncCfg2Str(&newSyncCfg); - sInfo("==config change== 0x11 old:%s new:%s isDrop:%d \n", sOld, sNew, isDrop); - taosMemoryFree(sOld); - taosMemoryFree(sNew); - } - } - - // always call FpReConfigCb - if (ths->pFsm->FpReConfigCb != NULL) { - cbMeta.code = 0; - cbMeta.currentTerm = ths->pRaftStore->currentTerm; - cbMeta.index = pEntry->index; - cbMeta.term = pEntry->term; - cbMeta.newCfg = newSyncCfg; - cbMeta.oldCfg = oldSyncCfg; - cbMeta.seqNum = pEntry->seqNum; - cbMeta.flag = 0x11; - cbMeta.isDrop = isDrop; - ths->pFsm->FpReConfigCb(ths->pFsm, &rpcMsg, cbMeta); - } - } - - // restore finish - if (pEntry->index == ths->pLogStore->getLastIndex(ths->pLogStore)) { - if (ths->restoreFinish == false) { - if (ths->pFsm->FpRestoreFinishCb != NULL) { - ths->pFsm->FpRestoreFinishCb(ths->pFsm); - } - ths->restoreFinish = true; - sInfo("==syncNodeOnAppendEntriesCb== restoreFinish set true %p vgId:%d", ths, ths->vgId); - - /* - tsem_post(&ths->restoreSem); - sInfo("==syncNodeOnAppendEntriesCb== RestoreFinish tsem_post %p", ths); - */ - } - } - - rpcFreeCont(rpcMsg.pCont); - syncEntryDestory(pEntry); - } - } - } - } - } - } - - return ret; -} -#endif - static int32_t syncNodeMakeLogSame(SSyncNode* ths, SyncAppendEntries* pMsg) { int32_t code; @@ -696,8 +347,9 @@ static int32_t syncNodeMakeLogSame(SSyncNode* ths, SyncAppendEntries* pMsg) { SRpcMsg rpcMsg; syncEntry2OriginalRpc(pRollBackEntry, &rpcMsg); - SFsmCbMeta cbMeta; + SFsmCbMeta cbMeta = {0}; cbMeta.index = pRollBackEntry->index; + cbMeta.lastConfigIndex = syncNodeGetSnapshotConfigIndex(ths, cbMeta.index); cbMeta.isWeak = pRollBackEntry->isWeak; cbMeta.code = 0; cbMeta.state = ths->state; @@ -713,8 +365,10 @@ static int32_t syncNodeMakeLogSame(SSyncNode* ths, SyncAppendEntries* pMsg) { // delete confict entries code = ths->pLogStore->syncLogTruncate(ths->pLogStore, delBegin); ASSERT(code == 0); - sDebug("vgId:%d sync event currentTerm:%lu log truncate, from %ld to %ld", ths->vgId, ths->pRaftStore->currentTerm, - delBegin, delEnd); + + char eventLog[128]; + snprintf(eventLog, sizeof(eventLog), "log truncate, from %ld to %ld", delBegin, delEnd); + syncNodeEventLog(ths, eventLog); logStoreSimpleLog2("after syncNodeMakeLogSame", ths->pLogStore); return code; @@ -725,8 +379,9 @@ static int32_t syncNodePreCommit(SSyncNode* ths, SSyncRaftEntry* pEntry) { syncEntry2OriginalRpc(pEntry, &rpcMsg); if (ths->pFsm != NULL) { if (ths->pFsm->FpPreCommitCb != NULL && syncUtilUserPreCommit(pEntry->originalRpcType)) { - SFsmCbMeta cbMeta; + SFsmCbMeta cbMeta = {0}; cbMeta.index = pEntry->index; + cbMeta.lastConfigIndex = syncNodeGetSnapshotConfigIndex(ths, cbMeta.index); cbMeta.isWeak = pEntry->isWeak; cbMeta.code = 2; cbMeta.state = ths->state; @@ -880,6 +535,72 @@ int32_t syncNodeOnAppendEntriesSnapshotCb(SSyncNode* ths, SyncAppendEntries* pMs } } while (0); + // fake match2 + // + // condition1: + // preIndex <= my commit index + // + // operation: + // if hasAppendEntries && pMsg->prevLogIndex == ths->commitIndex, append entry + // match my-commit-index or my-commit-index + 1 + // no operation on log + do { + bool condition = (pMsg->term == ths->pRaftStore->currentTerm) && (ths->state == TAOS_SYNC_STATE_FOLLOWER) && + (pMsg->prevLogIndex <= ths->commitIndex); + if (condition) { + sTrace("recv SyncAppendEntries, fake match2, msg-prevLogIndex:%ld, my-commitIndex:%ld", pMsg->prevLogIndex, + ths->commitIndex); + + SyncIndex matchIndex = ths->commitIndex; + bool hasAppendEntries = pMsg->dataLen > 0; + if (hasAppendEntries && pMsg->prevLogIndex == ths->commitIndex) { + // append entry + SSyncRaftEntry* pAppendEntry = syncEntryDeserialize(pMsg->data, pMsg->dataLen); + ASSERT(pAppendEntry != NULL); + + { + // has extra entries (> preIndex) in local log + SyncIndex logLastIndex = ths->pLogStore->syncLogLastIndex(ths->pLogStore); + bool hasExtraEntries = logLastIndex > pMsg->prevLogIndex; + + if (hasExtraEntries) { + // make log same, rollback deleted entries + code = syncNodeMakeLogSame(ths, pMsg); + ASSERT(code == 0); + } + } + + code = ths->pLogStore->syncLogAppendEntry(ths->pLogStore, pAppendEntry); + ASSERT(code == 0); + + // pre commit + code = syncNodePreCommit(ths, pAppendEntry); + ASSERT(code == 0); + + matchIndex = pMsg->prevLogIndex + 1; + + syncEntryDestory(pAppendEntry); + } + + // prepare response msg + SyncAppendEntriesReply* pReply = syncAppendEntriesReplyBuild(ths->vgId); + pReply->srcId = ths->myRaftId; + pReply->destId = pMsg->srcId; + pReply->term = ths->pRaftStore->currentTerm; + pReply->privateTerm = ths->pNewNodeReceiver->privateTerm; + pReply->success = true; + pReply->matchIndex = matchIndex; + + // send response + SRpcMsg rpcMsg; + syncAppendEntriesReply2RpcMsg(pReply, &rpcMsg); + syncNodeSendMsgById(&pReply->destId, ths, &rpcMsg); + syncAppendEntriesReplyDestroy(pReply); + + return ret; + } + } while (0); + // calculate logOK here, before will coredump, due to fake match bool logOK = syncNodeOnAppendEntriesLogOK(ths, pMsg); @@ -995,8 +716,10 @@ int32_t syncNodeOnAppendEntriesSnapshotCb(SSyncNode* ths, SyncAppendEntries* pMs SyncIndex commitEnd = snapshot.lastApplyIndex; ths->commitIndex = snapshot.lastApplyIndex; - sDebug("vgId:%d sync event currentTerm:%lu commit by snapshot from index:%ld to index:%ld, %s", ths->vgId, - ths->pRaftStore->currentTerm, commitBegin, commitEnd, syncUtilState2String(ths->state)); + char eventLog[128]; + snprintf(eventLog, sizeof(eventLog), "commit by snapshot from index:%ld to index:%ld", commitBegin, + commitEnd); + syncNodeEventLog(ths, eventLog); } SyncIndex beginIndex = ths->commitIndex + 1; diff --git a/source/libs/sync/src/syncAppendEntriesReply.c b/source/libs/sync/src/syncAppendEntriesReply.c index 5290d7d28eb57728531bcc1dd63ba6a808d82563..5e5e23b3120336755216f37fab7fc37365898f34 100644 --- a/source/libs/sync/src/syncAppendEntriesReply.c +++ b/source/libs/sync/src/syncAppendEntriesReply.c @@ -183,27 +183,9 @@ int32_t syncNodeOnAppendEntriesReplySnapshotCb(SSyncNode* ths, SyncAppendEntries pMsg->privateTerm < pSender->privateTerm) { snapshotSenderStart(pSender); - char host[128]; - uint16_t port; - syncUtilU642Addr(pSender->pSyncNode->replicasId[pSender->replicaIndex].addr, host, sizeof(host), &port); - - if (gRaftDetailLog) { - char* s = snapshotSender2Str(pSender); - sDebug( - "vgId:%d sync event currentTerm:%lu snapshot send to %s:%d start sender first time, lastApplyIndex:%ld " - "lastApplyTerm:%lu " - "lastConfigIndex:%ld privateTerm:%lu " - "sender:%s", - ths->vgId, ths->pRaftStore->currentTerm, host, port, pSender->snapshot.lastApplyIndex, - pSender->snapshot.lastApplyTerm, pSender->snapshot.lastConfigIndex, pSender->privateTerm, s); - taosMemoryFree(s); - } else { - sDebug( - "vgId:%d sync event currentTerm:%lu snapshot send to %s:%d start sender first time, lastApplyIndex:%ld " - "lastApplyTerm:%lu lastConfigIndex:%ld privateTerm:%lu", - ths->vgId, ths->pRaftStore->currentTerm, host, port, pSender->snapshot.lastApplyIndex, - pSender->snapshot.lastApplyTerm, pSender->snapshot.lastConfigIndex, pSender->privateTerm); - } + char* eventLog = snapshotSender2SimpleStr(pSender, "snapshot sender start"); + syncNodeEventLog(ths, eventLog); + taosMemoryFree(eventLog); } SyncIndex sentryIndex = pSender->snapshot.lastApplyIndex + 1; diff --git a/source/libs/sync/src/syncCommit.c b/source/libs/sync/src/syncCommit.c index efdb5b24ce6497890508c62881deccb6b408410c..738cbd3c2b8eb8a092a214efb4dd4e4623002123 100644 --- a/source/libs/sync/src/syncCommit.c +++ b/source/libs/sync/src/syncCommit.c @@ -56,9 +56,10 @@ void syncMaybeAdvanceCommitIndex(SSyncNode* pSyncNode) { SyncIndex commitEnd = snapshot.lastApplyIndex; pSyncNode->commitIndex = snapshot.lastApplyIndex; - sDebug("vgId:%d sync event currentTerm:%lu commit by snapshot from index:%ld to index:%ld, %s", pSyncNode->vgId, - pSyncNode->pRaftStore->currentTerm, pSyncNode->commitIndex, snapshot.lastApplyIndex, - syncUtilState2String(pSyncNode->state)); + char eventLog[128]; + snprintf(eventLog, sizeof(eventLog), "commit by snapshot from index:%ld to index:%ld", pSyncNode->commitIndex, + snapshot.lastApplyIndex); + syncNodeEventLog(pSyncNode, eventLog); } // update commit index diff --git a/source/libs/sync/src/syncMain.c b/source/libs/sync/src/syncMain.c index 557342be1cf46ec656c515436404348ce835f740..15a67adf77afbb21322b2c5ed1e5fa78c38ee8d9 100644 --- a/source/libs/sync/src/syncMain.c +++ b/source/libs/sync/src/syncMain.c @@ -189,24 +189,6 @@ int32_t syncReconfigBuild(int64_t rid, const SSyncCfg* pNewCfg, SRpcMsg* pRpcMsg int32_t ret = 0; bool IamInNew = syncNodeInConfig(pSyncNode, pNewCfg); -#if 0 - for (int i = 0; i < pNewCfg->replicaNum; ++i) { - if (strcmp((pNewCfg->nodeInfo)[i].nodeFqdn, pSyncNode->myNodeInfo.nodeFqdn) == 0 && - (pNewCfg->nodeInfo)[i].nodePort == pSyncNode->myNodeInfo.nodePort) { - IamInNew = true; - } - - /* - SRaftId newId; - newId.addr = syncUtilAddr2U64((pNewCfg->nodeInfo)[i].nodeFqdn, (pNewCfg->nodeInfo)[i].nodePort); - newId.vgId = pSyncNode->vgId; - if (syncUtilSameId(&(pSyncNode->myRaftId), &newId)) { - IamInNew = true; - } - */ - } -#endif - if (!IamInNew) { taosReleaseRef(tsNodeRefId, pSyncNode->rid); terrno = TSDB_CODE_SYN_NOT_IN_NEW_CONFIG; @@ -235,27 +217,6 @@ int32_t syncReconfig(int64_t rid, const SSyncCfg* pNewCfg) { bool IamInNew = syncNodeInConfig(pSyncNode, pNewCfg); -#if 0 - for (int i = 0; i < pNewCfg->replicaNum; ++i) { - if (strcmp((pNewCfg->nodeInfo)[i].nodeFqdn, pSyncNode->myNodeInfo.nodeFqdn) == 0 && - (pNewCfg->nodeInfo)[i].nodePort == pSyncNode->myNodeInfo.nodePort) { - IamInNew = true; - } - - /* - // some problem in inet_addr - - SRaftId newId = EMPTY_RAFT_ID; - newId.addr = syncUtilAddr2U64((pNewCfg->nodeInfo)[i].nodeFqdn, (pNewCfg->nodeInfo)[i].nodePort); - newId.vgId = pSyncNode->vgId; - - if (syncUtilSameId(&(pSyncNode->myRaftId), &newId)) { - IamInNew = true; - } - */ - } -#endif - if (!IamInNew) { sError("sync reconfig error, not in new config"); taosReleaseRef(tsNodeRefId, pSyncNode->rid); @@ -414,12 +375,51 @@ int32_t syncGetSnapshotMeta(int64_t rid, struct SSnapshotMeta* sMeta) { assert(rid == pSyncNode->rid); sMeta->lastConfigIndex = pSyncNode->pRaftCfg->lastConfigIndex; - sTrace("sync get snapshot meta: lastConfigIndex:%ld", pSyncNode->pRaftCfg->lastConfigIndex); + sTrace("vgId:%d, get snapshot meta, lastConfigIndex:%" PRId64, pSyncNode->vgId, pSyncNode->pRaftCfg->lastConfigIndex); taosReleaseRef(tsNodeRefId, pSyncNode->rid); return 0; } +int32_t syncGetSnapshotMetaByIndex(int64_t rid, SyncIndex snapshotIndex, struct SSnapshotMeta* sMeta) { + SSyncNode* pSyncNode = (SSyncNode*)taosAcquireRef(tsNodeRefId, rid); + if (pSyncNode == NULL) { + return -1; + } + assert(rid == pSyncNode->rid); + + ASSERT(pSyncNode->pRaftCfg->configIndexCount >= 1); + SyncIndex lastIndex = (pSyncNode->pRaftCfg->configIndexArr)[0]; + + for (int i = 0; i < pSyncNode->pRaftCfg->configIndexCount; ++i) { + if ((pSyncNode->pRaftCfg->configIndexArr)[i] > lastIndex && + (pSyncNode->pRaftCfg->configIndexArr)[i] <= snapshotIndex) { + lastIndex = (pSyncNode->pRaftCfg->configIndexArr)[i]; + } + } + sMeta->lastConfigIndex = lastIndex; + sTrace("vgId:%d, get snapshot meta by index:%" PRId64 " lastConfigIndex:%" PRId64, pSyncNode->vgId, snapshotIndex, + sMeta->lastConfigIndex); + + taosReleaseRef(tsNodeRefId, pSyncNode->rid); + return 0; +} + +SyncIndex syncNodeGetSnapshotConfigIndex(SSyncNode* pSyncNode, SyncIndex snapshotLastApplyIndex) { + ASSERT(pSyncNode->pRaftCfg->configIndexCount >= 1); + SyncIndex lastIndex = (pSyncNode->pRaftCfg->configIndexArr)[0]; + + for (int i = 0; i < pSyncNode->pRaftCfg->configIndexCount; ++i) { + if ((pSyncNode->pRaftCfg->configIndexArr)[i] > lastIndex && + (pSyncNode->pRaftCfg->configIndexArr)[i] <= snapshotLastApplyIndex) { + lastIndex = (pSyncNode->pRaftCfg->configIndexArr)[i]; + } + } + + sTrace("sync syncNodeGetSnapshotConfigIndex index:%ld lastConfigIndex:%ld", snapshotLastApplyIndex, lastIndex); + return lastIndex; +} + const char* syncGetMyRoleStr(int64_t rid) { const char* s = syncUtilState2String(syncGetMyRole(rid)); return s; @@ -575,8 +575,6 @@ int32_t syncPropose(int64_t rid, const SRpcMsg* pMsg, bool isWeak) { return -1; } assert(rid == pSyncNode->rid); - sDebug("vgId:%d sync event currentTerm:%lu propose msgType:%s,%d", pSyncNode->vgId, - pSyncNode->pRaftStore->currentTerm, TMSG_INFO(pMsg->msgType), pMsg->msgType); ret = syncNodePropose(pSyncNode, pMsg, isWeak); taosReleaseRef(tsNodeRefId, pSyncNode->rid); @@ -585,8 +583,10 @@ int32_t syncPropose(int64_t rid, const SRpcMsg* pMsg, bool isWeak) { int32_t syncNodePropose(SSyncNode* pSyncNode, const SRpcMsg* pMsg, bool isWeak) { int32_t ret = 0; - sDebug("vgId:%d sync event currentTerm:%lu propose msgType:%s,%d", pSyncNode->vgId, - pSyncNode->pRaftStore->currentTerm, TMSG_INFO(pMsg->msgType), pMsg->msgType); + + char eventLog[128]; + snprintf(eventLog, sizeof(eventLog), "propose type:%s,%d", TMSG_INFO(pMsg->msgType), pMsg->msgType); + syncNodeEventLog(pSyncNode, eventLog); if (pSyncNode->state == TAOS_SYNC_STATE_LEADER) { SRespStub stub; @@ -829,10 +829,7 @@ SSyncNode* syncNodeOpen(const SSyncInfo* pOldSyncInfo) { // start raft // syncNodeBecomeFollower(pSyncNode); - // snapshot meta - // pSyncNode->sMeta.lastConfigIndex = -1; - - sDebug("vgId:%d sync event currentTerm:%lu sync open", pSyncNode->vgId, pSyncNode->pRaftStore->currentTerm); + syncNodeEventLog(pSyncNode, "sync open"); return pSyncNode; } @@ -879,7 +876,7 @@ void syncNodeStartStandBy(SSyncNode* pSyncNode) { } void syncNodeClose(SSyncNode* pSyncNode) { - sDebug("vgId:%d sync event currentTerm:%lu sync close", pSyncNode->vgId, pSyncNode->pRaftStore->currentTerm); + syncNodeEventLog(pSyncNode, "sync close"); int32_t ret; assert(pSyncNode != NULL); @@ -1263,11 +1260,32 @@ char* syncNode2Str(const SSyncNode* pSyncNode) { return serialized; } +void syncNodeEventLog(const SSyncNode* pSyncNode, char* str) { + int32_t userStrLen = strlen(str); + if (userStrLen < 256) { + char logBuf[128 + 256]; + snprintf(logBuf, sizeof(logBuf), + "vgId:%d %s term:%lu commit:%ld standby:%d replica-num:%d lconfig:%ld sync event %s", pSyncNode->vgId, + syncUtilState2String(pSyncNode->state), pSyncNode->pRaftStore->currentTerm, pSyncNode->commitIndex, + pSyncNode->pRaftCfg->isStandBy, pSyncNode->replicaNum, pSyncNode->pRaftCfg->lastConfigIndex, str); + sDebug("%s", logBuf); + } else { + int len = 128 + userStrLen; + char* s = (char*)taosMemoryMalloc(len); + snprintf(s, len, "vgId:%d %s term:%lu commit:%ld standby:%d replica-num:%d lconfig:%ld sync event %s", + pSyncNode->vgId, syncUtilState2String(pSyncNode->state), pSyncNode->pRaftStore->currentTerm, + pSyncNode->commitIndex, pSyncNode->pRaftCfg->isStandBy, pSyncNode->replicaNum, + pSyncNode->pRaftCfg->lastConfigIndex, str); + sDebug("%s", s); + taosMemoryFree(s); + } +} + char* syncNode2SimpleStr(const SSyncNode* pSyncNode) { int len = 256; char* s = (char*)taosMemoryMalloc(len); snprintf(s, len, - "syncNode: vgId:%d currentTerm:%lu, commitIndex:%ld, state:%d %s, isStandBy:%d, " + "syncNode: vgId:%d, currentTerm:%lu, commitIndex:%ld, state:%d %s, isStandBy:%d, " "electTimerLogicClock:%lu, " "electTimerLogicClockUser:%lu, " "electTimerMS:%d, replicaNum:%d", @@ -1318,11 +1336,10 @@ void syncNodeUpdateConfig(SSyncNode* pSyncNode, SSyncCfg* pNewConfig, SyncIndex SSyncSnapshotSender* oldSenders[TSDB_MAX_REPLICA]; for (int i = 0; i < TSDB_MAX_REPLICA; ++i) { oldSenders[i] = (pSyncNode->senders)[i]; - sDebug("vgId:%d sync event currentTerm:%lu save senders %d, %p, privateTerm:%lu", pSyncNode->vgId, - pSyncNode->pRaftStore->currentTerm, i, oldSenders[i], oldSenders[i]->privateTerm); - if (gRaftDetailLog) { - ; - } + + char* eventLog = snapshotSender2SimpleStr(oldSenders[i], "snapshot sender save old"); + syncNodeEventLog(pSyncNode, eventLog); + taosMemoryFree(eventLog); } // init internal @@ -1371,9 +1388,14 @@ void syncNodeUpdateConfig(SSyncNode* pSyncNode, SSyncCfg* pNewConfig, SyncIndex char host[128]; uint16_t port; syncUtilU642Addr((pSyncNode->replicasId)[i].addr, host, sizeof(host), &port); - sDebug("vgId:%d sync event currentTerm:%lu reset sender for %lu, newIndex:%d, %s:%d, %p, privateTerm:%lu", - pSyncNode->vgId, pSyncNode->pRaftStore->currentTerm, (pSyncNode->replicasId)[i].addr, i, host, port, - oldSenders[j], oldSenders[j]->privateTerm); + + do { + char eventLog[128]; + snprintf(eventLog, sizeof(eventLog), "snapshot sender reset for %lu, newIndex:%d, %s:%d, %p", + (pSyncNode->replicasId)[i].addr, i, host, port, oldSenders[j]); + syncNodeEventLog(pSyncNode, eventLog); + } while (0); + (pSyncNode->senders)[i] = oldSenders[j]; oldSenders[j] = NULL; reset = true; @@ -1381,9 +1403,13 @@ void syncNodeUpdateConfig(SSyncNode* pSyncNode, SSyncCfg* pNewConfig, SyncIndex // reset replicaIndex int32_t oldreplicaIndex = (pSyncNode->senders)[i]->replicaIndex; (pSyncNode->senders)[i]->replicaIndex = i; - sDebug("vgId:%d sync event currentTerm:%lu udpate replicaIndex from %d to %d, %s:%d, %p, reset:%d", - pSyncNode->vgId, pSyncNode->pRaftStore->currentTerm, oldreplicaIndex, i, host, port, - (pSyncNode->senders)[i], reset); + + do { + char eventLog[128]; + snprintf(eventLog, sizeof(eventLog), "snapshot sender udpate replicaIndex from %d to %d, %s:%d, %p, reset:%d", + oldreplicaIndex, i, host, port, (pSyncNode->senders)[i], reset); + syncNodeEventLog(pSyncNode, eventLog); + } while (0); } } } @@ -1392,9 +1418,10 @@ void syncNodeUpdateConfig(SSyncNode* pSyncNode, SSyncCfg* pNewConfig, SyncIndex for (int i = 0; i < TSDB_MAX_REPLICA; ++i) { if ((pSyncNode->senders)[i] == NULL) { (pSyncNode->senders)[i] = snapshotSenderCreate(pSyncNode, i); - sDebug("vgId:%d sync event currentTerm:%lu create new sender %p replicaIndex:%d, privateTerm:%lu", - pSyncNode->vgId, pSyncNode->pRaftStore->currentTerm, (pSyncNode->senders)[i], i, - (pSyncNode->senders)[i]->privateTerm); + + char* eventLog = snapshotSender2SimpleStr((pSyncNode->senders)[i], "snapshot sender create new"); + syncNodeEventLog(pSyncNode, eventLog); + taosMemoryFree(eventLog); } } @@ -1402,8 +1429,13 @@ void syncNodeUpdateConfig(SSyncNode* pSyncNode, SSyncCfg* pNewConfig, SyncIndex for (int i = 0; i < TSDB_MAX_REPLICA; ++i) { if (oldSenders[i] != NULL) { snapshotSenderDestroy(oldSenders[i]); - sDebug("vgId:%d sync event currentTerm:%lu delete old sender %p replicaIndex:%d", pSyncNode->vgId, - pSyncNode->pRaftStore->currentTerm, oldSenders[i], i); + + do { + char eventLog[128]; + snprintf(eventLog, sizeof(eventLog), "snapshot sender delete old %p replica-index:%d", oldSenders[i], i); + syncNodeEventLog(pSyncNode, eventLog); + } while (0); + oldSenders[i] = NULL; } } @@ -1411,24 +1443,6 @@ void syncNodeUpdateConfig(SSyncNode* pSyncNode, SSyncCfg* pNewConfig, SyncIndex bool IamInOld = syncNodeInConfig(pSyncNode, &oldConfig); bool IamInNew = syncNodeInConfig(pSyncNode, pNewConfig); -#if 0 - for (int i = 0; i < oldConfig.replicaNum; ++i) { - if (strcmp((oldConfig.nodeInfo)[i].nodeFqdn, pSyncNode->myNodeInfo.nodeFqdn) == 0 && - (oldConfig.nodeInfo)[i].nodePort == pSyncNode->myNodeInfo.nodePort) { - IamInOld = true; - break; - } - } - - for (int i = 0; i < newConfig->replicaNum; ++i) { - if (strcmp((newConfig->nodeInfo)[i].nodeFqdn, pSyncNode->myNodeInfo.nodeFqdn) == 0 && - (newConfig->nodeInfo)[i].nodePort == pSyncNode->myNodeInfo.nodePort) { - IamInNew = true; - break; - } - } -#endif - *isDrop = true; if (IamInOld && !IamInNew) { *isDrop = true; @@ -1473,12 +1487,6 @@ void syncNodeUpdateTerm(SSyncNode* pSyncNode, SyncTerm term) { } void syncNodeBecomeFollower(SSyncNode* pSyncNode, const char* debugStr) { - sDebug( - "vgId:%d sync event currentTerm:%lu become follower, isStandBy:%d, replicaNum:%d, " - "restoreFinish:%d, %s", - pSyncNode->vgId, pSyncNode->pRaftStore->currentTerm, pSyncNode->pRaftCfg->isStandBy, pSyncNode->replicaNum, - pSyncNode->restoreFinish, debugStr); - // maybe clear leader cache if (pSyncNode->state == TAOS_SYNC_STATE_LEADER) { pSyncNode->leaderCache = EMPTY_RAFT_ID; @@ -1490,6 +1498,21 @@ void syncNodeBecomeFollower(SSyncNode* pSyncNode, const char* debugStr) { // reset elect timer syncNodeResetElectTimer(pSyncNode); + + // trace log + do { + int32_t debugStrLen = strlen(debugStr); + if (debugStrLen < 256) { + char eventLog[256 + 64]; + snprintf(eventLog, sizeof(eventLog), "become follower %s", debugStr); + syncNodeEventLog(pSyncNode, eventLog); + } else { + char* eventLog = taosMemoryMalloc(debugStrLen + 64); + snprintf(eventLog, debugStrLen, "become follower %s", debugStr); + syncNodeEventLog(pSyncNode, eventLog); + taosMemoryFree(eventLog); + } + } while (0); } // TLA+ Spec @@ -1514,10 +1537,6 @@ void syncNodeBecomeLeader(SSyncNode* pSyncNode, const char* debugStr) { // reset restoreFinish pSyncNode->restoreFinish = false; - sDebug("vgId:%d sync event currentTerm:%lu become leader, isStandBy:%d, replicaNum:%d, restoreFinish:%d, %s", - pSyncNode->vgId, pSyncNode->pRaftStore->currentTerm, pSyncNode->pRaftCfg->isStandBy, pSyncNode->replicaNum, - pSyncNode->restoreFinish, debugStr); - // state change pSyncNode->state = TAOS_SYNC_STATE_LEADER; @@ -1563,6 +1582,21 @@ void syncNodeBecomeLeader(SSyncNode* pSyncNode, const char* debugStr) { // start heartbeat timer syncNodeStartHeartbeatTimer(pSyncNode); + + // trace log + do { + int32_t debugStrLen = strlen(debugStr); + if (debugStrLen < 256) { + char eventLog[256 + 64]; + snprintf(eventLog, sizeof(eventLog), "become leader %s", debugStr); + syncNodeEventLog(pSyncNode, eventLog); + } else { + char* eventLog = taosMemoryMalloc(debugStrLen + 64); + snprintf(eventLog, debugStrLen, "become leader %s", debugStr); + syncNodeEventLog(pSyncNode, eventLog); + taosMemoryFree(eventLog); + } + } while (0); } void syncNodeCandidate2Leader(SSyncNode* pSyncNode) { @@ -1818,7 +1852,7 @@ static void syncNodeEqPingTimer(void* param, void* tmrId) { if (pSyncNode->FpEqMsg != NULL) { int32_t code = pSyncNode->FpEqMsg(pSyncNode->msgcb, &rpcMsg); if (code != 0) { - sError("vgId:%d sync enqueue ping msg error, code:%d", pSyncNode->vgId, code); + sError("vgId:%d, sync enqueue ping msg error, code:%d", pSyncNode->vgId, code); rpcFreeCont(rpcMsg.pCont); syncTimeoutDestroy(pSyncMsg); return; @@ -1852,7 +1886,7 @@ static void syncNodeEqElectTimer(void* param, void* tmrId) { if (pSyncNode->FpEqMsg != NULL) { int32_t code = pSyncNode->FpEqMsg(pSyncNode->msgcb, &rpcMsg); if (code != 0) { - sError("vgId:%d sync enqueue elect msg error, code:%d", pSyncNode->vgId, code); + sError("vgId:%d, sync enqueue elect msg error, code:%d", pSyncNode->vgId, code); rpcFreeCont(rpcMsg.pCont); syncTimeoutDestroy(pSyncMsg); return; @@ -1890,7 +1924,7 @@ static void syncNodeEqHeartbeatTimer(void* param, void* tmrId) { if (pSyncNode->FpEqMsg != NULL) { int32_t code = pSyncNode->FpEqMsg(pSyncNode->msgcb, &rpcMsg); if (code != 0) { - sError("vgId:%d sync enqueue timer msg error, code:%d", pSyncNode->vgId, code); + sError("vgId:%d, sync enqueue timer msg error, code:%d", pSyncNode->vgId, code); rpcFreeCont(rpcMsg.pCont); syncTimeoutDestroy(pSyncMsg); return; @@ -2028,8 +2062,9 @@ int32_t syncNodeOnClientRequestCb(SSyncNode* ths, SyncClientRequest* pMsg) { if (ths->pFsm != NULL) { // if (ths->pFsm->FpPreCommitCb != NULL && pEntry->originalRpcType != TDMT_SYNC_NOOP) { if (ths->pFsm->FpPreCommitCb != NULL && syncUtilUserPreCommit(pEntry->originalRpcType)) { - SFsmCbMeta cbMeta; + SFsmCbMeta cbMeta = {0}; cbMeta.index = pEntry->index; + cbMeta.lastConfigIndex = syncNodeGetSnapshotConfigIndex(ths, cbMeta.index); cbMeta.isWeak = pEntry->isWeak; cbMeta.code = 0; cbMeta.state = ths->state; @@ -2050,8 +2085,9 @@ int32_t syncNodeOnClientRequestCb(SSyncNode* ths, SyncClientRequest* pMsg) { if (ths->pFsm != NULL) { // if (ths->pFsm->FpPreCommitCb != NULL && pEntry->originalRpcType != TDMT_SYNC_NOOP) { if (ths->pFsm->FpPreCommitCb != NULL && syncUtilUserPreCommit(pEntry->originalRpcType)) { - SFsmCbMeta cbMeta; + SFsmCbMeta cbMeta = {0}; cbMeta.index = pEntry->index; + cbMeta.lastConfigIndex = syncNodeGetSnapshotConfigIndex(ths, cbMeta.index); cbMeta.isWeak = pEntry->isWeak; cbMeta.code = 1; cbMeta.state = ths->state; @@ -2090,34 +2126,33 @@ const char* syncStr(ESyncState state) { static int32_t syncDoLeaderTransfer(SSyncNode* ths, SRpcMsg* pRpcMsg, SSyncRaftEntry* pEntry) { SyncLeaderTransfer* pSyncLeaderTransfer = syncLeaderTransferFromRpcMsg2(pRpcMsg); - sDebug("vgId:%d sync event currentTerm:%lu begin leader transfer", ths->vgId, ths->pRaftStore->currentTerm); + syncNodeEventLog(ths, "begin leader transfer"); - if (strcmp(pSyncLeaderTransfer->newNodeInfo.nodeFqdn, ths->myNodeInfo.nodeFqdn) == 0 && - pSyncLeaderTransfer->newNodeInfo.nodePort == ths->myNodeInfo.nodePort) { - sDebug("vgId:%d sync event currentTerm:%lu maybe leader transfer to %s:%d %lu", ths->vgId, - ths->pRaftStore->currentTerm, pSyncLeaderTransfer->newNodeInfo.nodeFqdn, - pSyncLeaderTransfer->newNodeInfo.nodePort, pSyncLeaderTransfer->newLeaderId.addr); + bool sameId = syncUtilSameId(&(pSyncLeaderTransfer->newLeaderId), &(ths->myRaftId)); + bool sameNodeInfo = strcmp(pSyncLeaderTransfer->newNodeInfo.nodeFqdn, ths->myNodeInfo.nodeFqdn) == 0 && + pSyncLeaderTransfer->newNodeInfo.nodePort == ths->myNodeInfo.nodePort; + bool same = sameId || sameNodeInfo; + if (same) { // reset elect timer now! int32_t electMS = 1; int32_t ret = syncNodeRestartElectTimer(ths, electMS); ASSERT(ret == 0); + + char eventLog[256]; + snprintf(eventLog, sizeof(eventLog), "maybe leader transfer to %s:%d %lu", + pSyncLeaderTransfer->newNodeInfo.nodeFqdn, pSyncLeaderTransfer->newNodeInfo.nodePort, + pSyncLeaderTransfer->newLeaderId.addr); + syncNodeEventLog(ths, eventLog); } - /* - if (syncUtilSameId(&(pSyncLeaderTransfer->newLeaderId), &(ths->myRaftId))) { - // reset elect timer now! - int32_t electMS = 1; - int32_t ret = syncNodeRestartElectTimer(ths, electMS); - ASSERT(ret == 0); - } - */ if (ths->pFsm->FpLeaderTransferCb != NULL) { - SFsmCbMeta cbMeta; + SFsmCbMeta cbMeta = {0}; cbMeta.code = 0; cbMeta.currentTerm = ths->pRaftStore->currentTerm; cbMeta.flag = 0; cbMeta.index = pEntry->index; + cbMeta.lastConfigIndex = syncNodeGetSnapshotConfigIndex(ths, cbMeta.index); cbMeta.isWeak = pEntry->isWeak; cbMeta.seqNum = pEntry->seqNum; cbMeta.state = ths->state; @@ -2174,8 +2209,14 @@ static int32_t syncNodeConfigChange(SSyncNode* ths, SRpcMsg* pRpcMsg, SSyncRaftE // change isStandBy to normal if (!isDrop) { - char tmpbuf[128]; - snprintf(tmpbuf, sizeof(tmpbuf), "config change from %d to %d", oldSyncCfg.replicaNum, newSyncCfg.replicaNum); + char tmpbuf[512]; + char* oldStr = syncCfg2SimpleStr(&oldSyncCfg); + char* newStr = syncCfg2SimpleStr(&newSyncCfg); + snprintf(tmpbuf, sizeof(tmpbuf), "config change from %d to %d, index:%ld, %s --> %s", oldSyncCfg.replicaNum, + newSyncCfg.replicaNum, pEntry->index, oldStr, newStr); + taosMemoryFree(oldStr); + taosMemoryFree(newStr); + if (ths->state == TAOS_SYNC_STATE_LEADER) { syncNodeBecomeLeader(ths, tmpbuf); } else { @@ -2183,8 +2224,14 @@ static int32_t syncNodeConfigChange(SSyncNode* ths, SRpcMsg* pRpcMsg, SSyncRaftE } } } else { - char tmpbuf[128]; - snprintf(tmpbuf, sizeof(tmpbuf), "config change2 from %d to %d", oldSyncCfg.replicaNum, newSyncCfg.replicaNum); + char tmpbuf[512]; + char* oldStr = syncCfg2SimpleStr(&oldSyncCfg); + char* newStr = syncCfg2SimpleStr(&newSyncCfg); + snprintf(tmpbuf, sizeof(tmpbuf), "config change2 from %d to %d, index:%ld, %s --> %s", oldSyncCfg.replicaNum, + newSyncCfg.replicaNum, pEntry->index, oldStr, newStr); + taosMemoryFree(oldStr); + taosMemoryFree(newStr); + syncNodeBecomeFollower(ths, tmpbuf); } @@ -2203,6 +2250,7 @@ static int32_t syncNodeConfigChange(SSyncNode* ths, SRpcMsg* pRpcMsg, SSyncRaftE cbMeta.code = 0; cbMeta.currentTerm = ths->pRaftStore->currentTerm; cbMeta.index = pEntry->index; + cbMeta.lastConfigIndex = syncNodeGetSnapshotConfigIndex(ths, pEntry->index); cbMeta.term = pEntry->term; cbMeta.newCfg = newSyncCfg; cbMeta.oldCfg = oldSyncCfg; @@ -2218,8 +2266,10 @@ static int32_t syncNodeConfigChange(SSyncNode* ths, SRpcMsg* pRpcMsg, SSyncRaftE int32_t syncNodeCommit(SSyncNode* ths, SyncIndex beginIndex, SyncIndex endIndex, uint64_t flag) { int32_t code = 0; ESyncState state = flag; - sDebug("vgId:%d sync event currentTerm:%lu commit by wal from index:%" PRId64 " to index:%" PRId64 ", %s", ths->vgId, - ths->pRaftStore->currentTerm, beginIndex, endIndex, syncUtilState2String(state)); + + char eventLog[128]; + snprintf(eventLog, sizeof(eventLog), "commit by wal from index:%ld to index:%ld", beginIndex, endIndex); + syncNodeEventLog(ths, eventLog); // execute fsm if (ths->pFsm != NULL) { @@ -2235,8 +2285,9 @@ int32_t syncNodeCommit(SSyncNode* ths, SyncIndex beginIndex, SyncIndex endIndex, // user commit if (ths->pFsm->FpCommitCb != NULL && syncUtilUserCommit(pEntry->originalRpcType)) { - SFsmCbMeta cbMeta; + SFsmCbMeta cbMeta = {0}; cbMeta.index = pEntry->index; + cbMeta.lastConfigIndex = syncNodeGetSnapshotConfigIndex(ths, cbMeta.index); cbMeta.isWeak = pEntry->isWeak; cbMeta.code = 0; cbMeta.state = ths->state; @@ -2250,6 +2301,8 @@ int32_t syncNodeCommit(SSyncNode* ths, SyncIndex beginIndex, SyncIndex endIndex, // config change if (pEntry->originalRpcType == TDMT_SYNC_CONFIG_CHANGE) { + raftCfgAddConfigIndex(ths->pRaftCfg, pEntry->index); + raftCfgPersist(ths->pRaftCfg); code = syncNodeConfigChange(ths, &rpcMsg, pEntry); ASSERT(code == 0); } @@ -2261,14 +2314,17 @@ int32_t syncNodeCommit(SSyncNode* ths, SyncIndex beginIndex, SyncIndex endIndex, } // restore finish + // if only snapshot, a noop entry will be append, so syncLogLastIndex is always ok if (pEntry->index == ths->pLogStore->syncLogLastIndex(ths->pLogStore)) { if (ths->restoreFinish == false) { if (ths->pFsm->FpRestoreFinishCb != NULL) { ths->pFsm->FpRestoreFinishCb(ths->pFsm); } ths->restoreFinish = true; - sDebug("vgId:%d sync event currentTerm:%lu restore finish, %s, index:%ld", ths->vgId, - ths->pRaftStore->currentTerm, syncUtilState2String(ths->state), pEntry->index); + + char eventLog[128]; + snprintf(eventLog, sizeof(eventLog), "restore finish, index:%ld", pEntry->index); + syncNodeEventLog(ths, eventLog); } } diff --git a/source/libs/sync/src/syncRaftCfg.c b/source/libs/sync/src/syncRaftCfg.c index 45e00aca2c33be3a5f665d6eaaf7c43043c6a6e0..a8242d06e39cbcb30a175cb955454ee56292cb2e 100644 --- a/source/libs/sync/src/syncRaftCfg.c +++ b/source/libs/sync/src/syncRaftCfg.c @@ -66,6 +66,13 @@ int32_t raftCfgPersist(SRaftCfg *pRaftCfg) { return 0; } +int32_t raftCfgAddConfigIndex(SRaftCfg *pRaftCfg, SyncIndex configIndex) { + ASSERT(pRaftCfg->configIndexCount <= MAX_CONFIG_INDEX_COUNT); + (pRaftCfg->configIndexArr)[pRaftCfg->configIndexCount] = configIndex; + ++(pRaftCfg->configIndexCount); + return 0; +} + cJSON *syncCfg2Json(SSyncCfg *pSyncCfg) { char u64buf[128] = {0}; cJSON *pRoot = cJSON_CreateObject(); @@ -85,20 +92,38 @@ cJSON *syncCfg2Json(SSyncCfg *pSyncCfg) { } return pRoot; - /* - cJSON *pJson = cJSON_CreateObject(); - cJSON_AddItemToObject(pJson, "SSyncCfg", pRoot); - return pJson; - */ } char *syncCfg2Str(SSyncCfg *pSyncCfg) { cJSON *pJson = syncCfg2Json(pSyncCfg); - char * serialized = cJSON_Print(pJson); + char *serialized = cJSON_Print(pJson); cJSON_Delete(pJson); return serialized; } +char *syncCfg2SimpleStr(SSyncCfg *pSyncCfg) { + int32_t len = 512; + char *s = taosMemoryMalloc(len); + memset(s, 0, len); + + snprintf(s, len, "{replica-num:%d, my-index:%d, ", pSyncCfg->replicaNum, pSyncCfg->myIndex); + char *p = s + strlen(s); + for (int i = 0; i < pSyncCfg->replicaNum; ++i) { + /* + if (p + 128 + 32 > s + len) { + break; + } + */ + char buf[128 + 32]; + snprintf(buf, sizeof(buf), "%s:%d, ", pSyncCfg->nodeInfo[i].nodeFqdn, pSyncCfg->nodeInfo[i].nodePort); + strncpy(p, buf, sizeof(buf)); + p = s + strlen(s); + } + strcpy(p - 2, "}"); + + return s; +} + int32_t syncCfgFromJson(const cJSON *pRoot, SSyncCfg *pSyncCfg) { memset(pSyncCfg, 0, sizeof(SSyncCfg)); // cJSON *pJson = cJSON_GetObjectItem(pRoot, "SSyncCfg"); @@ -154,6 +179,16 @@ cJSON *raftCfg2Json(SRaftCfg *pRaftCfg) { snprintf(buf64, sizeof(buf64), "%ld", pRaftCfg->lastConfigIndex); cJSON_AddStringToObject(pRoot, "lastConfigIndex", buf64); + cJSON_AddNumberToObject(pRoot, "configIndexCount", pRaftCfg->configIndexCount); + cJSON *pIndexArr = cJSON_CreateArray(); + cJSON_AddItemToObject(pRoot, "configIndexArr", pIndexArr); + for (int i = 0; i < pRaftCfg->configIndexCount; ++i) { + snprintf(buf64, sizeof(buf64), "%ld", (pRaftCfg->configIndexArr)[i]); + cJSON *pIndexObj = cJSON_CreateObject(); + cJSON_AddStringToObject(pIndexObj, "index", buf64); + cJSON_AddItemToArray(pIndexArr, pIndexObj); + } + cJSON *pJson = cJSON_CreateObject(); cJSON_AddItemToObject(pJson, "RaftCfg", pRoot); return pJson; @@ -161,7 +196,7 @@ cJSON *raftCfg2Json(SRaftCfg *pRaftCfg) { char *raftCfg2Str(SRaftCfg *pRaftCfg) { cJSON *pJson = raftCfg2Json(pRaftCfg); - char * serialized = cJSON_Print(pJson); + char *serialized = cJSON_Print(pJson); cJSON_Delete(pJson); return serialized; } @@ -177,6 +212,9 @@ int32_t raftCfgCreateFile(SSyncCfg *pCfg, SRaftCfgMeta meta, const char *path) { raftCfg.isStandBy = meta.isStandBy; raftCfg.snapshotEnable = meta.snapshotEnable; raftCfg.lastConfigIndex = meta.lastConfigIndex; + raftCfg.configIndexCount = 1; + memset(raftCfg.configIndexArr, 0, sizeof(raftCfg.configIndexArr)); + raftCfg.configIndexArr[0] = -1; char *s = raftCfg2Str(&raftCfg); char buf[CONFIG_FILE_LEN] = {0}; @@ -207,7 +245,24 @@ int32_t raftCfgFromJson(const cJSON *pRoot, SRaftCfg *pRaftCfg) { cJSON *pJsonLastConfigIndex = cJSON_GetObjectItem(pJson, "lastConfigIndex"); pRaftCfg->lastConfigIndex = atoll(cJSON_GetStringValue(pJsonLastConfigIndex)); - cJSON * pJsonSyncCfg = cJSON_GetObjectItem(pJson, "SSyncCfg"); + cJSON *pJsonConfigIndexCount = cJSON_GetObjectItem(pJson, "configIndexCount"); + pRaftCfg->configIndexCount = cJSON_GetNumberValue(pJsonConfigIndexCount); + + cJSON *pIndexArr = cJSON_GetObjectItem(pJson, "configIndexArr"); + int arraySize = cJSON_GetArraySize(pIndexArr); + assert(arraySize == pRaftCfg->configIndexCount); + + memset(pRaftCfg->configIndexArr, 0, sizeof(pRaftCfg->configIndexArr)); + for (int i = 0; i < arraySize; ++i) { + cJSON *pIndexObj = cJSON_GetArrayItem(pIndexArr, i); + assert(pIndexObj != NULL); + + cJSON *pIndex = cJSON_GetObjectItem(pIndexObj, "index"); + assert(cJSON_IsString(pIndex)); + (pRaftCfg->configIndexArr)[i] = atoll(pIndex->valuestring); + } + + cJSON *pJsonSyncCfg = cJSON_GetObjectItem(pJson, "SSyncCfg"); int32_t code = syncCfgFromJson(pJsonSyncCfg, &(pRaftCfg->cfg)); ASSERT(code == 0); @@ -252,6 +307,12 @@ void syncCfgLog2(char *s, SSyncCfg *pCfg) { taosMemoryFree(serialized); } +void syncCfgLog3(char *s, SSyncCfg *pCfg) { + char *serialized = syncCfg2SimpleStr(pCfg); + sTrace("syncCfgLog3 | len:%lu | %s | %s", strlen(serialized), s, serialized); + taosMemoryFree(serialized); +} + void raftCfgPrint(SRaftCfg *pCfg) { char *serialized = raftCfg2Str(pCfg); printf("raftCfgPrint | len:%lu | %s \n", strlen(serialized), serialized); diff --git a/source/libs/sync/src/syncRaftLog.c b/source/libs/sync/src/syncRaftLog.c index 31ac44fa81b91a7bd13af621f1e291b5ac60ca7a..79d9b329c10bf8e2fc6db6ee7865bd31fbc764ae 100644 --- a/source/libs/sync/src/syncRaftLog.c +++ b/source/libs/sync/src/syncRaftLog.c @@ -163,10 +163,10 @@ static int32_t raftLogAppendEntry(struct SSyncLogStore* pLogStore, SSyncRaftEntr walFsync(pWal, true); - sDebug("vgId:%d sync event currentTerm:%lu write index:%ld, %s, isStandBy:%d, msgType:%s,%d, originalRpcType:%s,%d", - pData->pSyncNode->vgId, pData->pSyncNode->pRaftStore->currentTerm, pEntry->index, - syncUtilState2String(pData->pSyncNode->state), pData->pSyncNode->pRaftCfg->isStandBy, - TMSG_INFO(pEntry->msgType), pEntry->msgType, TMSG_INFO(pEntry->originalRpcType), pEntry->originalRpcType); + char eventLog[128]; + snprintf(eventLog, sizeof(eventLog), "write index:%ld, type:%s,%d, type2:%s,%d", pEntry->index, + TMSG_INFO(pEntry->msgType), pEntry->msgType, TMSG_INFO(pEntry->originalRpcType), pEntry->originalRpcType); + syncNodeEventLog(pData->pSyncNode, eventLog); return code; } @@ -318,15 +318,13 @@ int32_t logStoreAppendEntry(SSyncLogStore* pLogStore, SSyncRaftEntry* pEntry) { linuxErrMsg); ASSERT(0); } - // assert(code == 0); walFsync(pWal, true); - sDebug( - "vgId:%d sync event currentTerm:%lu old write index:%ld, %s, isStandBy:%d, msgType:%s,%d, originalRpcType:%s,%d", - pData->pSyncNode->vgId, pData->pSyncNode->pRaftStore->currentTerm, pEntry->index, - syncUtilState2String(pData->pSyncNode->state), pData->pSyncNode->pRaftCfg->isStandBy, TMSG_INFO(pEntry->msgType), - pEntry->msgType, TMSG_INFO(pEntry->originalRpcType), pEntry->originalRpcType); + char eventLog[128]; + snprintf(eventLog, sizeof(eventLog), "old write index:%ld, type:%s,%d, type2:%s,%d", pEntry->index, + TMSG_INFO(pEntry->msgType), pEntry->msgType, TMSG_INFO(pEntry->originalRpcType), pEntry->originalRpcType); + syncNodeEventLog(pData->pSyncNode, eventLog); return code; } diff --git a/source/libs/sync/src/syncRespMgr.c b/source/libs/sync/src/syncRespMgr.c index 5b127793d644a2387d13e53fc855455bb0266a07..354575d29e6be2b4abf5a88743572c869d3344f7 100644 --- a/source/libs/sync/src/syncRespMgr.c +++ b/source/libs/sync/src/syncRespMgr.c @@ -46,9 +46,11 @@ int64_t syncRespMgrAdd(SSyncRespMgr *pObj, SRespStub *pStub) { taosHashPut(pObj->pRespHash, &keyCode, sizeof(keyCode), pStub, sizeof(SRespStub)); SSyncNode *pSyncNode = pObj->data; - sDebug("vgId:%d sync event currentTerm:%lu resp mgr add, msgType:%s,%d seq:%lu handle:%p ahandle:%p", pSyncNode->vgId, - pSyncNode->pRaftStore->currentTerm, TMSG_INFO(pStub->rpcMsg.msgType), pStub->rpcMsg.msgType, keyCode, - pStub->rpcMsg.info.handle, pStub->rpcMsg.info.ahandle); + char eventLog[128]; + snprintf(eventLog, sizeof(eventLog), "resp mgr add, type:%s,%d, seq:%lu, handle:%p, ahandle:%p", + TMSG_INFO(pStub->rpcMsg.msgType), pStub->rpcMsg.msgType, keyCode, pStub->rpcMsg.info.handle, + pStub->rpcMsg.info.ahandle); + syncNodeEventLog(pSyncNode, eventLog); taosThreadMutexUnlock(&(pObj->mutex)); return keyCode; @@ -71,9 +73,11 @@ int32_t syncRespMgrGet(SSyncRespMgr *pObj, uint64_t index, SRespStub *pStub) { memcpy(pStub, pTmp, sizeof(SRespStub)); SSyncNode *pSyncNode = pObj->data; - sDebug("vgId:%d sync event currentTerm:%lu resp mgr get, msgType:%s,%d seq:%lu handle:%p ahandle:%p", - pSyncNode->vgId, pSyncNode->pRaftStore->currentTerm, TMSG_INFO(pStub->rpcMsg.msgType), pStub->rpcMsg.msgType, - index, pStub->rpcMsg.info.handle, pStub->rpcMsg.info.ahandle); + char eventLog[128]; + snprintf(eventLog, sizeof(eventLog), "resp mgr get, type:%s,%d, seq:%lu, handle:%p, ahandle:%p", + TMSG_INFO(pStub->rpcMsg.msgType), pStub->rpcMsg.msgType, index, pStub->rpcMsg.info.handle, + pStub->rpcMsg.info.ahandle); + syncNodeEventLog(pSyncNode, eventLog); taosThreadMutexUnlock(&(pObj->mutex)); return 1; // get one object @@ -90,9 +94,11 @@ int32_t syncRespMgrGetAndDel(SSyncRespMgr *pObj, uint64_t index, SRespStub *pStu memcpy(pStub, pTmp, sizeof(SRespStub)); SSyncNode *pSyncNode = pObj->data; - sDebug("vgId:%d sync event currentTerm:%lu resp mgr get and del, msgType:%s,%d seq:%lu handle:%p ahandle:%p", - pSyncNode->vgId, pSyncNode->pRaftStore->currentTerm, TMSG_INFO(pStub->rpcMsg.msgType), pStub->rpcMsg.msgType, - index, pStub->rpcMsg.info.handle, pStub->rpcMsg.info.ahandle); + char eventLog[128]; + snprintf(eventLog, sizeof(eventLog), "resp mgr get-and-del, type:%s,%d, seq:%lu, handle:%p, ahandle:%p", + TMSG_INFO(pStub->rpcMsg.msgType), pStub->rpcMsg.msgType, index, pStub->rpcMsg.info.handle, + pStub->rpcMsg.info.ahandle); + syncNodeEventLog(pSyncNode, eventLog); taosHashRemove(pObj->pRespHash, &index, sizeof(index)); taosThreadMutexUnlock(&(pObj->mutex)); diff --git a/source/libs/sync/src/syncSnapshot.c b/source/libs/sync/src/syncSnapshot.c index d3785060d4acae4f5a7f64b063fdbe9844cc85ed..d1bfc39d1eb6ef475937a64b8f734be9eba5a1d7 100644 --- a/source/libs/sync/src/syncSnapshot.c +++ b/source/libs/sync/src/syncSnapshot.c @@ -50,6 +50,7 @@ SSyncSnapshotSender *snapshotSenderCreate(SSyncNode *pSyncNode, int32_t replicaI } else { sError("snapshotSenderCreate cannot create sender"); } + return pSender; } @@ -84,6 +85,10 @@ void snapshotSenderStart(SSyncSnapshotSender *pSender) { // get current snapshot info pSender->pSyncNode->pFsm->FpGetSnapshot(pSender->pSyncNode->pFsm, &(pSender->snapshot)); + + sTrace("snapshotSenderStart lastApplyIndex:%ld, lastApplyTerm:%lu, lastConfigIndex:%ld", + pSender->snapshot.lastApplyIndex, pSender->snapshot.lastApplyTerm, pSender->snapshot.lastConfigIndex); + if (pSender->snapshot.lastConfigIndex != SYNC_INDEX_INVALID) { /* SSyncRaftEntry *pEntry = NULL; @@ -134,86 +139,13 @@ void snapshotSenderStart(SSyncSnapshotSender *pSender) { syncSnapshotSend2RpcMsg(pMsg, &rpcMsg); syncNodeSendMsgById(&(pMsg->destId), pSender->pSyncNode, &rpcMsg); - char host[128]; - uint16_t port; - syncUtilU642Addr(pSender->pSyncNode->replicasId[pSender->replicaIndex].addr, host, sizeof(host), &port); - - if (gRaftDetailLog) { - char *msgStr = syncSnapshotSend2Str(pMsg); - sDebug( - "vgId:%d sync event currentTerm:%lu snapshot send to %s:%d begin seq:%d ack:%d lastApplyIndex:%ld " - "lastApplyTerm:%lu " - "lastConfigIndex:%ld privateTerm:%lu send " - "msg:%s", - pSender->pSyncNode->vgId, pSender->pSyncNode->pRaftStore->currentTerm, host, port, pSender->seq, pSender->ack, - pSender->snapshot.lastApplyIndex, pSender->snapshot.lastApplyTerm, pSender->snapshot.lastConfigIndex, - pSender->privateTerm, msgStr); - taosMemoryFree(msgStr); - } else { - sDebug( - "vgId:%d sync event currentTerm:%lu snapshot send to %s:%d begin seq:%d ack:%d lastApplyIndex:%ld " - "lastApplyTerm:%lu " - "lastConfigIndex:%ld privateTerm:%lu", - pSender->pSyncNode->vgId, pSender->pSyncNode->pRaftStore->currentTerm, host, port, pSender->seq, pSender->ack, - pSender->snapshot.lastApplyIndex, pSender->snapshot.lastApplyTerm, pSender->snapshot.lastConfigIndex, - pSender->privateTerm); - } + char *eventLog = snapshotSender2SimpleStr(pSender, "snapshot sender send"); + syncNodeEventLog(pSender->pSyncNode, eventLog); + taosMemoryFree(eventLog); syncSnapshotSendDestroy(pMsg); } -#if 0 -// when entry in snapshot, start sender -void snapshotSenderStart(SSyncSnapshotSender *pSender) { - if (!(pSender->start)) { - // start - snapshotSenderDoStart(pSender); - pSender->start = true; - } else { - // already start - ASSERT(pSender->pSyncNode->pRaftStore->currentTerm >= pSender->term); - - // if current term is higher, need start again - if (pSender->pSyncNode->pRaftStore->currentTerm > pSender->term) { - // force peer rollback - SyncSnapshotSend *pMsg = syncSnapshotSendBuild(0, pSender->pSyncNode->vgId); - pMsg->srcId = pSender->pSyncNode->myRaftId; - pMsg->destId = (pSender->pSyncNode->replicasId)[pSender->replicaIndex]; - pMsg->term = pSender->pSyncNode->pRaftStore->currentTerm; - pMsg->lastIndex = pSender->snapshot.lastApplyIndex; - pMsg->lastTerm = pSender->snapshot.lastApplyTerm; - pMsg->seq = SYNC_SNAPSHOT_SEQ_FORCE_CLOSE; - - SRpcMsg rpcMsg; - syncSnapshotSend2RpcMsg(pMsg, &rpcMsg); - syncNodeSendMsgById(&(pMsg->destId), pSender->pSyncNode, &rpcMsg); - - char *msgStr = syncSnapshotSend2Str(pMsg); - sTrace("snapshot send force close seq:%d ack:%d send msg:%s", pSender->seq, pSender->ack, msgStr); - taosMemoryFree(msgStr); - - syncSnapshotSendDestroy(pMsg); - - // close reader - int32_t ret = pSender->pSyncNode->pFsm->FpSnapshotStopRead(pSender->pSyncNode->pFsm, pSender->pReader); - ASSERT(ret == 0); - pSender->pReader = NULL; - - // start again - snapshotSenderDoStart(pSender); - pSender->start = true; - } else { - // current term, do nothing - ASSERT(pSender->pSyncNode->pRaftStore->currentTerm == pSender->term); - } - } - - char *s = snapshotSender2Str(pSender); - sInfo("snapshotSenderStart %s", s); - taosMemoryFree(s); -} -#endif - void snapshotSenderStop(SSyncSnapshotSender *pSender) { if (pSender->pReader != NULL) { int32_t ret = pSender->pSyncNode->pFsm->FpSnapshotStopRead(pSender->pSyncNode->pFsm, pSender->pReader); @@ -275,39 +207,15 @@ int32_t snapshotSend(SSyncSnapshotSender *pSender) { syncSnapshotSend2RpcMsg(pMsg, &rpcMsg); syncNodeSendMsgById(&(pMsg->destId), pSender->pSyncNode, &rpcMsg); - char host[128]; - uint16_t port; - syncUtilU642Addr(pSender->pSyncNode->replicasId[pSender->replicaIndex].addr, host, sizeof(host), &port); - if (pSender->seq == SYNC_SNAPSHOT_SEQ_END) { - if (gRaftDetailLog) { - char *msgStr = syncSnapshotSend2Str(pMsg); - sDebug( - "vgId:%d sync event currentTerm:%lu snapshot send to %s:%d finish seq:%d ack:%d lastApplyIndex:%ld " - "lastApplyTerm:%lu " - "lastConfigIndex:%ld privateTerm:%lu send " - "msg:%s", - pSender->pSyncNode->vgId, pSender->pSyncNode->pRaftStore->currentTerm, host, port, pSender->seq, pSender->ack, - pSender->snapshot.lastApplyIndex, pSender->snapshot.lastApplyTerm, pSender->snapshot.lastConfigIndex, - pSender->privateTerm, msgStr); - taosMemoryFree(msgStr); - } else { - sDebug( - "vgId:%d sync event currentTerm:%lu snapshot send to %s:%d finish seq:%d ack:%d lastApplyIndex:%ld " - "lastApplyTerm:%lu " - "lastConfigIndex:%ld privateTerm:%lu", - pSender->pSyncNode->vgId, pSender->pSyncNode->pRaftStore->currentTerm, host, port, pSender->seq, pSender->ack, - pSender->snapshot.lastApplyIndex, pSender->snapshot.lastApplyTerm, pSender->snapshot.lastConfigIndex, - pSender->privateTerm); - } + char *eventLog = snapshotSender2SimpleStr(pSender, "snapshot sender finish"); + syncNodeEventLog(pSender->pSyncNode, eventLog); + taosMemoryFree(eventLog); + } else { - sDebug( - "vgId:%d sync event currentTerm:%lu snapshot send to %s:%d sending seq:%d ack:%d lastApplyIndex:%ld " - "lastApplyTerm:%lu " - "lastConfigIndex:%ld privateTerm:%lu", - pSender->pSyncNode->vgId, pSender->pSyncNode->pRaftStore->currentTerm, host, port, pSender->seq, pSender->ack, - pSender->snapshot.lastApplyIndex, pSender->snapshot.lastApplyTerm, pSender->snapshot.lastConfigIndex, - pSender->privateTerm); + char *eventLog = snapshotSender2SimpleStr(pSender, "snapshot sender sending"); + syncNodeEventLog(pSender->pSyncNode, eventLog); + taosMemoryFree(eventLog); } syncSnapshotSendDestroy(pMsg); @@ -332,22 +240,9 @@ int32_t snapshotReSend(SSyncSnapshotSender *pSender) { syncSnapshotSend2RpcMsg(pMsg, &rpcMsg); syncNodeSendMsgById(&(pMsg->destId), pSender->pSyncNode, &rpcMsg); - char host[128]; - uint16_t port; - syncUtilU642Addr(pSender->pSyncNode->replicasId[pSender->replicaIndex].addr, host, sizeof(host), &port); - - if (gRaftDetailLog) { - char *msgStr = syncSnapshotSend2Str(pMsg); - sDebug( - "vgId:%d sync event currentTerm:%lu snapshot send to %s:%d resend seq:%d ack:%d privateTerm:%lu send msg:%s", - pSender->pSyncNode->vgId, pSender->pSyncNode->pRaftStore->currentTerm, host, port, pSender->seq, pSender->ack, - pSender->privateTerm, msgStr); - taosMemoryFree(msgStr); - } else { - sDebug("vgId:%d sync event currentTerm:%lu snapshot send to %s:%d resend seq:%d ack:%d privateTerm:%lu", - pSender->pSyncNode->vgId, pSender->pSyncNode->pRaftStore->currentTerm, host, port, pSender->seq, - pSender->ack, pSender->privateTerm); - } + char *eventLog = snapshotSender2SimpleStr(pSender, "snapshot sender resend"); + syncNodeEventLog(pSender->pSyncNode, eventLog); + taosMemoryFree(eventLog); syncSnapshotSendDestroy(pMsg); } @@ -411,6 +306,23 @@ char *snapshotSender2Str(SSyncSnapshotSender *pSender) { return serialized; } +char *snapshotSender2SimpleStr(SSyncSnapshotSender *pSender, char *event) { + int32_t len = 256; + char *s = taosMemoryMalloc(len); + + SRaftId destId = pSender->pSyncNode->replicasId[pSender->replicaIndex]; + char host[128]; + uint16_t port; + syncUtilU642Addr(destId.addr, host, sizeof(host), &port); + + snprintf(s, len, "%s %p laindex:%ld laterm:%lu lcindex:%ld seq:%d ack:%d finish:%d pterm:%lu replica-index:%d %s:%d", + event, pSender, pSender->snapshot.lastApplyIndex, pSender->snapshot.lastApplyTerm, + pSender->snapshot.lastConfigIndex, pSender->seq, pSender->ack, pSender->finish, pSender->privateTerm, + pSender->replicaIndex, host, port); + + return s; +} + // ------------------------------------- SSyncSnapshotReceiver *snapshotReceiverCreate(SSyncNode *pSyncNode, SRaftId fromId) { bool condition = (pSyncNode->pFsm->FpSnapshotStartWrite != NULL) && (pSyncNode->pFsm->FpSnapshotStopWrite != NULL) && @@ -556,6 +468,21 @@ char *snapshotReceiver2Str(SSyncSnapshotReceiver *pReceiver) { return serialized; } +char *snapshotReceiver2SimpleStr(SSyncSnapshotReceiver *pReceiver, char *event) { + int32_t len = 256; + char *s = taosMemoryMalloc(len); + + SRaftId fromId = pReceiver->fromId; + char host[128]; + uint16_t port; + syncUtilU642Addr(fromId.addr, host, sizeof(host), &port); + + snprintf(s, len, "%s %p start:%d ack:%d term:%lu pterm:%lu %s:%d ", event, pReceiver, pReceiver->start, + pReceiver->ack, pReceiver->term, pReceiver->privateTerm, host, port); + + return s; +} + // receiver do something int32_t syncNodeOnSnapshotSendCb(SSyncNode *pSyncNode, SyncSnapshotSend *pMsg) { // get receiver @@ -572,25 +499,9 @@ int32_t syncNodeOnSnapshotSendCb(SSyncNode *pSyncNode, SyncSnapshotSend *pMsg) { pReceiver->ack = pMsg->seq; needRsp = true; - char host[128]; - uint16_t port; - syncUtilU642Addr(pMsg->srcId.addr, host, sizeof(host), &port); - - if (gRaftDetailLog) { - char *msgStr = syncSnapshotSend2Str(pMsg); - sDebug( - "vgId:%d sync event currentTerm:%lu snapshot recv from %s:%d begin ack:%d, lastIndex:%ld, lastTerm:%lu, " - "lastConfigIndex:%ld, privateTerm:%lu, recv msg:%s", - pSyncNode->vgId, pSyncNode->pRaftStore->currentTerm, host, port, pReceiver->ack, pMsg->lastIndex, - pMsg->lastTerm, pMsg->lastConfigIndex, pReceiver->privateTerm, msgStr); - taosMemoryFree(msgStr); - } else { - sDebug( - "vgId:%d sync event currentTerm:%lu snapshot recv from %s:%d begin ack:%d, lastIndex:%ld, lastTerm:%lu, " - "lastConfigIndex:%ld privateTerm:%lu", - pSyncNode->vgId, pSyncNode->pRaftStore->currentTerm, host, port, pReceiver->ack, pMsg->lastIndex, - pMsg->lastTerm, pMsg->lastConfigIndex, pReceiver->privateTerm); - } + char *eventLog = snapshotReceiver2SimpleStr(pReceiver, "snapshot receiver begin"); + syncNodeEventLog(pSyncNode, eventLog); + taosMemoryFree(eventLog); } else if (pMsg->seq == SYNC_SNAPSHOT_SEQ_END) { // end, finish FSM @@ -602,48 +513,42 @@ int32_t syncNodeOnSnapshotSendCb(SSyncNode *pSyncNode, SyncSnapshotSend *pMsg) { // maybe update lastconfig if (pMsg->lastConfigIndex >= SYNC_INDEX_BEGIN) { - int32_t oldReplicaNum = pSyncNode->replicaNum; + // int32_t oldReplicaNum = pSyncNode->replicaNum; + SSyncCfg oldSyncCfg = pSyncNode->pRaftCfg->cfg; // update new config myIndex SSyncCfg newSyncCfg = pMsg->lastConfig; syncNodeUpdateNewConfigIndex(pSyncNode, &newSyncCfg); bool IamInNew = syncNodeInConfig(pSyncNode, &newSyncCfg); -#if 0 - // update new config myIndex - bool IamInNew = false; - SSyncCfg newSyncCfg = pMsg->lastConfig; - for (int i = 0; i < newSyncCfg.replicaNum; ++i) { - if (strcmp(pSyncNode->myNodeInfo.nodeFqdn, (newSyncCfg.nodeInfo)[i].nodeFqdn) == 0 && - pSyncNode->myNodeInfo.nodePort == (newSyncCfg.nodeInfo)[i].nodePort) { - newSyncCfg.myIndex = i; - IamInNew = true; - break; - } - } -#endif - - bool isDrop; + bool isDrop = false; if (IamInNew) { - sDebug( - "vgId:%d sync event currentTerm:%lu update config by snapshot, lastIndex:%ld, lastTerm:%lu, " - "lastConfigIndex:%ld ", - pSyncNode->vgId, pSyncNode->pRaftStore->currentTerm, pMsg->lastIndex, pMsg->lastTerm, - pMsg->lastConfigIndex); + char eventLog[128]; + snprintf(eventLog, sizeof(eventLog), + "update config by snapshot, lastIndex:%ld, lastTerm:%lu, lastConfigIndex:%ld", pMsg->lastIndex, + pMsg->lastTerm, pMsg->lastConfigIndex); + syncNodeEventLog(pSyncNode, eventLog); + syncNodeUpdateConfig(pSyncNode, &newSyncCfg, pMsg->lastConfigIndex, &isDrop); + } else { - sDebug( - "vgId:%d sync event currentTerm:%lu do not update config by snapshot, I am not in newCfg, " - "lastIndex:%ld, lastTerm:%lu, " - "lastConfigIndex:%ld ", - pSyncNode->vgId, pSyncNode->pRaftStore->currentTerm, pMsg->lastIndex, pMsg->lastTerm, - pMsg->lastConfigIndex); + char eventLog[128]; + snprintf(eventLog, sizeof(eventLog), + "do not update config by snapshot, not in new, lastIndex:%ld, lastTerm:%lu, lastConfigIndex:%ld", + pMsg->lastIndex, pMsg->lastTerm, pMsg->lastConfigIndex); + syncNodeEventLog(pSyncNode, eventLog); } // change isStandBy to normal if (!isDrop) { - char tmpbuf[128]; - snprintf(tmpbuf, sizeof(tmpbuf), "config change3 from %d to %d", oldReplicaNum, newSyncCfg.replicaNum); + char tmpbuf[512]; + char *oldStr = syncCfg2SimpleStr(&oldSyncCfg); + char *newStr = syncCfg2SimpleStr(&newSyncCfg); + snprintf(tmpbuf, sizeof(tmpbuf), "config change3 from %d to %d, index:%ld, %s --> %s", + oldSyncCfg.replicaNum, newSyncCfg.replicaNum, pMsg->lastConfigIndex, oldStr, newStr); + taosMemoryFree(oldStr); + taosMemoryFree(newStr); + if (pSyncNode->state == TAOS_SYNC_STATE_LEADER) { syncNodeBecomeLeader(pSyncNode, tmpbuf); } else { @@ -655,77 +560,33 @@ int32_t syncNodeOnSnapshotSendCb(SSyncNode *pSyncNode, SyncSnapshotSend *pMsg) { SSnapshot snapshot; pSyncNode->pFsm->FpGetSnapshot(pSyncNode->pFsm, &snapshot); - char host[128]; - uint16_t port; - syncUtilU642Addr(pMsg->srcId.addr, host, sizeof(host), &port); - - if (gRaftDetailLog) { - char *logSimpleStr = logStoreSimple2Str(pSyncNode->pLogStore); - sDebug( - "vgId:%d sync event currentTerm:%lu snapshot recv from %s:%d finish, update log begin index:%ld, " - "snapshot.lastApplyIndex:%ld, " - "snapshot.lastApplyTerm:%lu, snapshot.lastConfigIndex:%ld, privateTerm:%lu, raft log:%s", - pSyncNode->vgId, pSyncNode->pRaftStore->currentTerm, host, port, pMsg->lastIndex + 1, - snapshot.lastApplyIndex, snapshot.lastApplyTerm, snapshot.lastConfigIndex, pReceiver->privateTerm, - logSimpleStr); - taosMemoryFree(logSimpleStr); - } else { - sDebug( - "vgId:%d sync event currentTerm:%lu snapshot recv from %s:%d finish, update log begin index:%ld, " - "snapshot.lastApplyIndex:%ld, " - "snapshot.lastApplyTerm:%lu, snapshot.lastConfigIndex:%ld, privateTerm:%lu", - pSyncNode->vgId, pSyncNode->pRaftStore->currentTerm, host, port, pMsg->lastIndex + 1, - snapshot.lastApplyIndex, snapshot.lastApplyTerm, snapshot.lastConfigIndex, pReceiver->privateTerm); - } + do { + char *eventLog = snapshotReceiver2SimpleStr(pReceiver, "snapshot receiver finish"); + syncNodeEventLog(pSyncNode, eventLog); + taosMemoryFree(eventLog); + } while (0); pReceiver->pWriter = NULL; snapshotReceiverStop(pReceiver, true); pReceiver->ack = pMsg->seq; needRsp = true; - if (gRaftDetailLog) { - char *msgStr = syncSnapshotSend2Str(pMsg); - sDebug( - "vgId:%d sync event currentTerm:%lu snapshot recv from %s:%d end ack:%d, lastIndex:%ld, lastTerm:%lu, " - "lastConfigIndex:%ld, privateTerm:%lu, recv msg:%s", - pReceiver->pSyncNode->vgId, pReceiver->pSyncNode->pRaftStore->currentTerm, host, port, pReceiver->ack, - pMsg->lastIndex, pMsg->lastTerm, pMsg->lastConfigIndex, pReceiver->privateTerm, msgStr); - taosMemoryFree(msgStr); - } else { - sDebug( - "vgId:%d sync event currentTerm:%lu snapshot recv from %s:%d end ack:%d, lastIndex:%ld, lastTerm:%lu, " - "lastConfigIndex:%ld, privateTerm:%lu", - pReceiver->pSyncNode->vgId, pReceiver->pSyncNode->pRaftStore->currentTerm, host, port, pReceiver->ack, - pMsg->lastIndex, pMsg->lastTerm, pMsg->lastConfigIndex, pReceiver->privateTerm); - } + do { + char *eventLog = snapshotReceiver2SimpleStr(pReceiver, "snapshot receiver end"); + syncNodeEventLog(pSyncNode, eventLog); + taosMemoryFree(eventLog); + } while (0); } else if (pMsg->seq == SYNC_SNAPSHOT_SEQ_FORCE_CLOSE) { pSyncNode->pFsm->FpSnapshotStopWrite(pSyncNode->pFsm, pReceiver->pWriter, false); snapshotReceiverStop(pReceiver, false); needRsp = false; - char host[128]; - uint16_t port; - syncUtilU642Addr(pMsg->srcId.addr, host, sizeof(host), &port); - - if (gRaftDetailLog) { - char *msgStr = syncSnapshotSend2Str(pMsg); - sDebug( - "vgId:%d sync event currentTerm:%lu snapshot recv from %s:%d force close ack:%d, lastIndex:%ld, " - "lastTerm:%lu, " - "lastConfigIndex:%ld, privateTerm:%lu, recv " - "msg:%s", - pReceiver->pSyncNode->vgId, pReceiver->pSyncNode->pRaftStore->currentTerm, host, port, pReceiver->ack, - pMsg->lastIndex, pMsg->lastTerm, pMsg->lastConfigIndex, pReceiver->privateTerm, msgStr); - taosMemoryFree(msgStr); - } else { - sDebug( - "vgId:%d sync event currentTerm:%lu snapshot recv from %s:%d force close ack:%d, lastIndex:%ld, " - "lastTerm:%lu, " - "lastConfigIndex:%ld, privateTerm:%lu", - pReceiver->pSyncNode->vgId, pReceiver->pSyncNode->pRaftStore->currentTerm, host, port, pReceiver->ack, - pMsg->lastIndex, pMsg->lastTerm, pMsg->lastConfigIndex, pReceiver->privateTerm); - } + do { + char *eventLog = snapshotReceiver2SimpleStr(pReceiver, "snapshot receiver force close"); + syncNodeEventLog(pSyncNode, eventLog); + taosMemoryFree(eventLog); + } while (0); } else if (pMsg->seq > SYNC_SNAPSHOT_SEQ_BEGIN && pMsg->seq < SYNC_SNAPSHOT_SEQ_END) { // transfering @@ -737,27 +598,11 @@ int32_t syncNodeOnSnapshotSendCb(SSyncNode *pSyncNode, SyncSnapshotSend *pMsg) { } needRsp = true; - char host[128]; - uint16_t port; - syncUtilU642Addr(pMsg->srcId.addr, host, sizeof(host), &port); - - if (gRaftDetailLog) { - char *msgStr = syncSnapshotSend2Str(pMsg); - sDebug( - "vgId:%d sync event currentTerm:%lu snapshot recv from %s:%d receiving ack:%d, lastIndex:%ld, " - "lastTerm:%lu, " - "lastConfigIndex:%ld, privateTerm:%lu, recv msg:%s", - pSyncNode->vgId, pSyncNode->pRaftStore->currentTerm, host, port, pReceiver->ack, pMsg->lastIndex, - pMsg->lastTerm, pMsg->lastConfigIndex, pReceiver->privateTerm, msgStr); - taosMemoryFree(msgStr); - } else { - sDebug( - "vgId:%d sync event currentTerm:%lu snapshot recv from %s:%d receiving ack:%d, lastIndex:%ld, " - "lastTerm:%lu, " - "lastConfigIndex:%ld, privateTerm:%lu", - pSyncNode->vgId, pSyncNode->pRaftStore->currentTerm, host, port, pReceiver->ack, pMsg->lastIndex, - pMsg->lastTerm, pMsg->lastConfigIndex, pReceiver->privateTerm); - } + do { + char *eventLog = snapshotReceiver2SimpleStr(pReceiver, "snapshot receiver receiving"); + syncNodeEventLog(pSyncNode, eventLog); + taosMemoryFree(eventLog); + } while (0); } else { ASSERT(0); diff --git a/source/libs/sync/src/syncUtil.c b/source/libs/sync/src/syncUtil.c index 40265965481654736e8d8b7448c369bd1918e573..cbc1298113ca630101205ea8da8eeadebb54c744 100644 --- a/source/libs/sync/src/syncUtil.c +++ b/source/libs/sync/src/syncUtil.c @@ -168,14 +168,26 @@ char* syncUtilRaftId2Str(const SRaftId* p) { } const char* syncUtilState2String(ESyncState state) { + /* + if (state == TAOS_SYNC_STATE_FOLLOWER) { + return "TAOS_SYNC_STATE_FOLLOWER"; + } else if (state == TAOS_SYNC_STATE_CANDIDATE) { + return "TAOS_SYNC_STATE_CANDIDATE"; + } else if (state == TAOS_SYNC_STATE_LEADER) { + return "TAOS_SYNC_STATE_LEADER"; + } else { + return "TAOS_SYNC_STATE_UNKNOWN"; + } + */ + if (state == TAOS_SYNC_STATE_FOLLOWER) { - return "TAOS_SYNC_STATE_FOLLOWER"; + return "follower"; } else if (state == TAOS_SYNC_STATE_CANDIDATE) { - return "TAOS_SYNC_STATE_CANDIDATE"; + return "candidate"; } else if (state == TAOS_SYNC_STATE_LEADER) { - return "TAOS_SYNC_STATE_LEADER"; + return "leader"; } else { - return "TAOS_SYNC_STATE_UNKNOWN"; + return "state_error"; } } diff --git a/source/libs/sync/test/syncRaftCfgTest.cpp b/source/libs/sync/test/syncRaftCfgTest.cpp index 8c6a704e2dcf9538058a75a086043e51f7eda080..8171d266d4eb26cd6f7b72a8d617f20be7846048 100644 --- a/source/libs/sync/test/syncRaftCfgTest.cpp +++ b/source/libs/sync/test/syncRaftCfgTest.cpp @@ -27,6 +27,14 @@ SRaftCfg* createRaftCfg() { } pCfg->isStandBy = taosGetTimestampSec() % 100; + pCfg->configIndexCount = 5; + for (int i = 0; i < MAX_CONFIG_INDEX_COUNT; ++i) { + (pCfg->configIndexArr)[i] = -1; + } + for (int i = 0; i < pCfg->configIndexCount; ++i) { + (pCfg->configIndexArr)[i] = i * 100; + } + return pCfg; } @@ -47,6 +55,8 @@ SSyncCfg* createSyncCfg() { void test1() { SSyncCfg* pCfg = createSyncCfg(); syncCfgLog2((char*)__FUNCTION__, pCfg); + syncCfgLog3((char*)__FUNCTION__, pCfg); + taosMemoryFree(pCfg); } @@ -100,6 +110,15 @@ void test5() { pCfg->isStandBy += 2; pCfg->snapshotEnable += 3; pCfg->lastConfigIndex += 1000; + + pCfg->configIndexCount = 5; + for (int i = 0; i < MAX_CONFIG_INDEX_COUNT; ++i) { + (pCfg->configIndexArr)[i] = -1; + } + for (int i = 0; i < pCfg->configIndexCount; ++i) { + (pCfg->configIndexArr)[i] = i * 100; + } + raftCfgPersist(pCfg); printf("%s update json file: %s myIndex->%d \n", (char*)__FUNCTION__, "./test3_raft_cfg.json", pCfg->cfg.myIndex); @@ -118,6 +137,6 @@ int main() { test3(); test4(); test5(); - + return 0; } diff --git a/source/libs/tfs/src/tfs.c b/source/libs/tfs/src/tfs.c index 92beeffa0cb7246986bb10ab508fdc4d4688a562..2d3d41de64b7939c0a3f543811adf75715b59811 100644 --- a/source/libs/tfs/src/tfs.c +++ b/source/libs/tfs/src/tfs.c @@ -282,6 +282,8 @@ int32_t tfsMkdir(STfs *pTfs, const char *rname) { } int32_t tfsRmdir(STfs *pTfs, const char *rname) { + ASSERT(rname[0] != 0); + char aname[TMPNAME_LEN] = "\0"; for (int32_t level = 0; level < pTfs->nlevel; level++) { @@ -289,6 +291,7 @@ int32_t tfsRmdir(STfs *pTfs, const char *rname) { for (int32_t id = 0; id < pTier->ndisk; id++) { STfsDisk *pDisk = pTier->disks[id]; snprintf(aname, TMPNAME_LEN, "%s%s%s", pDisk->path, TD_DIRSEP, rname); + uInfo("====> tfs remove dir : path:%s aname:%s rname:[%s]", pDisk->path, aname, rname); taosRemoveDir(aname); } } diff --git a/source/libs/transport/src/transSvr.c b/source/libs/transport/src/transSvr.c index 9a3fdeb3167c723e539d2616d1186552e5efdb3d..f9558d7252ceabfd6514f53665baefa60dd2ef23 100644 --- a/source/libs/transport/src/transSvr.c +++ b/source/libs/transport/src/transSvr.c @@ -312,7 +312,7 @@ static void uvHandleReq(SSvrConn* pConn) { } // set up conn info - SRpcConnInfo* pConnInfo = &(transMsg.info.connInfo); + SRpcConnInfo* pConnInfo = &(transMsg.info.conn); pConnInfo->clientIp = (uint32_t)(pConn->addr.sin_addr.s_addr); pConnInfo->clientPort = ntohs(pConn->addr.sin_port); tstrncpy(pConnInfo->user, pConn->user, sizeof(pConnInfo->user)); diff --git a/source/os/src/osSocket.c b/source/os/src/osSocket.c index 4a0d9e286629dfb4c788eb489ab41f9c6802d831..4d61e7036d695e4c0df34ef88d02a4609e6dcf1e 100644 --- a/source/os/src/osSocket.c +++ b/source/os/src/osSocket.c @@ -946,8 +946,7 @@ int32_t taosGetFqdn(char *fqdn) { #endif // __APPLE__ int32_t ret = getaddrinfo(hostname, NULL, &hints, &result); if (!result) { - // printf("failed to get fqdn, code:%d, reason:%s", ret, gai_strerror(ret)); - assert(0); + fprintf(stderr,"failed to get fqdn, code:%d, reason:%s", ret, gai_strerror(ret)); return -1; } diff --git a/source/util/src/tenv.c b/source/util/src/tenv.c index e717e82c5b27bb9911364571538cf8f6863098b8..4fc0542816397e780a962f9b6a52055114f97197 100644 --- a/source/util/src/tenv.c +++ b/source/util/src/tenv.c @@ -72,6 +72,9 @@ int32_t taosEnvToCfg(const char *envStr, char *cfgStr) { if (cfgNameLen > 0) { memcpy(cfgStr, buf, cfgNameLen); memset(&cfgStr[cfgNameLen], ' ', p - cfgStr - cfgNameLen + 1); + } else { + *cfgStr = '\0'; + return -1; } } return strlen(cfgStr); diff --git a/source/util/src/terror.c b/source/util/src/terror.c index 596aab0e09d08db07b7c47c5089838eb96c99171..7976cb09afd1fe8e560a33cd5ace1c7136636f33 100644 --- a/source/util/src/terror.c +++ b/source/util/src/terror.c @@ -132,6 +132,7 @@ TAOS_DEFINE_ERROR(TSDB_CODE_TSC_INVALID_INPUT, "Invalid tsc input") TAOS_DEFINE_ERROR(TSDB_CODE_TSC_STMT_API_ERROR, "Stmt API usage error") TAOS_DEFINE_ERROR(TSDB_CODE_TSC_STMT_TBNAME_ERROR, "Stmt table name not set") TAOS_DEFINE_ERROR(TSDB_CODE_TSC_STMT_CLAUSE_ERROR, "not supported stmt clause") +TAOS_DEFINE_ERROR(TSDB_CODE_TSC_QUERY_KILLED, "Query killed") // mnode-common TAOS_DEFINE_ERROR(TSDB_CODE_MND_APP_ERROR, "Mnode internal error") @@ -276,9 +277,14 @@ TAOS_DEFINE_ERROR(TSDB_CODE_MND_INVALID_TOPIC, "Invalid topic") TAOS_DEFINE_ERROR(TSDB_CODE_MND_INVALID_TOPIC_QUERY, "Topic with invalid query") TAOS_DEFINE_ERROR(TSDB_CODE_MND_INVALID_TOPIC_OPTION, "Topic with invalid option") TAOS_DEFINE_ERROR(TSDB_CODE_MND_CONSUMER_NOT_EXIST, "Consumer not exist") -TAOS_DEFINE_ERROR(TSDB_CODE_MND_CGROUP_USED, "Consumer group being used by some consumer") +TAOS_DEFINE_ERROR(TSDB_CODE_MND_TOPIC_OPTION_UNCHNAGED, "Consumer unchanged") +TAOS_DEFINE_ERROR(TSDB_CODE_MND_SUBSCRIBE_NOT_EXIST, "Subcribe not exist") +TAOS_DEFINE_ERROR(TSDB_CODE_MND_OFFSET_NOT_EXIST, "Offset not exist") +TAOS_DEFINE_ERROR(TSDB_CODE_MND_CONSUMER_NOT_READY, "Consumer not ready") TAOS_DEFINE_ERROR(TSDB_CODE_MND_TOPIC_SUBSCRIBED, "Topic subscribed cannot be dropped") +TAOS_DEFINE_ERROR(TSDB_CODE_MND_CGROUP_USED, "Consumer group being used by some consumer") +// mnode-stream TAOS_DEFINE_ERROR(TSDB_CODE_MND_STREAM_ALREADY_EXIST, "Stream already exists") TAOS_DEFINE_ERROR(TSDB_CODE_MND_STREAM_NOT_EXIST, "Stream not exist") TAOS_DEFINE_ERROR(TSDB_CODE_MND_INVALID_STREAM_OPTION, "Invalid stream option") @@ -450,6 +456,7 @@ TAOS_DEFINE_ERROR(TSDB_CODE_CTG_VG_META_MISMATCH, "table meta and vgroup TAOS_DEFINE_ERROR(TSDB_CODE_SCH_STATUS_ERROR, "scheduler status error") TAOS_DEFINE_ERROR(TSDB_CODE_SCH_INTERNAL_ERROR, "scheduler internal error") TAOS_DEFINE_ERROR(TSDB_CODE_SCH_TIMEOUT_ERROR, "Task timeout") +TAOS_DEFINE_ERROR(TSDB_CODE_SCH_JOB_IS_DROPPING, "Job is dropping") TAOS_DEFINE_ERROR(TSDB_CODE_QW_MSG_ERROR, "Invalid msg order") // parser @@ -572,9 +579,14 @@ TAOS_DEFINE_ERROR(TSDB_CODE_TSMA_NO_INDEX_IN_CACHE, "No tsma index in ca TAOS_DEFINE_ERROR(TSDB_CODE_RSMA_INVALID_ENV, "Invalid rsma env") TAOS_DEFINE_ERROR(TSDB_CODE_RSMA_INVALID_STAT, "Invalid rsma state") +//tq +TAOS_DEFINE_ERROR(TSDB_CODE_TQ_NO_COMMITTED_OFFSET, "No committed offset") + TAOS_DEFINE_ERROR(TSDB_CODE_INDEX_REBUILDING, "Index is rebuilding") +TAOS_DEFINE_ERROR(TSDB_CODE_TMQ_INVALID_MSG, "Invalid message") + #ifdef TAOS_ERROR_C }; #endif diff --git a/tests/script/jenkins/basic.txt b/tests/script/jenkins/basic.txt index 6fe1251ed7af715735be1bd708375f42640b1805..a8ec0f515180cddc7f84260cbd277c6d2963c065 100644 --- a/tests/script/jenkins/basic.txt +++ b/tests/script/jenkins/basic.txt @@ -132,7 +132,7 @@ #./test.sh -f tsim/mnode/basic1.sim -m # --- sma -./test.sh -f tsim/sma/tsmaCreateInsertData.sim +./test.sh -f tsim/sma/tsmaCreateInsertQuery.sim ./test.sh -f tsim/sma/rsmaCreateInsertQuery.sim # --- valgrind diff --git a/tests/script/sh/copy_udf.bat b/tests/script/sh/copy_udf.bat new file mode 100644 index 0000000000000000000000000000000000000000..5144cf8d25dba767774ba7457b77c5652b8eaa0b --- /dev/null +++ b/tests/script/sh/copy_udf.bat @@ -0,0 +1,23 @@ +@echo off + +echo Executing copy_udf.bat +set SCRIPT_DIR=%cd% +echo SCRIPT_DIR: %SCRIPT_DIR% + +cd ..\..\.. +set TAOS_DIR=%cd% +echo find udf library in %TAOS_DIR% +set UDF1_DIR=%TAOS_DIR%\debug\build\lib\udf1.dll +set UDF2_DIR=%TAOS_DIR%\debug\build\lib\udf2.dll + +echo %UDF1_DIR% +echo %UDF2_DIR% + +set UDF_TMP=C:\Windows\Temp\udf +rm -rf %UDF_TMP% +mkdir %UDF_TMP% + +echo Copy udf shared library files to %UDF_TMP% + +cp %UDF1_DIR% %UDF_TMP% +cp %UDF2_DIR% %UDF_TMP% diff --git a/tests/script/tsim/db/alter_option.sim b/tests/script/tsim/db/alter_option.sim index 4351ee5cb1415d20f74daee4fa18b97471bb9654..fede960a6ae1709f8fc267856b2a243f6dd349f4 100644 --- a/tests/script/tsim/db/alter_option.sim +++ b/tests/script/tsim/db/alter_option.sim @@ -92,7 +92,7 @@ endi if $data5_db != no_strict then # strict return -1 endi -if $data6_db != 345600 then # duration +if $data6_db != 345600m then # duration return -1 endi if $data7_db != 1440000m,1440000m,1440000m then # keep diff --git a/tests/script/tsim/db/basic6.sim b/tests/script/tsim/db/basic6.sim index 64103b5dac39530b6ba6c8ef61cff9421a3421cc..7525fe20870c7a25d64b0c77d60799a7c58c2001 100644 --- a/tests/script/tsim/db/basic6.sim +++ b/tests/script/tsim/db/basic6.sim @@ -34,7 +34,7 @@ endi if $data24 != 1 then return -1 endi -if $data26 != 2880 then +if $data26 != 2880m then return -1 endi if $data27 != 14400m,14400m,14400m then @@ -78,7 +78,7 @@ endi if $data24 != 1 then return -1 endi -if $data26 != 21600 then +if $data26 != 21600m then return -1 endi diff --git a/tests/script/tsim/db/create_all_options.sim b/tests/script/tsim/db/create_all_options.sim index 284875ee08489b3e745dcbf819fb9170ede49ccc..efe7ff99cfdd7bfb79c614dbaafd021b004cdfdc 100644 --- a/tests/script/tsim/db/create_all_options.sim +++ b/tests/script/tsim/db/create_all_options.sim @@ -113,7 +113,7 @@ endi if $data5_db != no_strict then # strict return -1 endi -if $data6_db != 14400 then # duration +if $data6_db != 14400m then # duration return -1 endi if $data7_db != 5256000m,5256000m,5256000m then # keep diff --git a/tests/script/tsim/insert/update0.sim b/tests/script/tsim/insert/update0.sim index 0ba3e98c913e1c37c50c351bed7d7385a1cad0d3..ed74188bcb9b673f008e714877e9b9caeae88629 100644 --- a/tests/script/tsim/insert/update0.sim +++ b/tests/script/tsim/insert/update0.sim @@ -9,7 +9,7 @@ sql create database d0 keep 365000d,365000d,365000d sql use d0 print =============== create super table and register rsma -sql create table if not exists stb (ts timestamp, c1 int) tags (city binary(20),district binary(20)) rollup(min) file_factor 0.1; +sql create table if not exists stb (ts timestamp, c1 int) tags (city binary(20),district binary(20)) rollup(min); sql show stables if $rows != 1 then diff --git a/tests/script/tsim/mnode/basic5.sim b/tests/script/tsim/mnode/basic5.sim index 399dced16909586443a2bcfc151d5b525904c161..fc591aa25d3677ebfaf6e982555ab09512badad8 100644 --- a/tests/script/tsim/mnode/basic5.sim +++ b/tests/script/tsim/mnode/basic5.sim @@ -3,6 +3,10 @@ system sh/deploy.sh -n dnode1 -i 1 system sh/deploy.sh -n dnode2 -i 2 system sh/deploy.sh -n dnode3 -i 3 system sh/deploy.sh -n dnode4 -i 4 +system sh/cfg.sh -n dnode1 -c transPullupInterval -v 1 +system sh/cfg.sh -n dnode2 -c transPullupInterval -v 1 +system sh/cfg.sh -n dnode3 -c transPullupInterval -v 1 +system sh/cfg.sh -n dnode4 -c transPullupInterval -v 1 system sh/exec.sh -n dnode1 -s start sql connect @@ -260,7 +264,7 @@ $x = 0 step92: $x = $x + 1 sleep 1000 - if $x == 10 then + if $x == 20 then return -1 endi sql show mnodes diff --git a/tests/script/tsim/query/udf.sim b/tests/script/tsim/query/udf.sim index 93cae4e3912cab0b5c36e60d28743f0c10f1e45a..d9821a04955e6d4f5f80db84afa8531762432672 100644 --- a/tests/script/tsim/query/udf.sim +++ b/tests/script/tsim/query/udf.sim @@ -19,8 +19,14 @@ sql show databases; sql create table t (ts timestamp, f int); sql insert into t values(now, 1)(now+1s, 2); -sql create function udf1 as '/tmp/udf/libudf1.so' outputtype int bufSize 8; -sql create aggregate function udf2 as '/tmp/udf/libudf2.so' outputtype double bufSize 8; +system_content printf %OS% +if $system_content == Windows_NT then + sql create function udf1 as 'C:\\Windows\\Temp\\udf1.dll' outputtype int bufSize 8; + sql create aggregate function udf2 as 'C:\\Windows\\Temp\\udf2.dll' outputtype double bufSize 8; +else + sql create function udf1 as '/tmp/udf/libudf1.so' outputtype int bufSize 8; + sql create aggregate function udf2 as '/tmp/udf/libudf2.so' outputtype double bufSize 8; +endi sql show functions; if $rows != 2 then return -1 diff --git a/tests/script/tsim/show/basic.sim b/tests/script/tsim/show/basic.sim index 95201bc48e0db1b57471039c80169ccdf4e30094..f23d75d78b432d5f69391b18a5bb0318e93af08b 100644 --- a/tests/script/tsim/show/basic.sim +++ b/tests/script/tsim/show/basic.sim @@ -98,7 +98,7 @@ if $rows != 1 then endi #sql select * from information_schema.`streams` sql select * from information_schema.user_tables -if $rows != 28 then +if $rows != 29 then return -1 endi #sql select * from information_schema.user_table_distributed @@ -196,7 +196,7 @@ if $rows != 1 then endi #sql select * from performance_schema.`streams` sql select * from information_schema.user_tables -if $rows != 28 then +if $rows != 29 then return -1 endi #sql select * from information_schema.user_table_distributed @@ -210,4 +210,4 @@ if $rows != 3 then endi system sh/exec.sh -n dnode1 -s stop -x SIGINT -system sh/exec.sh -n dnode2 -s stop -x SIGINT \ No newline at end of file +system sh/exec.sh -n dnode2 -s stop -x SIGINT diff --git a/tests/script/tsim/sma/rsmaCreateInsertQuery.sim b/tests/script/tsim/sma/rsmaCreateInsertQuery.sim index f929dda18cb1b287c3ffe05487464624ff0eebc5..645b28f7717d3889c8f23d3ebed514967acef511 100644 --- a/tests/script/tsim/sma/rsmaCreateInsertQuery.sim +++ b/tests/script/tsim/sma/rsmaCreateInsertQuery.sim @@ -9,7 +9,7 @@ sql create database d0 retentions 15s:7d,1m:21d,15m:365d; sql use d0 print =============== create super table and register rsma -sql create table if not exists stb (ts timestamp, c1 int) tags (city binary(20),district binary(20)) rollup(min) file_factor 0.1; +sql create table if not exists stb (ts timestamp, c1 int) tags (city binary(20),district binary(20)) rollup(min); sql show stables if $rows != 1 then diff --git a/tests/script/tsim/sma/tsmaCreateInsertData.sim b/tests/script/tsim/sma/tsmaCreateInsertData.sim deleted file mode 100644 index 7a8cc2a555e4c8c3c044d3674ab28b8cd97014d3..0000000000000000000000000000000000000000 --- a/tests/script/tsim/sma/tsmaCreateInsertData.sim +++ /dev/null @@ -1,56 +0,0 @@ -system sh/stop_dnodes.sh -system sh/deploy.sh -n dnode1 -i 1 -system sh/exec.sh -n dnode1 -s start -sleep 50 -sql connect - -print =============== create database -sql create database d1 vgroups 1 -sql use d1 - -print =============== create super table, include column type for count/sum/min/max/first -sql create table if not exists stb (ts timestamp, c1 int, c2 float, c3 double) tags (t1 int unsigned) - -sql show stables -if $rows != 1 then - return -1 -endi - -print =============== create child table -sql create table ct1 using stb tags(1000) - -sql show tables -if $rows != 1 then - return -1 -endi - -print =============== insert data, mode1: one row one table in sql -sql insert into ct1 values(now+0s, 10, 2.0, 3.0) -sql insert into ct1 values(now+1s, 11, 2.1, 3.1)(now+2s, -12, -2.2, -3.2)(now+3s, -13, -2.3, -3.3) - - -print =============== create sma index from super table -sql create sma index sma_index_name1 on stb function(max(c1),max(c2),min(c1)) interval(5m,10s) sliding(2m) -print $data00 $data01 $data02 $data03 - -print =============== trigger stream to execute sma aggr task and insert sma data into sma store -sql insert into ct1 values(now+5s, 20, 20.0, 30.0) -#=================================================================== - -print =============== show streams ================================ -sql show streams; -print $data00 $data01 $data02 - -if $data00 != d1 then - return -1 -endi - -print =============== select * from ct1 from memory -sql select * from ct1; -print $data00 $data01 -if $rows != 5 then - print rows $rows != 5 - return -1 -endi - -system sh/exec.sh -n dnode1 -s stop -x SIGINT diff --git a/tests/script/tsim/sma/tsmaCreateInsertQuery.sim b/tests/script/tsim/sma/tsmaCreateInsertQuery.sim new file mode 100644 index 0000000000000000000000000000000000000000..3a1ed62a7203566ae2e8c4c43df4bee11e5c9aa3 --- /dev/null +++ b/tests/script/tsim/sma/tsmaCreateInsertQuery.sim @@ -0,0 +1,181 @@ +system sh/stop_dnodes.sh +system sh/deploy.sh -n dnode1 -i 1 +system sh/exec.sh -n dnode1 -s start +sleep 50 +sql connect + +print =============== create database +sql create database d1 vgroups 1 +sql use d1 + +print =============== create super table, include column type for count/sum/min/max/first +sql create table if not exists stb (ts timestamp, c1 int, c2 float, c3 double) tags (t1 int unsigned) + +sql show stables +if $rows != 1 then + return -1 +endi + +print =============== create child table +sql create table ct1 using stb tags(1000) + +sql show tables +if $rows != 1 then + return -1 +endi + +print =============== insert data, mode1: one row one table in sql +sql insert into ct1 values(now+0s, 10, 2.0, 3.0) +sql insert into ct1 values(now+1s, 11, 2.1, 3.1)(now+2s, -12, -2.2, -3.2)(now+3s, -13, -2.3, -3.3) + + +print =============== create sma index from super table +sql create sma index sma_index_name1 on stb function(max(c1),max(c2),min(c1)) interval(5m,10s) sliding(5m) +print $data00 $data01 $data02 $data03 + +print =============== trigger stream to execute sma aggr task and insert sma data into sma store +sql insert into ct1 values(now+5s, 20, 20.0, 30.0) +#=================================================================== + +print =============== show streams ================================ +sql show streams; +print $data00 $data01 $data02 + +if $data00 != d1 then + return -1 +endi + +print =============== select * from ct1 from memory +sql select * from ct1; +print $data00 $data01 $data02 $data03 +print $data10 $data11 $data12 $data13 +print $data20 $data21 $data22 $data23 +print $data30 $data31 $data32 $data33 +print $data40 $data41 $data42 $data43 +if $rows != 5 then + print rows $rows != 5 + return -1 +endi + +print =============== select * from stb from memory in designated vgroup +sql select _wstartts, _wendts, min(c1),max(c2),max(c1) from stb interval(5m,10s) sliding(5m); +print $data00 $data01 $data02 $data03 $data04 +if $rows != 1 then + print rows $rows != 1 + return -1 +endi + +if $data02 != -13 then + print data02 $data02 != -13 + return -1 +endi + +if $data03 != 20.00000 then + print data03 $data03 != 20.00000 + return -1 +endi + +if $data04 != 20 then + print data04 $data04 != 20 + return -1 +endi + +print =============== select * from stb from memory in common vgroups +sql select _wstartts, _wendts, min(c1),max(c2),max(c1),max(c3) from stb interval(5m,10s) sliding(5m); +print $data00 $data01 $data02 $data03 $data04 $data05 +if $rows != 1 then + print rows $rows != 1 + return -1 +endi + +if $data02 != -13 then + print data02 $data02 != -13 + return -1 +endi + +if $data03 != 20.00000 then + print data03 $data03 != 20.00000 + return -1 +endi + +if $data04 != 20 then + print data04 $data04 != 20 + return -1 +endi + +if $data05 != 30.000000000 then + print data05 $data05 != 30.000000000 + return -1 +endi + + + +system sh/exec.sh -n dnode1 -s stop -x SIGINT +system sh/exec.sh -n dnode1 -s start +sleep 50 + + +print =============== select * from ct1 from file +sql select * from ct1; +print $data00 $data01 $data02 $data03 +print $data10 $data11 $data12 $data13 +print $data20 $data21 $data22 $data23 +print $data30 $data31 $data32 $data33 +print $data40 $data41 $data42 $data43 +if $rows != 5 then + print rows $rows != 5 + return -1 +endi + +print =============== select * from stb from file in designated vgroup +sql select _wstartts, _wendts, min(c1),max(c2),max(c1) from stb interval(5m,10s) sliding(5m); +print $data00 $data01 $data02 $data03 $data04 +if $rows != 1 then + print rows $rows != 1 + return -1 +endi + +if $data02 != -13 then + print data02 $data02 != -13 + return -1 +endi + +if $data03 != 20.00000 then + print data03 $data03 != 20.00000 + return -1 +endi + +if $data04 != 20 then + print data04 $data04 != 20 + return -1 +endi + +print =============== select * from stb from file in common vgroups +sql select _wstartts, _wendts, min(c1),max(c2),max(c1),max(c3) from stb interval(5m,10s) sliding(5m); +print $data00 $data01 $data02 $data03 $data04 $data05 +if $rows != 1 then + print rows $rows != 1 + return -1 +endi + +if $data02 != -13 then + print data02 $data02 != -13 + return -1 +endi + +if $data03 != 20.00000 then + print data03 $data03 != 20.00000 + return -1 +endi + +if $data04 != 20 then + print data04 $data04 != 20 + return -1 +endi + +if $data05 != 30.000000000 then + print data05 $data05 != 30.000000000 + return -1 +endi + + diff --git a/tests/script/tsim/testsuit.sim b/tests/script/tsim/testsuit.sim index 90210fd436f70ce7d6188b9fd78a0d970e1764c9..1636bb38c800350a2b39a26d5dcd9a25ab56dc23 100644 --- a/tests/script/tsim/testsuit.sim +++ b/tests/script/tsim/testsuit.sim @@ -1,35 +1,35 @@ -#run tsim/user/pass_alter.sim -#run tsim/user/basic1.sim -#run tsim/user/privilege2.sim -#run tsim/user/user_len.sim -#run tsim/user/privilege1.sim -#run tsim/user/pass_len.sim -#run tsim/table/basic1.sim -#run tsim/trans/lossdata1.sim -#run tsim/trans/create_db.sim -#run tsim/stable/alter_metrics.sim -#run tsim/stable/tag_modify.sim -#run tsim/stable/alter_comment.sim -#run tsim/stable/column_drop.sim -#run tsim/stable/column_modify.sim -#run tsim/stable/tag_rename.sim -#run tsim/stable/vnode3.sim -#run tsim/stable/metrics.sim -#run tsim/stable/alter_insert2.sim -#run tsim/stable/show.sim -#run tsim/stable/alter_import.sim -#run tsim/stable/tag_add.sim -#run tsim/stable/tag_drop.sim -#run tsim/stable/column_add.sim -#run tsim/stable/alter_count.sim -#run tsim/stable/values.sim +run tsim/user/pass_alter.sim +run tsim/user/basic1.sim +run tsim/user/privilege2.sim +run tsim/user/user_len.sim +run tsim/user/privilege1.sim +run tsim/user/pass_len.sim +run tsim/table/basic1.sim +run tsim/trans/lossdata1.sim +run tsim/trans/create_db.sim +run tsim/stable/alter_metrics.sim +run tsim/stable/tag_modify.sim +run tsim/stable/alter_comment.sim +run tsim/stable/column_drop.sim +run tsim/stable/column_modify.sim +run tsim/stable/tag_rename.sim +run tsim/stable/vnode3.sim +run tsim/stable/metrics.sim +run tsim/stable/alter_insert2.sim +run tsim/stable/show.sim +run tsim/stable/alter_import.sim +run tsim/stable/tag_add.sim +run tsim/stable/tag_drop.sim +run tsim/stable/column_add.sim +run tsim/stable/alter_count.sim +run tsim/stable/values.sim run tsim/stable/dnode3.sim -#run tsim/stable/alter_insert1.sim -#run tsim/stable/refcount.sim -#run tsim/stable/disk.sim +run tsim/stable/alter_insert1.sim +run tsim/stable/refcount.sim +run tsim/stable/disk.sim run tsim/db/basic1.sim run tsim/db/basic3.sim -#run tsim/db/basic7.sim +run tsim/db/basic7.sim run tsim/db/basic6.sim run tsim/db/create_all_options.sim run tsim/db/basic2.sim diff --git a/tests/system-test/1-insert/alter_stable.py b/tests/system-test/1-insert/alter_stable.py index c92efb403cf6c5bb7e283d176032907e32528607..cd64e3ddfe3b5d58292d98764df0e8b46033e27b 100644 --- a/tests/system-test/1-insert/alter_stable.py +++ b/tests/system-test/1-insert/alter_stable.py @@ -77,6 +77,7 @@ class TDTestCase: tdSql.error(f'alter stable {stbname} modify column c9 double') tdSql.error(f'alter stable {stbname} modify column c10 float') tdSql.error(f'alter stable {stbname} modify column c11 int') + tdSql.error(f'alter stable {stbname} drop tag t0') tdSql.execute(f'drop database {dbname}') def alter_stable_tag_check(self,dbname,stbname,tbname): @@ -126,6 +127,11 @@ class TDTestCase: for i in ['int','unsigned int','float','binary(10)','nchar(10)']: tdSql.error(f'alter stable {stbname} modify tag t8 {i}') tdSql.error(f'alter stable {stbname} modify tag t4 int') + tdSql.error(f'alter stable {stbname} drop column t0') + #!bug TD-16410 + # tdSql.error(f'alter stable {tbname} set tag t1=100 ') + # tdSql.execute(f'create table ntb (ts timestamp,c0 int)') + tdSql.error(f'alter stable ntb add column c2 ') tdSql.execute(f'drop database {dbname}') def run(self): diff --git a/tests/system-test/1-insert/alter_table.py b/tests/system-test/1-insert/alter_table.py index 3c0def86e4f053b941d0cc88144d0ac26db388de..a4e40d1b0b59f556c0307b740ed5333e9c0131e8 100644 --- a/tests/system-test/1-insert/alter_table.py +++ b/tests/system-test/1-insert/alter_table.py @@ -52,8 +52,6 @@ class TDTestCase: tdSql.execute(f'create database if not exists {dbname}') stbname = self.get_long_name(length=3, mode="letters") tbname = self.get_long_name(length=3, mode="letters") - tdLog.info('--------------------------child table tag check--------------------------------------') - tdLog.info(f'-----------------create stable {stbname} and child table {tbname}-------------------') tdSql.execute(f'create stable if not exists {dbname}.{stbname} (col_ts timestamp, c1 int) tags (tag_ts timestamp, t1 tinyint, t2 smallint, t3 int, \ t4 bigint, t5 tinyint unsigned, t6 smallint unsigned, t7 int unsigned, t8 bigint unsigned, t9 float, t10 double, t11 bool,t12 binary(20),t13 nchar(20))') tdSql.execute(f'create table if not exists {dbname}.{tbname} using {dbname}.{stbname} tags(now, 1, 2, 3, 4, 5, 6, 7, 8, 9.9, 10.1, True,"abc123","涛思数据")') @@ -90,13 +88,16 @@ class TDTestCase: tdSql.checkData(0,15,tag_nchar) # bug TD-16211 insert length more than setting binary and nchar - # tag_binary = self.get_long_name(length=21, mode="letters") - # tag_nchar = self.get_long_name(length=21, mode="letters") - # tdSql.error(f'alter table {dbname}.{tbname} set tag t12 = "{tag_binary}"') - # tdSql.error(f'alter table {dbname}.{tbname} set tag t13 = "{tag_nchar}"') - + # error_tag_binary = self.get_long_name(length=21, mode="letters") + # error_tag_nchar = self.get_long_name(length=21, mode="letters") + # tdSql.error(f'alter table {dbname}.{tbname} set tag t12 = "{error_tag_binary}"') + # tdSql.error(f'alter table {dbname}.{tbname} set tag t13 = "{error_tag_nchar}"') + error_tag_binary = self.get_long_name(length=25, mode="letters") + error_tag_nchar = self.get_long_name(length=25, mode="letters") + tdSql.error(f'alter table {dbname}.{tbname} set tag t12 = "{error_tag_binary}"') + tdSql.error(f'alter table {dbname}.{tbname} set tag t13 = "{error_tag_nchar}"') # bug TD-16210 modify binary to nchar - # tdSql.error(f'alter table {dbname}.{tbname} modify tag t12 nchar(10)') + tdSql.error(f'alter table {dbname}.{tbname} modify tag t12 nchar(10)') tdSql.execute(f"drop database {dbname}") def alter_ntb_column_check(self): ''' @@ -125,6 +126,20 @@ class TDTestCase: tdSql.execute(f'alter table {dbname}.{tbname} drop column `c15`') tdSql.query(f'describe {dbname}.{tbname}') tdSql.checkRows(14) + #! TD-16422 + # tdSql.execute(f'alter table {dbname}.{tbname} add column c16 binary(10)') + # tdSql.query(f'describe {dbname}.{tbname}') + # tdSql.checkRows(15) + # tdSql.checkEqual(tdSql.queryResult[14][2],10) + # tdSql.execute(f'alter table {dbname}.{tbname} drop column c16') + + # tdSql.execute(f'alter table {dbname}.{tbname} add column c16 nchar(10)') + # tdSql.query(f'describe {dbname}.{tbname}') + # tdSql.checkRows(15) + # tdSql.checkEqual(tdSql.queryResult[14][2],10) + # tdSql.execute(f'alter table {dbname}.{tbname} drop column c16') + + tdSql.execute(f'alter table {dbname}.{tbname} modify column c12 binary(30)') tdSql.query(f'describe {dbname}.{tbname}') tdSql.checkData(12,2,30) diff --git a/tests/system-test/1-insert/insertWithMoreVgroup.py b/tests/system-test/1-insert/insertWithMoreVgroup.py index 7708ebb476ace3d8c5df779acb01ca4fe5b46295..29c293c6084c9edb7107e4289e194b320cd43207 100644 --- a/tests/system-test/1-insert/insertWithMoreVgroup.py +++ b/tests/system-test/1-insert/insertWithMoreVgroup.py @@ -119,7 +119,7 @@ class TDTestCase: # tdLog.debug("spent %.2fs to create 1 stable and %d table, create speed is %.2f table/s... [OK]"% (spendTime,count,speedCreate)) return - def mutiThread_create_tables(self,host,dbname,stbname,vgroups,threadNumbers,childrowcount): + def mutiThread_create_tables(self,host,dbname,stbname,vgroups,threadNumbers,childcount): buildPath = self.getBuildPath() config = buildPath+ "../sim/dnode1/cfg/" @@ -128,7 +128,7 @@ class TDTestCase: tsql.execute("drop database if exists %s"%dbname) tsql.execute("create database %s vgroups %d"%(dbname,vgroups)) tsql.execute("use %s" %dbname) - count=int(childrowcount) + count=int(childcount) threads = [] for i in range(threadNumbers): tsql.execute("create stable %s%d(ts timestamp, c1 int, c2 binary(10)) tags(t1 int)"%(stbname,i)) @@ -264,19 +264,88 @@ class TDTestCase: speedCreate=count/spendTime tdLog.debug("spent %.2fs to create 1 stable and %d table, create speed is %.2f table/s... [OK]"% (spendTime,count,speedCreate)) return + + def checkData(self,dbname,stbname,stableCount,CtableCount,rowsPerSTable,): + tdSql.execute("use %s"%dbname) + tdSql.query("show stables") + tdSql.checkRows(stableCount) + tdSql.query("show tables") + tdSql.checkRows(CtableCount) + for i in range(stableCount): + tdSql.query("select count(*) from %s%d"%(stbname,i)) + tdSql.checkData(0,0,rowsPerSTable) + return + + # test case1 base def test_case1(self): + #stableCount=threadNumbersCtb + parameterDict = {'vgroups': 1, \ + 'threadNumbersCtb': 5, \ + 'threadNumbersIda': 5, \ + 'stableCount': 5, \ + 'tablesPerStb': 50, \ + 'rowsPerTable': 10, \ + 'dbname': 'db', \ + 'stbname': 'stb', \ + 'host': 'localhost', \ + 'startTs': 1640966400000} # 2022-01-01 00:00:00.000 + tdLog.debug("-----create database and muti-thread create tables test------- ") #host,dbname,stbname,vgroups,threadNumbers,tcountStart,tcountStop #host, dbname, stbname, threadNumbers, chilCount, ts_start, childrowcount - self.mutiThread_create_tables(host="localhost",dbname="db",stbname="stb", vgroups=1, threadNumbers=5, childrowcount=50) - self.mutiThread_insert_data(host="localhost",dbname="db",stbname="stb", threadNumbers=5,chilCount=50,ts_start=self.ts,childrowcount=10) - - return - - + self.mutiThread_create_tables( + host=parameterDict['host'], + dbname=parameterDict['dbname'], + stbname=parameterDict['stbname'], + vgroups=parameterDict['vgroups'], + threadNumbers=parameterDict['threadNumbersCtb'], + childcount=parameterDict['tablesPerStb']) + + self.mutiThread_insert_data( + host=parameterDict['host'], + dbname=parameterDict['dbname'], + stbname=parameterDict['stbname'], + threadNumbers=parameterDict['threadNumbersIda'], + chilCount=parameterDict['tablesPerStb'], + ts_start=parameterDict['startTs'], + childrowcount=parameterDict['rowsPerTable']) + + tableCount=parameterDict['threadNumbersCtb']*parameterDict['tablesPerStb'] + rowsPerStable=parameterDict['rowsPerTable']*parameterDict['tablesPerStb'] + + self.checkData(dbname=parameterDict['dbname'],stbname=parameterDict['stbname'], stableCount=parameterDict['threadNumbersCtb'],CtableCount=tableCount,rowsPerSTable=rowsPerStable) + def test_case3(self): - self.taosBenchCreate("127.0.0.1","no","db1", "stb1", 1, 8, 1*10) + #stableCount=threadNumbersCtb + parameterDict = {'vgroups': 1, \ + 'threadNumbersCtb': 8, \ + 'stableCount': 5, \ + 'tablesPerStb': 10, \ + 'rowsPerTable': 100, \ + 'dbname': 'db1', \ + 'stbname': 'stb1', \ + 'host': 'localhost', \ + 'startTs': 1640966400000} # 2022-01-01 00:00:00.000 + + self.taosBenchCreate( + parameterDict['host'], + "no", + parameterDict['dbname'], + parameterDict['stbname'], + parameterDict['vgroups'], + parameterDict['threadNumbersCtb'], + parameterDict['tablesPerStb']) + tableCount=parameterDict['threadNumbersCtb']*parameterDict['tablesPerStb'] + rowsPerStable=parameterDict['rowsPerTable']*parameterDict['tablesPerStb'] + + self.checkData( + dbname=parameterDict['dbname'], + stbname=parameterDict['stbname'], + stableCount=parameterDict['threadNumbersCtb'], + CtableCount=tableCount, + rowsPerSTable=rowsPerStable) + # self.taosBenchCreate("test209","no","db2", "stb2", 1, 8, 1*10000) # self.taosBenchCreate("chenhaoran02","no","db1", "stb1", 1, 8, 1*10000) @@ -320,14 +389,6 @@ class TDTestCase: # tdSql.execute("create qnode on dnode %s"%dnodeId) - - # self.taosBenchCreate("test209","no","db2", "stb2", 1, 8, 1*10000) - - # self.taosBenchCreate("chenhaoran02","no","db1", "stb1", 1, 8, 1*10000) - - # self.taosBenchCreate("db1", "stb1", 4, 5, 100*10000) - # self.taosBenchCreate("db1", "stb1", 1, 5, 100*10000) - # run case def run(self): diff --git a/tests/system-test/1-insert/manyVgroups.json b/tests/system-test/1-insert/manyVgroups.json index 5dea41476c8cf7777b5a548f470577e03c576663..20ac3205523af96cec2e7c646c6245c53a55c7e8 100644 --- a/tests/system-test/1-insert/manyVgroups.json +++ b/tests/system-test/1-insert/manyVgroups.json @@ -11,7 +11,7 @@ "confirm_parameter_prompt": "no", "insert_interval": 0, "interlace_rows": 0, - "num_of_records_per_req": 100, + "num_of_records_per_req": 100000, "databases": [ { "dbinfo": { @@ -29,7 +29,7 @@ "batch_create_tbl_num": 50000, "data_source": "rand", "insert_mode": "taosc", - "insert_rows": 1, + "insert_rows": 100, "interlace_rows": 0, "insert_interval": 0, "max_sql_len": 10000000, @@ -45,28 +45,28 @@ }, { "type": "DOUBLE", - "count": 100 + "count": 1 }, { "type": "BINARY", - "len": 400, - "count": 10 + "len": 40, + "count": 1 }, { "type": "nchar", - "len": 200, - "count": 20 + "len": 20, + "count": 1 } ], "tags": [ { "type": "TINYINT", - "count": 2 + "count": 1 }, { "type": "BINARY", "len": 16, - "count": 2 + "count": 1 } ] } diff --git a/tests/system-test/2-query/bottom.py b/tests/system-test/2-query/bottom.py index 008f59aa6ad406917a0e52805348868a399deee8..1037b0a8f307754ee5aa7c7f3aa30f2a47d74082 100644 --- a/tests/system-test/2-query/bottom.py +++ b/tests/system-test/2-query/bottom.py @@ -11,9 +11,12 @@ # -*- coding: utf-8 -*- +import random +import string from util.log import * from util.cases import * from util.sql import * +from util.common import * @@ -23,79 +26,89 @@ class TDTestCase: tdSql.init(conn.cursor()) self.rowNum = 10 + self.tbnum = 20 self.ts = 1537146000000 - - def run(self): + self.binary_str = 'taosdata' + self.nchar_str = '涛思数据' + def bottom_check_base(self): tdSql.prepare() - - tdSql.execute('''create table test(ts timestamp, col1 tinyint, col2 smallint, col3 int, col4 bigint, col5 float, col6 double, - col7 bool, col8 binary(20), col9 nchar(20), col11 tinyint unsigned, col12 smallint unsigned, col13 int unsigned, col14 bigint unsigned) tags(loc nchar(20))''') - tdSql.execute("create table test1 using test tags('beijing')") + tdSql.execute('''create table stb(ts timestamp, col1 tinyint, col2 smallint, col3 int, col4 bigint, col5 tinyint unsigned, col6 smallint unsigned, + col7 int unsigned, col8 bigint unsigned, col9 float, col10 double, col11 bool, col12 binary(20), col13 nchar(20)) tags(loc nchar(20))''') + tdSql.execute("create table stb_1 using stb tags('beijing')") + column_list = ['col1','col2','col3','col4','col5','col6','col7','col8'] + error_column_list = ['col11','col12','col13'] + error_param_list = [0,101] for i in range(self.rowNum): - tdSql.execute("insert into test1 values(%d, %d, %d, %d, %d, %f, %f, %d, 'taosdata%d', '涛思数据%d', %d, %d, %d, %d)" - % (self.ts + i, i + 1, i + 1, i + 1, i + 1, i + 0.1, i + 0.1, i % 2, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1)) - - # bottom verifacation - tdSql.error("select bottom(ts, 10) from test") - tdSql.error("select bottom(col1, 0) from test") - tdSql.error("select bottom(col1, 101) from test") - tdSql.error("select bottom(col2, 0) from test") - tdSql.error("select bottom(col2, 101) from test") - tdSql.error("select bottom(col3, 0) from test") - tdSql.error("select bottom(col3, 101) from test") - tdSql.error("select bottom(col4, 0) from test") - tdSql.error("select bottom(col4, 101) from test") - tdSql.error("select bottom(col5, 0) from test") - tdSql.error("select bottom(col5, 101) from test") - tdSql.error("select bottom(col6, 0) from test") - tdSql.error("select bottom(col6, 101) from test") - tdSql.error("select bottom(col7, 10) from test") - tdSql.error("select bottom(col8, 10) from test") - tdSql.error("select bottom(col9, 10) from test") - - tdSql.query("select bottom(col1, 2) from test") - tdSql.checkRows(2) - tdSql.checkEqual(tdSql.queryResult,[(2,),(1,)]) - tdSql.query("select bottom(col2, 2) from test") - tdSql.checkRows(2) - tdSql.checkEqual(tdSql.queryResult,[(2,),(1,)]) - - tdSql.query("select bottom(col3, 2) from test") - tdSql.checkRows(2) - tdSql.checkEqual(tdSql.queryResult,[(2,),(1,)]) - - tdSql.query("select bottom(col4, 2) from test") - tdSql.checkRows(2) - tdSql.checkEqual(tdSql.queryResult,[(2,),(1,)]) - - tdSql.query("select bottom(col11, 2) from test") - tdSql.checkRows(2) - tdSql.checkEqual(tdSql.queryResult,[(2,),(1,)]) - - tdSql.query("select bottom(col12, 2) from test") - tdSql.checkRows(2) - tdSql.checkEqual(tdSql.queryResult,[(2,),(1,)]) - - tdSql.query("select bottom(col13, 2) from test") - tdSql.checkRows(2) - tdSql.checkEqual(tdSql.queryResult,[(2,),(1,)]) - - tdSql.query("select bottom(col13,50) from test") - tdSql.checkRows(10) - - tdSql.query("select bottom(col14, 2) from test") - tdSql.checkRows(2) - tdSql.checkEqual(tdSql.queryResult,[(2,),(1,)]) - tdSql.query("select ts,bottom(col1, 2) from test1") - tdSql.checkRows(2) - tdSql.query("select ts,bottom(col1, 2),ts from test group by tbname") + tdSql.execute(f"insert into stb_1 values(%d, %d, %d, %d, %d, %d, %d, %d, %d, %f, %f, %d, '{self.binary_str}%d', '{self.nchar_str}%d')" + % (self.ts + i, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 0.1, i + 0.1, i % 2, i + 1, i + 1)) + + for i in column_list: + tdSql.query(f'select bottom({i},2) from stb_1') + tdSql.checkRows(2) + tdSql.checkEqual(tdSql.queryResult,[(2,),(1,)]) + for j in error_param_list: + tdSql.error(f'select bottom({i},{j}) from stb_1') + for i in error_column_list: + tdSql.error(f'select bottom({i},10) from stb_1') + tdSql.query("select ts,bottom(col1, 2),ts from stb_1 group by tbname") tdSql.checkRows(2) - - tdSql.query('select bottom(col2,1) from test interval(1y) order by col2') + tdSql.query('select bottom(col2,1) from stb_1 interval(1y) order by col2') tdSql.checkData(0,0,1) + tdSql.error('select * from stb_1 where bottom(col2,1)=1') + tdSql.execute('drop database db') + def bottom_check_distribute(self): + # prepare data for vgroup 4 + dbname = tdCom.getLongName(5, "letters") + stbname = tdCom.getLongName(5, "letters") + vgroup_num = 2 + child_table_num = 20 + tdSql.execute(f"create database if not exists {dbname} vgroups {vgroup_num}") + tdSql.execute(f'use {dbname}') + # build 20 child tables,every table insert 10 rows + tdSql.execute(f'''create table {stbname}(ts timestamp, col1 tinyint, col2 smallint, col3 int, col4 bigint, col5 tinyint unsigned, col6 smallint unsigned, + col7 int unsigned, col8 bigint unsigned, col9 float, col10 double, col11 bool, col12 binary(20), col13 nchar(20)) tags(loc nchar(20))''') + for i in range(child_table_num): + tdSql.execute(f"create table {stbname}_{i} using {stbname} tags('beijing')") + tdSql.execute(f"insert into {stbname}_{i}(ts) values(%d)" % (self.ts - 1-i)) + column_list = ['col1','col2','col3','col4','col5','col6','col7','col8'] + error_column_list = ['col11','col12','col13'] + error_param_list = [0,101] + for i in [f'{stbname}', f'{dbname}.{stbname}']: + for j in column_list: + tdSql.query(f"select bottom({j},1) from {i}") + tdSql.checkRows(0) + tdSql.query('show tables') + vgroup_list = [] + for i in range(len(tdSql.queryResult)): + vgroup_list.append(tdSql.queryResult[i][6]) + vgroup_list_set = set(vgroup_list) + + for i in vgroup_list_set: + vgroups_num = vgroup_list.count(i) + if vgroups_num >=2: + tdLog.info(f'This scene with {vgroups_num} vgroups is ok!') + continue + else: + tdLog.exit(f'This scene does not meet the requirements with {vgroups_num} vgroup!\n') + for i in range(self.rowNum): + for j in range(child_table_num): + tdSql.execute(f"insert into {stbname}_{j} values(%d, %d, %d, %d, %d, %d, %d, %d, %d, %f, %f, %d, '{self.binary_str}%d', '{self.nchar_str}%d')" + % (self.ts + i, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 0.1, i + 0.1, i % 2, i + 1, i + 1)) + for i in column_list: + tdSql.query(f'select bottom({i},2) from {stbname}') + tdSql.checkRows(2) + tdSql.checkEqual(tdSql.queryResult,[(1,),(1,)]) + for j in error_param_list: + tdSql.error(f'select bottom({i},{j}) from {stbname}') + for i in error_column_list: + tdSql.error(f'select bottom({i},10) from {stbname}') + + tdSql.execute(f'drop database {dbname}') + def run(self): - tdSql.error('select * from test where bottom(col2,1)=1') + self.bottom_check_base() + self.bottom_check_distribute() def stop(self): diff --git a/tests/system-test/2-query/distribute_agg_apercentile.py b/tests/system-test/2-query/distribute_agg_apercentile.py new file mode 100644 index 0000000000000000000000000000000000000000..fd1455ce16b8868de25d9800b73399ab0e88ec6f --- /dev/null +++ b/tests/system-test/2-query/distribute_agg_apercentile.py @@ -0,0 +1,198 @@ +from util.log import * +from util.cases import * +from util.sql import * +import numpy as np +import random + + +class TDTestCase: + updatecfgDict = {'debugFlag': 143 ,"cDebugFlag":143,"uDebugFlag":143 ,"rpcDebugFlag":143 , "tmrDebugFlag":143 , + "jniDebugFlag":143 ,"simDebugFlag":143,"dDebugFlag":143, "dDebugFlag":143,"vDebugFlag":143,"mDebugFlag":143,"qDebugFlag":143, + "wDebugFlag":143,"sDebugFlag":143,"tsdbDebugFlag":143,"tqDebugFlag":143 ,"fsDebugFlag":143 ,"fnDebugFlag":143, + "maxTablesPerVnode":2 ,"minTablesPerVnode":2,"tableIncStepPerVnode":2 } + + def init(self, conn, logSql): + tdLog.debug("start to execute %s" % __file__) + tdSql.init(conn.cursor()) + self.vnode_disbutes = None + self.ts = 1537146000000 + + def prepare_datas_of_distribute(self): + + # prepate datas for 20 tables distributed at different vgroups + tdSql.execute("create database if not exists testdb keep 3650 duration 1000 vgroups 5") + tdSql.execute(" use testdb ") + tdSql.execute( + '''create table stb1 + (ts timestamp, c1 int, c2 bigint, c3 smallint, c4 tinyint, c5 float, c6 double, c7 bool, c8 binary(16),c9 nchar(32), c10 timestamp) + tags (t0 timestamp, t1 int, t2 bigint, t3 smallint, t4 tinyint, t5 float, t6 double, t7 bool, t8 binary(16),t9 nchar(32)) + ''' + ) + + tdSql.execute( + ''' + create table t1 + (ts timestamp, c1 int, c2 bigint, c3 smallint, c4 tinyint, c5 float, c6 double, c7 bool, c8 binary(16),c9 nchar(32), c10 timestamp) + ''' + ) + for i in range(20): + tdSql.execute(f'create table ct{i+1} using stb1 tags ( now(), {1*i}, {11111*i}, {111*i}, {11*i}, {1.11*i}, {11.11*i}, {i%2}, "binary{i}", "nchar{i}" )') + + for i in range(9): + tdSql.execute( + f"insert into ct1 values ( now()-{i*10}s, {1*i}, {11111*i}, {111*i}, {11*i}, {1.11*i}, {11.11*i}, {i%2}, 'binary{i}', 'nchar{i}', now()+{1*i}a )" + ) + tdSql.execute( + f"insert into ct4 values ( now()-{i*90}d, {1*i}, {11111*i}, {111*i}, {11*i}, {1.11*i}, {11.11*i}, {i%2}, 'binary{i}', 'nchar{i}', now()+{1*i}a )" + ) + + for i in range(1,21): + if i ==1 or i == 4: + continue + else: + tbname = "ct"+f'{i}' + for j in range(9): + tdSql.execute( + f"insert into {tbname} values ( now()-{(i+j)*10}s, {1*(j+i)}, {11111*(j+i)}, {111*(j+i)}, {11*(j)}, {1.11*(j+i)}, {11.11*(j+i)}, {(j+i)%2}, 'binary{j}', 'nchar{j}', now()+{1*j}a )" + ) + tdSql.execute("insert into ct1 values (now()-45s, 0, 0, 0, 0, 0, 0, 0, 'binary0', 'nchar0', now()+8a )") + tdSql.execute("insert into ct1 values (now()+10s, 9, -99999, -999, -99, -9.99, -99.99, 1, 'binary9', 'nchar9', now()+9a )") + tdSql.execute("insert into ct1 values (now()+15s, 9, -99999, -999, -99, -9.99, NULL, 1, 'binary9', 'nchar9', now()+9a )") + tdSql.execute("insert into ct1 values (now()+20s, 9, -99999, -999, NULL, -9.99, -99.99, 1, 'binary9', 'nchar9', now()+9a )") + + tdSql.execute("insert into ct4 values (now()-810d, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) ") + tdSql.execute("insert into ct4 values (now()-400d, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) ") + tdSql.execute("insert into ct4 values (now()+90d, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) ") + + tdSql.execute( + f'''insert into t1 values + ( '2020-04-21 01:01:01.000', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) + ( '2020-10-21 01:01:01.000', 1, 11111, 111, 11, 1.11, 11.11, 1, "binary1", "nchar1", now()+1a ) + ( '2020-12-31 01:01:01.000', 2, 22222, 222, 22, 2.22, 22.22, 0, "binary2", "nchar2", now()+2a ) + ( '2021-01-01 01:01:06.000', 3, 33333, 333, 33, 3.33, 33.33, 0, "binary3", "nchar3", now()+3a ) + ( '2021-05-07 01:01:10.000', 4, 44444, 444, 44, 4.44, 44.44, 1, "binary4", "nchar4", now()+4a ) + ( '2021-07-21 01:01:01.000', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) + ( '2021-09-30 01:01:16.000', 5, 55555, 555, 55, 5.55, 55.55, 0, "binary5", "nchar5", now()+5a ) + ( '2022-02-01 01:01:20.000', 6, 66666, 666, 66, 6.66, 66.66, 1, "binary6", "nchar6", now()+6a ) + ( '2022-10-28 01:01:26.000', 7, 00000, 000, 00, 0.00, 00.00, 1, "binary7", "nchar7", "1970-01-01 08:00:00.000" ) + ( '2022-12-01 01:01:30.000', 8, -88888, -888, -88, -8.88, -88.88, 0, "binary8", "nchar8", "1969-01-01 01:00:00.000" ) + ( '2022-12-31 01:01:36.000', 9, -99999999999999999, -999, -99, -9.99, -999999999999999999999.99, 1, "binary9", "nchar9", "1900-01-01 00:00:00.000" ) + ( '2023-02-21 01:01:01.000', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) + ''' + ) + + tdLog.info(" prepare data for distributed_aggregate done! ") + + def check_distribute_datas(self): + # get vgroup_ids of all + tdSql.query("show vgroups ") + vgroups = tdSql.queryResult + + vnode_tables={} + + for vgroup_id in vgroups: + vnode_tables[vgroup_id[0]]=[] + + + # check sub_table of per vnode ,make sure sub_table has been distributed + tdSql.query("show tables like 'ct%'") + table_names = tdSql.queryResult + tablenames = [] + for table_name in table_names: + vnode_tables[table_name[6]].append(table_name[0]) + self.vnode_disbutes = vnode_tables + + count = 0 + for k ,v in vnode_tables.items(): + if len(v)>=2: + count+=1 + if count < 2: + tdLog.exit(" the datas of all not satisfy sub_table has been distributed ") + + def distribute_agg_query(self): + # basic filter + tdSql.query("select apercentile(c1 , 20) from stb1 where c1 is null") + tdSql.checkRows(0) + + tdSql.query("select apercentile(c1 , 20) from stb1 where t1=1") + tdSql.checkData(0,0,2.800000000) + + tdSql.query("select apercentile(c1+c2 ,100) from stb1 where c1 =1 ") + tdSql.checkData(0,0,11112.000000000) + + tdSql.query("select apercentile(c1 ,10 ) from stb1 where tbname=\"ct2\"") + tdSql.checkData(0,0,2.000000000) + + tdSql.query("select apercentile(c1,20) from stb1 partition by tbname") + tdSql.checkRows(20) + + tdSql.query("select apercentile(c1,20) from stb1 where t1> 4 partition by tbname") + tdSql.checkRows(15) + + # union all + tdSql.query("select apercentile(c1,20) from stb1 union all select apercentile(c1,20) from stb1 ") + tdSql.checkRows(2) + tdSql.checkData(0,0,7.389181281) + + # join + + tdSql.execute(" create database if not exists db ") + tdSql.execute(" use db ") + tdSql.execute(" create stable st (ts timestamp , c1 int ,c2 float) tags(t1 int) ") + tdSql.execute(" create table tb1 using st tags(1) ") + tdSql.execute(" create table tb2 using st tags(2) ") + + + for i in range(10): + ts = i*10 + self.ts + tdSql.execute(f" insert into tb1 values({ts},{i},{i}.0)") + tdSql.execute(f" insert into tb2 values({ts},{i},{i}.0)") + + tdSql.query("select apercentile(tb1.c1,100), apercentile(tb2.c2,100) from tb1, tb2 where tb1.ts=tb2.ts") + tdSql.checkRows(1) + tdSql.checkData(0,0,9.000000000) + tdSql.checkData(0,0,9.000000000) + + # group by + tdSql.execute(" use testdb ") + tdSql.query(" select max(c1),c1 from stb1 group by t1 ") + tdSql.checkRows(20) + tdSql.query(" select max(c1),c1 from stb1 group by c1 ") + tdSql.checkRows(30) + tdSql.query(" select max(c1),c2 from stb1 group by c2 ") + tdSql.checkRows(31) + + # partition by tbname or partition by tag + tdSql.query("select apercentile(c1 ,10)from stb1 partition by tbname") + query_data = tdSql.queryResult + + # nest query for support max + tdSql.query("select apercentile(c2+2,10)+1 from (select max(c1) c2 from stb1)") + tdSql.checkData(0,0,31.000000000) + tdSql.query("select apercentile(c1+2,10)+1 as c2 from (select ts ,c1 ,c2 from stb1)") + tdSql.checkData(0,0,7.560701700) + tdSql.query("select apercentile(a+2,10)+1 as c2 from (select ts ,abs(c1) a ,c2 from stb1)") + tdSql.checkData(0,0,7.560701700) + + # mixup with other functions + tdSql.query("select max(c1),count(c1),last(c2,c3),spread(c1), apercentile(c1,10) from stb1") + tdSql.checkData(0,0,28) + tdSql.checkData(0,1,184) + tdSql.checkData(0,2,-99999) + tdSql.checkData(0,3,-999) + tdSql.checkData(0,4,28.000000000) + tdSql.checkData(0,5,4.560701700) + + def run(self): + + self.prepare_datas_of_distribute() + self.check_distribute_datas() + self.distribute_agg_query() + + + def stop(self): + tdSql.close() + tdLog.success("%s successfully executed" % __file__) + +tdCases.addWindows(__file__, TDTestCase()) +tdCases.addLinux(__file__, TDTestCase()) diff --git a/tests/system-test/2-query/distribute_agg_count.py b/tests/system-test/2-query/distribute_agg_count.py new file mode 100644 index 0000000000000000000000000000000000000000..2ac9c86df0789830eafd088f22eadc621edaec63 --- /dev/null +++ b/tests/system-test/2-query/distribute_agg_count.py @@ -0,0 +1,296 @@ +from util.log import * +from util.cases import * +from util.sql import * +import numpy as np +import random + + +class TDTestCase: + updatecfgDict = {'debugFlag': 143 ,"cDebugFlag":143,"uDebugFlag":143 ,"rpcDebugFlag":143 , "tmrDebugFlag":143 , + "jniDebugFlag":143 ,"simDebugFlag":143,"dDebugFlag":143, "dDebugFlag":143,"vDebugFlag":143,"mDebugFlag":143,"qDebugFlag":143, + "wDebugFlag":143,"sDebugFlag":143,"tsdbDebugFlag":143,"tqDebugFlag":143 ,"fsDebugFlag":143 ,"fnDebugFlag":143, + "maxTablesPerVnode":2 ,"minTablesPerVnode":2,"tableIncStepPerVnode":2 } + + def init(self, conn, logSql): + tdLog.debug("start to execute %s" % __file__) + tdSql.init(conn.cursor()) + self.vnode_disbutes = None + self.ts = 1537146000000 + + + def check_count_functions(self, tbname , col_name): + + max_sql = f"select count({col_name}) from {tbname};" + + same_sql = f"select sum(c) from (select {col_name} ,1 as c from {tbname} where {col_name} is not null) " + + tdSql.query(max_sql) + max_result = tdSql.queryResult + + tdSql.query(same_sql) + same_result = tdSql.queryResult + + if max_result !=same_result: + tdLog.exit(" count function work not as expected, sql : %s "% max_sql) + else: + tdLog.info(" count function work as expected, sql : %s "% max_sql) + + + def prepare_datas_of_distribute(self): + + # prepate datas for 20 tables distributed at different vgroups + tdSql.execute("create database if not exists testdb keep 3650 duration 1000 vgroups 5") + tdSql.execute(" use testdb ") + tdSql.execute( + '''create table stb1 + (ts timestamp, c1 int, c2 bigint, c3 smallint, c4 tinyint, c5 float, c6 double, c7 bool, c8 binary(16),c9 nchar(32), c10 timestamp) + tags (t0 timestamp, t1 int, t2 bigint, t3 smallint, t4 tinyint, t5 float, t6 double, t7 bool, t8 binary(16),t9 nchar(32)) + ''' + ) + + tdSql.execute( + ''' + create table t1 + (ts timestamp, c1 int, c2 bigint, c3 smallint, c4 tinyint, c5 float, c6 double, c7 bool, c8 binary(16),c9 nchar(32), c10 timestamp) + ''' + ) + for i in range(20): + tdSql.execute(f'create table ct{i+1} using stb1 tags ( now(), {1*i}, {11111*i}, {111*i}, {11*i}, {1.11*i}, {11.11*i}, {i%2}, "binary{i}", "nchar{i}" )') + + for i in range(9): + tdSql.execute( + f"insert into ct1 values ( now()-{i*10}s, {1*i}, {11111*i}, {111*i}, {11*i}, {1.11*i}, {11.11*i}, {i%2}, 'binary{i}', 'nchar{i}', now()+{1*i}a )" + ) + tdSql.execute( + f"insert into ct4 values ( now()-{i*90}d, {1*i}, {11111*i}, {111*i}, {11*i}, {1.11*i}, {11.11*i}, {i%2}, 'binary{i}', 'nchar{i}', now()+{1*i}a )" + ) + + for i in range(1,21): + if i ==1 or i == 4: + continue + else: + tbname = "ct"+f'{i}' + for j in range(9): + tdSql.execute( + f"insert into {tbname} values ( now()-{(i+j)*10}s, {1*(j+i)}, {11111*(j+i)}, {111*(j+i)}, {11*(j)}, {1.11*(j+i)}, {11.11*(j+i)}, {(j+i)%2}, 'binary{j}', 'nchar{j}', now()+{1*j}a )" + ) + tdSql.execute("insert into ct1 values (now()-45s, 0, 0, 0, 0, 0, 0, 0, 'binary0', 'nchar0', now()+8a )") + tdSql.execute("insert into ct1 values (now()+10s, 9, -99999, -999, -99, -9.99, -99.99, 1, 'binary9', 'nchar9', now()+9a )") + tdSql.execute("insert into ct1 values (now()+15s, 9, -99999, -999, -99, -9.99, NULL, 1, 'binary9', 'nchar9', now()+9a )") + tdSql.execute("insert into ct1 values (now()+20s, 9, -99999, -999, NULL, -9.99, -99.99, 1, 'binary9', 'nchar9', now()+9a )") + + tdSql.execute("insert into ct4 values (now()-810d, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) ") + tdSql.execute("insert into ct4 values (now()-400d, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) ") + tdSql.execute("insert into ct4 values (now()+90d, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) ") + + tdSql.execute( + f'''insert into t1 values + ( '2020-04-21 01:01:01.000', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) + ( '2020-10-21 01:01:01.000', 1, 11111, 111, 11, 1.11, 11.11, 1, "binary1", "nchar1", now()+1a ) + ( '2020-12-31 01:01:01.000', 2, 22222, 222, 22, 2.22, 22.22, 0, "binary2", "nchar2", now()+2a ) + ( '2021-01-01 01:01:06.000', 3, 33333, 333, 33, 3.33, 33.33, 0, "binary3", "nchar3", now()+3a ) + ( '2021-05-07 01:01:10.000', 4, 44444, 444, 44, 4.44, 44.44, 1, "binary4", "nchar4", now()+4a ) + ( '2021-07-21 01:01:01.000', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) + ( '2021-09-30 01:01:16.000', 5, 55555, 555, 55, 5.55, 55.55, 0, "binary5", "nchar5", now()+5a ) + ( '2022-02-01 01:01:20.000', 6, 66666, 666, 66, 6.66, 66.66, 1, "binary6", "nchar6", now()+6a ) + ( '2022-10-28 01:01:26.000', 7, 00000, 000, 00, 0.00, 00.00, 1, "binary7", "nchar7", "1970-01-01 08:00:00.000" ) + ( '2022-12-01 01:01:30.000', 8, -88888, -888, -88, -8.88, -88.88, 0, "binary8", "nchar8", "1969-01-01 01:00:00.000" ) + ( '2022-12-31 01:01:36.000', 9, -99999999999999999, -999, -99, -9.99, -999999999999999999999.99, 1, "binary9", "nchar9", "1900-01-01 00:00:00.000" ) + ( '2023-02-21 01:01:01.000', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) + ''' + ) + + tdLog.info(" prepare data for distributed_aggregate done! ") + + def check_distribute_datas(self): + # get vgroup_ids of all + tdSql.query("show vgroups ") + vgroups = tdSql.queryResult + + vnode_tables={} + + for vgroup_id in vgroups: + vnode_tables[vgroup_id[0]]=[] + + + # check sub_table of per vnode ,make sure sub_table has been distributed + tdSql.query("show tables like 'ct%'") + table_names = tdSql.queryResult + tablenames = [] + for table_name in table_names: + vnode_tables[table_name[6]].append(table_name[0]) + self.vnode_disbutes = vnode_tables + + count = 0 + for k ,v in vnode_tables.items(): + if len(v)>=2: + count+=1 + if count < 2: + tdLog.exit(" the datas of all not satisfy sub_table has been distributed ") + + def check_count_distribute_diff_vnode(self,col_name): + + vgroup_ids = [] + for k ,v in self.vnode_disbutes.items(): + if len(v)>=2: + vgroup_ids.append(k) + + distribute_tbnames = [] + + for vgroup_id in vgroup_ids: + vnode_tables = self.vnode_disbutes[vgroup_id] + distribute_tbnames.append(random.sample(vnode_tables,1)[0]) + tbname_ins = "" + for tbname in distribute_tbnames: + tbname_ins += "'%s' ,"%tbname + + tbname_filters = tbname_ins[:-1] + + max_sql = f"select count({col_name}) from stb1 where tbname in ({tbname_filters});" + + same_sql = f"select sum(c) from (select {col_name} ,1 as c from stb1 where tbname in ({tbname_filters}) and {col_name} is not null) " + + tdSql.query(max_sql) + max_result = tdSql.queryResult + + tdSql.query(same_sql) + same_result = tdSql.queryResult + + if max_result !=same_result: + tdLog.exit(" count function work not as expected, sql : %s "% max_sql) + else: + tdLog.info(" count function work as expected, sql : %s "% max_sql) + + def check_count_status(self): + # check max function work status + + tdSql.query("show tables like 'ct%'") + table_names = tdSql.queryResult + tablenames = [] + for table_name in table_names: + tablenames.append(table_name[0]) + + tdSql.query("desc stb1") + col_names = tdSql.queryResult + + colnames = [] + for col_name in col_names: + if col_name[1] in ["INT" ,"BIGINT" ,"SMALLINT" ,"TINYINT" , "FLOAT" ,"DOUBLE"]: + colnames.append(col_name[0]) + + for tablename in tablenames: + for colname in colnames: + self.check_count_functions(tablename,colname) + + # check max function for different vnode + + for colname in colnames: + if colname.startswith("c"): + self.check_count_distribute_diff_vnode(colname) + else: + # self.check_count_distribute_diff_vnode(colname) # bug for tag + pass + + + def distribute_agg_query(self): + # basic filter + tdSql.query("select count(c1) from stb1 ") + tdSql.checkData(0,0,184) + + tdSql.query("select count(c1) from stb1 where t1=1") + tdSql.checkData(0,0,9) + + tdSql.query("select count(c1+c2) from stb1 where c1 =1 ") + tdSql.checkData(0,0,2) + + tdSql.query("select count(c1) from stb1 where tbname=\"ct2\"") + tdSql.checkData(0,0,9) + + tdSql.query("select count(c1) from stb1 partition by tbname") + tdSql.checkRows(20) + + tdSql.query("select count(c1) from stb1 where t1> 4 partition by tbname") + tdSql.checkRows(15) + + # union all + tdSql.query("select count(c1) from stb1 union all select count(c1) from stb1 ") + tdSql.checkRows(2) + tdSql.checkData(0,0,184) + + # join + + tdSql.execute(" create database if not exists db ") + tdSql.execute(" use db ") + tdSql.execute(" create stable st (ts timestamp , c1 int ,c2 float) tags(t1 int) ") + tdSql.execute(" create table tb1 using st tags(1) ") + tdSql.execute(" create table tb2 using st tags(2) ") + + + for i in range(10): + ts = i*10 + self.ts + tdSql.execute(f" insert into tb1 values({ts},{i},{i}.0)") + tdSql.execute(f" insert into tb2 values({ts},{i},{i}.0)") + + tdSql.query("select count(tb1.c1), count(tb2.c2) from tb1, tb2 where tb1.ts=tb2.ts") + tdSql.checkRows(1) + tdSql.checkData(0,0,10) + tdSql.checkData(0,1,10) + + # group by + tdSql.execute(" use testdb ") + + tdSql.query(" select count(*) from stb1 ") + tdSql.checkData(0,0,187) + tdSql.query(" select count(*) from stb1 group by t1 ") + tdSql.checkRows(20) + tdSql.query(" select count(*) from stb1 group by c1 ") + tdSql.checkRows(30) + tdSql.query(" select count(*) from stb1 group by c2 ") + tdSql.checkRows(31) + + # partition by tbname or partition by tag + tdSql.query("select max(c1),tbname from stb1 partition by tbname") + query_data = tdSql.queryResult + + for row in query_data: + tbname = row[1] + tdSql.query(" select max(c1) from %s "%tbname) + tdSql.checkData(0,0,row[0]) + + tdSql.query("select max(c1),tbname from stb1 partition by t1") + query_data = tdSql.queryResult + + for row in query_data: + tbname = row[1] + tdSql.query(" select max(c1) from %s "%tbname) + tdSql.checkData(0,0,row[0]) + + # nest query for support max + tdSql.query("select abs(c2+2)+1 from (select count(c1) c2 from stb1)") + tdSql.checkData(0,0,187.000000000) + tdSql.query("select count(c1+2) as c2 from (select ts ,c1 ,c2 from stb1)") + tdSql.checkData(0,0,184) + tdSql.query("select count(a+2) as c2 from (select ts ,abs(c1) a ,c2 from stb1)") + tdSql.checkData(0,0,184) + + # mixup with other functions + tdSql.query("select max(c1),count(c1),last(c2,c3) from stb1") + tdSql.checkData(0,0,28) + tdSql.checkData(0,1,184) + tdSql.checkData(0,2,-99999) + tdSql.checkData(0,3,-999) + + def run(self): + + self.prepare_datas_of_distribute() + self.check_distribute_datas() + self.check_count_status() + self.distribute_agg_query() + + + def stop(self): + tdSql.close() + tdLog.success("%s successfully executed" % __file__) + +tdCases.addWindows(__file__, TDTestCase()) +tdCases.addLinux(__file__, TDTestCase()) diff --git a/tests/system-test/2-query/distribute_agg_max.py b/tests/system-test/2-query/distribute_agg_max.py new file mode 100644 index 0000000000000000000000000000000000000000..ae0ab5aafab4b64ac9072b4de392d4359a4129de --- /dev/null +++ b/tests/system-test/2-query/distribute_agg_max.py @@ -0,0 +1,293 @@ +from util.log import * +from util.cases import * +from util.sql import * +import numpy as np +import random + + +class TDTestCase: + updatecfgDict = {'debugFlag': 143 ,"cDebugFlag":143,"uDebugFlag":143 ,"rpcDebugFlag":143 , "tmrDebugFlag":143 , + "jniDebugFlag":143 ,"simDebugFlag":143,"dDebugFlag":143, "dDebugFlag":143,"vDebugFlag":143,"mDebugFlag":143,"qDebugFlag":143, + "wDebugFlag":143,"sDebugFlag":143,"tsdbDebugFlag":143,"tqDebugFlag":143 ,"fsDebugFlag":143 ,"fnDebugFlag":143, + "maxTablesPerVnode":2 ,"minTablesPerVnode":2,"tableIncStepPerVnode":2 } + + def init(self, conn, logSql): + tdLog.debug("start to execute %s" % __file__) + tdSql.init(conn.cursor()) + self.vnode_disbutes = None + self.ts = 1537146000000 + + + def check_max_functions(self, tbname , col_name): + + max_sql = f"select max({col_name}) from {tbname};" + + same_sql = f"select {col_name} from {tbname} order by {col_name} desc limit 1" + + tdSql.query(max_sql) + max_result = tdSql.queryResult + + tdSql.query(same_sql) + same_result = tdSql.queryResult + + if max_result !=same_result: + tdLog.exit(" max function work not as expected, sql : %s "% max_sql) + else: + tdLog.info(" max function work as expected, sql : %s "% max_sql) + + + def prepare_datas_of_distribute(self): + + # prepate datas for 20 tables distributed at different vgroups + tdSql.execute("create database if not exists testdb keep 3650 duration 1000 vgroups 5") + tdSql.execute(" use testdb ") + tdSql.execute( + '''create table stb1 + (ts timestamp, c1 int, c2 bigint, c3 smallint, c4 tinyint, c5 float, c6 double, c7 bool, c8 binary(16),c9 nchar(32), c10 timestamp) + tags (t0 timestamp, t1 int, t2 bigint, t3 smallint, t4 tinyint, t5 float, t6 double, t7 bool, t8 binary(16),t9 nchar(32)) + ''' + ) + + tdSql.execute( + ''' + create table t1 + (ts timestamp, c1 int, c2 bigint, c3 smallint, c4 tinyint, c5 float, c6 double, c7 bool, c8 binary(16),c9 nchar(32), c10 timestamp) + ''' + ) + for i in range(20): + tdSql.execute(f'create table ct{i+1} using stb1 tags ( now(), {1*i}, {11111*i}, {111*i}, {11*i}, {1.11*i}, {11.11*i}, {i%2}, "binary{i}", "nchar{i}" )') + + for i in range(9): + tdSql.execute( + f"insert into ct1 values ( now()-{i*10}s, {1*i}, {11111*i}, {111*i}, {11*i}, {1.11*i}, {11.11*i}, {i%2}, 'binary{i}', 'nchar{i}', now()+{1*i}a )" + ) + tdSql.execute( + f"insert into ct4 values ( now()-{i*90}d, {1*i}, {11111*i}, {111*i}, {11*i}, {1.11*i}, {11.11*i}, {i%2}, 'binary{i}', 'nchar{i}', now()+{1*i}a )" + ) + + for i in range(1,21): + if i ==1 or i == 4: + continue + else: + tbname = "ct"+f'{i}' + for j in range(9): + tdSql.execute( + f"insert into {tbname} values ( now()-{(i+j)*10}s, {1*(j+i)}, {11111*(j+i)}, {111*(j+i)}, {11*(j)}, {1.11*(j+i)}, {11.11*(j+i)}, {(j+i)%2}, 'binary{j}', 'nchar{j}', now()+{1*j}a )" + ) + tdSql.execute("insert into ct1 values (now()-45s, 0, 0, 0, 0, 0, 0, 0, 'binary0', 'nchar0', now()+8a )") + tdSql.execute("insert into ct1 values (now()+10s, 9, -99999, -999, -99, -9.99, -99.99, 1, 'binary9', 'nchar9', now()+9a )") + tdSql.execute("insert into ct1 values (now()+15s, 9, -99999, -999, -99, -9.99, NULL, 1, 'binary9', 'nchar9', now()+9a )") + tdSql.execute("insert into ct1 values (now()+20s, 9, -99999, -999, NULL, -9.99, -99.99, 1, 'binary9', 'nchar9', now()+9a )") + + tdSql.execute("insert into ct4 values (now()-810d, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) ") + tdSql.execute("insert into ct4 values (now()-400d, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) ") + tdSql.execute("insert into ct4 values (now()+90d, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) ") + + tdSql.execute( + f'''insert into t1 values + ( '2020-04-21 01:01:01.000', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) + ( '2020-10-21 01:01:01.000', 1, 11111, 111, 11, 1.11, 11.11, 1, "binary1", "nchar1", now()+1a ) + ( '2020-12-31 01:01:01.000', 2, 22222, 222, 22, 2.22, 22.22, 0, "binary2", "nchar2", now()+2a ) + ( '2021-01-01 01:01:06.000', 3, 33333, 333, 33, 3.33, 33.33, 0, "binary3", "nchar3", now()+3a ) + ( '2021-05-07 01:01:10.000', 4, 44444, 444, 44, 4.44, 44.44, 1, "binary4", "nchar4", now()+4a ) + ( '2021-07-21 01:01:01.000', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) + ( '2021-09-30 01:01:16.000', 5, 55555, 555, 55, 5.55, 55.55, 0, "binary5", "nchar5", now()+5a ) + ( '2022-02-01 01:01:20.000', 6, 66666, 666, 66, 6.66, 66.66, 1, "binary6", "nchar6", now()+6a ) + ( '2022-10-28 01:01:26.000', 7, 00000, 000, 00, 0.00, 00.00, 1, "binary7", "nchar7", "1970-01-01 08:00:00.000" ) + ( '2022-12-01 01:01:30.000', 8, -88888, -888, -88, -8.88, -88.88, 0, "binary8", "nchar8", "1969-01-01 01:00:00.000" ) + ( '2022-12-31 01:01:36.000', 9, -99999999999999999, -999, -99, -9.99, -999999999999999999999.99, 1, "binary9", "nchar9", "1900-01-01 00:00:00.000" ) + ( '2023-02-21 01:01:01.000', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) + ''' + ) + + tdLog.info(" prepare data for distributed_aggregate done! ") + + def check_distribute_datas(self): + # get vgroup_ids of all + tdSql.query("show vgroups ") + vgroups = tdSql.queryResult + + vnode_tables={} + + for vgroup_id in vgroups: + vnode_tables[vgroup_id[0]]=[] + + + # check sub_table of per vnode ,make sure sub_table has been distributed + tdSql.query("show tables like 'ct%'") + table_names = tdSql.queryResult + tablenames = [] + for table_name in table_names: + vnode_tables[table_name[6]].append(table_name[0]) + self.vnode_disbutes = vnode_tables + + count = 0 + for k ,v in vnode_tables.items(): + if len(v)>=2: + count+=1 + if count < 2: + tdLog.exit(" the datas of all not satisfy sub_table has been distributed ") + + def check_max_distribute_diff_vnode(self,col_name): + + vgroup_ids = [] + for k ,v in self.vnode_disbutes.items(): + if len(v)>=2: + vgroup_ids.append(k) + + distribute_tbnames = [] + + for vgroup_id in vgroup_ids: + vnode_tables = self.vnode_disbutes[vgroup_id] + distribute_tbnames.append(random.sample(vnode_tables,1)[0]) + tbname_ins = "" + for tbname in distribute_tbnames: + tbname_ins += "'%s' ,"%tbname + + tbname_filters = tbname_ins[:-1] + + max_sql = f"select max({col_name}) from stb1 where tbname in ({tbname_filters});" + + same_sql = f"select {col_name} from stb1 where tbname in ({tbname_filters}) order by {col_name} desc limit 1" + + tdSql.query(max_sql) + max_result = tdSql.queryResult + + tdSql.query(same_sql) + same_result = tdSql.queryResult + + if max_result !=same_result: + tdLog.exit(" max function work not as expected, sql : %s "% max_sql) + else: + tdLog.info(" max function work as expected, sql : %s "% max_sql) + + def check_max_status(self): + # check max function work status + + tdSql.query("show tables like 'ct%'") + table_names = tdSql.queryResult + tablenames = [] + for table_name in table_names: + tablenames.append(table_name[0]) + + tdSql.query("desc stb1") + col_names = tdSql.queryResult + + colnames = [] + for col_name in col_names: + if col_name[1] in ["INT" ,"BIGINT" ,"SMALLINT" ,"TINYINT" , "FLOAT" ,"DOUBLE"]: + colnames.append(col_name[0]) + + for tablename in tablenames: + for colname in colnames: + self.check_max_functions(tablename,colname) + + # check max function for different vnode + + for colname in colnames: + if colname.startswith("c"): + self.check_max_distribute_diff_vnode(colname) + else: + # self.check_max_distribute_diff_vnode(colname) # bug for tag + pass + + + def distribute_agg_query(self): + # basic filter + tdSql.query("select max(c1) from stb1 where c1 is null") + tdSql.checkRows(0) + + tdSql.query("select max(c1) from stb1 where t1=1") + tdSql.checkData(0,0,10) + + tdSql.query("select max(c1+c2) from stb1 where c1 =1 ") + tdSql.checkData(0,0,11112.000000000) + + tdSql.query("select max(c1) from stb1 where tbname=\"ct2\"") + tdSql.checkData(0,0,10) + + tdSql.query("select max(c1) from stb1 partition by tbname") + tdSql.checkRows(20) + + tdSql.query("select max(c1) from stb1 where t1> 4 partition by tbname") + tdSql.checkRows(15) + + # union all + tdSql.query("select max(c1) from stb1 union all select max(c1) from stb1 ") + tdSql.checkRows(2) + tdSql.checkData(0,0,28) + + # join + + tdSql.execute(" create database if not exists db ") + tdSql.execute(" use db ") + tdSql.execute(" create stable st (ts timestamp , c1 int ,c2 float) tags(t1 int) ") + tdSql.execute(" create table tb1 using st tags(1) ") + tdSql.execute(" create table tb2 using st tags(2) ") + + + for i in range(10): + ts = i*10 + self.ts + tdSql.execute(f" insert into tb1 values({ts},{i},{i}.0)") + tdSql.execute(f" insert into tb2 values({ts},{i},{i}.0)") + + tdSql.query("select max(tb1.c1), tb2.c2 from tb1, tb2 where tb1.ts=tb2.ts") + tdSql.checkRows(1) + tdSql.checkData(0,0,9) + tdSql.checkData(0,0,9.00000) + + # group by + tdSql.execute(" use testdb ") + tdSql.query(" select max(c1),c1 from stb1 group by t1 ") + tdSql.checkRows(20) + tdSql.query(" select max(c1),c1 from stb1 group by c1 ") + tdSql.checkRows(30) + tdSql.query(" select max(c1),c2 from stb1 group by c2 ") + tdSql.checkRows(31) + + # partition by tbname or partition by tag + tdSql.query("select max(c1),tbname from stb1 partition by tbname") + query_data = tdSql.queryResult + + for row in query_data: + tbname = row[1] + tdSql.query(" select max(c1) from %s "%tbname) + tdSql.checkData(0,0,row[0]) + + tdSql.query("select max(c1),tbname from stb1 partition by t1") + query_data = tdSql.queryResult + + for row in query_data: + tbname = row[1] + tdSql.query(" select max(c1) from %s "%tbname) + tdSql.checkData(0,0,row[0]) + + # nest query for support max + tdSql.query("select abs(c2+2)+1 from (select max(c1) c2 from stb1)") + tdSql.checkData(0,0,31.000000000) + tdSql.query("select max(c1+2)+1 as c2 from (select ts ,c1 ,c2 from stb1)") + tdSql.checkData(0,0,31.000000000) + tdSql.query("select max(a+2)+1 as c2 from (select ts ,abs(c1) a ,c2 from stb1)") + tdSql.checkData(0,0,31.000000000) + + # mixup with other functions + tdSql.query("select max(c1),count(c1),last(c2,c3) from stb1") + tdSql.checkData(0,0,28) + tdSql.checkData(0,1,184) + tdSql.checkData(0,2,-99999) + tdSql.checkData(0,3,-999) + + def run(self): + + self.prepare_datas_of_distribute() + self.check_distribute_datas() + self.check_max_status() + self.distribute_agg_query() + + + def stop(self): + tdSql.close() + tdLog.success("%s successfully executed" % __file__) + +tdCases.addWindows(__file__, TDTestCase()) +tdCases.addLinux(__file__, TDTestCase()) diff --git a/tests/system-test/2-query/distribute_agg_min.py b/tests/system-test/2-query/distribute_agg_min.py new file mode 100644 index 0000000000000000000000000000000000000000..8a458c74df4f46929bb93abbebee2aad3dd7b76f --- /dev/null +++ b/tests/system-test/2-query/distribute_agg_min.py @@ -0,0 +1,294 @@ +from util.log import * +from util.cases import * +from util.sql import * +import numpy as np +import random + + +class TDTestCase: + updatecfgDict = {'debugFlag': 143 ,"cDebugFlag":143,"uDebugFlag":143 ,"rpcDebugFlag":143 , "tmrDebugFlag":143 , + "jniDebugFlag":143 ,"simDebugFlag":143,"dDebugFlag":143, "dDebugFlag":143,"vDebugFlag":143,"mDebugFlag":143,"qDebugFlag":143, + "wDebugFlag":143,"sDebugFlag":143,"tsdbDebugFlag":143,"tqDebugFlag":143 ,"fsDebugFlag":143 ,"fnDebugFlag":143, + "maxTablesPerVnode":2 ,"minTablesPerVnode":2,"tableIncStepPerVnode":2 } + + def init(self, conn, logSql): + tdLog.debug("start to execute %s" % __file__) + tdSql.init(conn.cursor()) + self.vnode_disbutes = None + self.ts = 1537146000000 + + + def check_min_functions(self, tbname , col_name): + + min_sql = f"select min({col_name}) from {tbname};" + + same_sql = f"select {col_name} from {tbname} where {col_name} is not null order by {col_name} asc limit 1" + + tdSql.query(min_sql) + min_result = tdSql.queryResult + + tdSql.query(same_sql) + same_result = tdSql.queryResult + + if min_result !=same_result: + tdLog.exit(" min function work not as expected, sql : %s "% min_sql) + else: + tdLog.info(" min function work as expected, sql : %s "% min_sql) + + + def prepare_datas_of_distribute(self): + + # prepate datas for 20 tables distributed at different vgroups + tdSql.execute("create database if not exists testdb keep 3650 duration 1000 vgroups 5") + tdSql.execute(" use testdb ") + tdSql.execute( + '''create table stb1 + (ts timestamp, c1 int, c2 bigint, c3 smallint, c4 tinyint, c5 float, c6 double, c7 bool, c8 binary(16),c9 nchar(32), c10 timestamp) + tags (t0 timestamp, t1 int, t2 bigint, t3 smallint, t4 tinyint, t5 float, t6 double, t7 bool, t8 binary(16),t9 nchar(32)) + ''' + ) + + tdSql.execute( + ''' + create table t1 + (ts timestamp, c1 int, c2 bigint, c3 smallint, c4 tinyint, c5 float, c6 double, c7 bool, c8 binary(16),c9 nchar(32), c10 timestamp) + ''' + ) + for i in range(20): + tdSql.execute(f'create table ct{i+1} using stb1 tags ( now(), {1*i}, {11111*i}, {111*i}, {11*i}, {1.11*i}, {11.11*i}, {i%2}, "binary{i}", "nchar{i}" )') + + for i in range(9): + tdSql.execute( + f"insert into ct1 values ( now()-{i*10}s, {1*i}, {11111*i}, {111*i}, {11*i}, {1.11*i}, {11.11*i}, {i%2}, 'binary{i}', 'nchar{i}', now()+{1*i}a )" + ) + tdSql.execute( + f"insert into ct4 values ( now()-{i*90}d, {1*i}, {11111*i}, {111*i}, {11*i}, {1.11*i}, {11.11*i}, {i%2}, 'binary{i}', 'nchar{i}', now()+{1*i}a )" + ) + + for i in range(1,21): + if i ==1 or i == 4: + continue + else: + tbname = "ct"+f'{i}' + for j in range(9): + tdSql.execute( + f"insert into {tbname} values ( now()-{(i+j)*10}s, {1*(j+i)}, {11111*(j+i)}, {111*(j+i)}, {11*(j)}, {1.11*(j+i)}, {11.11*(j+i)}, {(j+i)%2}, 'binary{j}', 'nchar{j}', now()+{1*j}a )" + ) + tdSql.execute("insert into ct1 values (now()-45s, 0, 0, 0, 0, 0, 0, 0, 'binary0', 'nchar0', now()+8a )") + tdSql.execute("insert into ct1 values (now()+10s, 9, -99999, -999, -99, -9.99, -99.99, 1, 'binary9', 'nchar9', now()+9a )") + tdSql.execute("insert into ct1 values (now()+15s, 9, -99999, -999, -99, -9.99, NULL, 1, 'binary9', 'nchar9', now()+9a )") + tdSql.execute("insert into ct1 values (now()+20s, 9, -99999, -999, NULL, -9.99, -99.99, 1, 'binary9', 'nchar9', now()+9a )") + + tdSql.execute("insert into ct4 values (now()-810d, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) ") + tdSql.execute("insert into ct4 values (now()-400d, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) ") + tdSql.execute("insert into ct4 values (now()+90d, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) ") + + tdSql.execute( + f'''insert into t1 values + ( '2020-04-21 01:01:01.000', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) + ( '2020-10-21 01:01:01.000', 1, 11111, 111, 11, 1.11, 11.11, 1, "binary1", "nchar1", now()+1a ) + ( '2020-12-31 01:01:01.000', 2, 22222, 222, 22, 2.22, 22.22, 0, "binary2", "nchar2", now()+2a ) + ( '2021-01-01 01:01:06.000', 3, 33333, 333, 33, 3.33, 33.33, 0, "binary3", "nchar3", now()+3a ) + ( '2021-05-07 01:01:10.000', 4, 44444, 444, 44, 4.44, 44.44, 1, "binary4", "nchar4", now()+4a ) + ( '2021-07-21 01:01:01.000', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) + ( '2021-09-30 01:01:16.000', 5, 55555, 555, 55, 5.55, 55.55, 0, "binary5", "nchar5", now()+5a ) + ( '2022-02-01 01:01:20.000', 6, 66666, 666, 66, 6.66, 66.66, 1, "binary6", "nchar6", now()+6a ) + ( '2022-10-28 01:01:26.000', 7, 00000, 000, 00, 0.00, 00.00, 1, "binary7", "nchar7", "1970-01-01 08:00:00.000" ) + ( '2022-12-01 01:01:30.000', 8, -88888, -888, -88, -8.88, -88.88, 0, "binary8", "nchar8", "1969-01-01 01:00:00.000" ) + ( '2022-12-31 01:01:36.000', 9, -99999999999999999, -999, -99, -9.99, -999999999999999999999.99, 1, "binary9", "nchar9", "1900-01-01 00:00:00.000" ) + ( '2023-02-21 01:01:01.000', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) + ''' + ) + + tdLog.info(" prepare data for distributed_aggregate done! ") + + def check_distribute_datas(self): + # get vgroup_ids of all + tdSql.query("show vgroups ") + vgroups = tdSql.queryResult + + vnode_tables={} + + for vgroup_id in vgroups: + vnode_tables[vgroup_id[0]]=[] + + + # check sub_table of per vnode ,make sure sub_table has been distributed + tdSql.query("show tables like 'ct%'") + table_names = tdSql.queryResult + tablenames = [] + for table_name in table_names: + vnode_tables[table_name[6]].append(table_name[0]) + self.vnode_disbutes = vnode_tables + + count = 0 + for k ,v in vnode_tables.items(): + if len(v)>=2: + count+=1 + if count < 2: + tdLog.exit(" the datas of all not satisfy sub_table has been distributed ") + + def check_min_distribute_diff_vnode(self,col_name): + + vgroup_ids = [] + for k ,v in self.vnode_disbutes.items(): + if len(v)>=2: + vgroup_ids.append(k) + + distribute_tbnames = [] + + for vgroup_id in vgroup_ids: + vnode_tables = self.vnode_disbutes[vgroup_id] + distribute_tbnames.append(random.sample(vnode_tables,1)[0]) + tbname_ins = "" + for tbname in distribute_tbnames: + tbname_ins += "'%s' ,"%tbname + + tbname_filters = tbname_ins[:-1] + + min_sql = f"select min({col_name}) from stb1 where tbname in ({tbname_filters});" + + same_sql = f"select {col_name} from stb1 where tbname in ({tbname_filters}) and {col_name} is not null order by {col_name} asc limit 1" + + tdSql.query(min_sql) + min_result = tdSql.queryResult + + tdSql.query(same_sql) + same_result = tdSql.queryResult + + if min_result !=same_result: + tdLog.exit(" min function work not as expected, sql : %s "% min_sql) + else: + tdLog.info(" min function work as expected, sql : %s "% min_sql) + + def check_min_status(self): + # check max function work status + + tdSql.query("show tables like 'ct%'") + table_names = tdSql.queryResult + tablenames = [] + for table_name in table_names: + tablenames.append(table_name[0]) + + tdSql.query("desc stb1") + col_names = tdSql.queryResult + + colnames = [] + for col_name in col_names: + if col_name[1] in ["INT" ,"BIGINT" ,"SMALLINT" ,"TINYINT" , "FLOAT" ,"DOUBLE"]: + colnames.append(col_name[0]) + + for tablename in tablenames: + for colname in colnames: + self.check_min_functions(tablename,colname) + + # check max function for different vnode + + for colname in colnames: + if colname.startswith("c"): + self.check_min_distribute_diff_vnode(colname) + else: + # self.check_min_distribute_diff_vnode(colname) # bug for tag + pass + + + def distribute_agg_query(self): + # basic filter + tdSql.query("select min(c1) from stb1 where c1 is null") + tdSql.checkRows(0) + + tdSql.query("select min(c1) from stb1 where t1=1") + tdSql.checkData(0,0,2) + + tdSql.query("select min(c1+c2) from stb1 where c1 =1 ") + tdSql.checkData(0,0,11112.000000000) + + tdSql.query("select min(c1) from stb1 where tbname=\"ct2\"") + tdSql.checkData(0,0,2) + + tdSql.query("select min(c1) from stb1 partition by tbname") + tdSql.checkRows(20) + + tdSql.query("select min(c1) from stb1 where t1> 4 partition by tbname") + tdSql.checkRows(15) + + # union all + tdSql.query("select min(c1) from stb1 union all select min(c1) from stb1 ") + tdSql.checkRows(2) + tdSql.checkData(0,0,0) + + # join + + tdSql.execute(" create database if not exists db ") + tdSql.execute(" use db ") + tdSql.execute(" create stable st (ts timestamp , c1 int ,c2 float) tags(t1 int) ") + tdSql.execute(" create table tb1 using st tags(1) ") + tdSql.execute(" create table tb2 using st tags(2) ") + + + for i in range(10): + ts = i*10 + self.ts + tdSql.execute(f" insert into tb1 values({ts},{i},{i}.0)") + tdSql.execute(f" insert into tb2 values({ts},{i},{i}.0)") + + tdSql.query("select min(tb1.c1), tb2.c2 from tb1, tb2 where tb1.ts=tb2.ts") + tdSql.checkRows(1) + tdSql.checkData(0,0,0) + tdSql.checkData(0,0,0.00000) + + # group by + tdSql.execute(" use testdb ") + tdSql.query(" select min(c1),c1 from stb1 group by t1 ") + tdSql.checkRows(20) + tdSql.query(" select min(c1),c1 from stb1 group by c1 ") + tdSql.checkRows(30) + tdSql.query(" select min(c1),c2 from stb1 group by c2 ") + tdSql.checkRows(31) + + # partition by tbname or partition by tag + tdSql.query("select min(c1),tbname from stb1 partition by tbname") + query_data = tdSql.queryResult + + for row in query_data: + tbname = row[1] + tdSql.query(" select min(c1) from %s "%tbname) + tdSql.checkData(0,0,row[0]) + + tdSql.query("select min(c1),tbname from stb1 partition by t1") + query_data = tdSql.queryResult + + for row in query_data: + tbname = row[1] + tdSql.query(" select min(c1) from %s "%tbname) + tdSql.checkData(0,0,row[0]) + + # nest query for support max + tdSql.query("select abs(c2+2)+1 from (select min(c1) c2 from stb1)") + tdSql.checkData(0,0,3.000000000) + tdSql.query("select min(c1+2)+1 as c2 from (select ts ,c1 ,c2 from stb1)") + tdSql.checkData(0,0,3.000000000) + tdSql.query("select min(a+2)+1 as c2 from (select ts ,abs(c1) a ,c2 from stb1)") + tdSql.checkData(0,0,3.000000000) + + # mixup with other functions + tdSql.query("select max(c1),count(c1),last(c2,c3),min(c1) from stb1") + tdSql.checkData(0,0,28) + tdSql.checkData(0,1,184) + tdSql.checkData(0,2,-99999) + tdSql.checkData(0,3,-999) + tdSql.checkData(0,4,0) + + def run(self): + + self.prepare_datas_of_distribute() + self.check_distribute_datas() + self.check_min_status() + self.distribute_agg_query() + + + def stop(self): + tdSql.close() + tdLog.success("%s successfully executed" % __file__) + +tdCases.addWindows(__file__, TDTestCase()) +tdCases.addLinux(__file__, TDTestCase()) diff --git a/tests/system-test/2-query/distribute_agg_spread.py b/tests/system-test/2-query/distribute_agg_spread.py new file mode 100644 index 0000000000000000000000000000000000000000..94f1a61d7727d17d3f1bc9484ca8ed4d3155c886 --- /dev/null +++ b/tests/system-test/2-query/distribute_agg_spread.py @@ -0,0 +1,281 @@ +from util.log import * +from util.cases import * +from util.sql import * +import numpy as np +import random + + +class TDTestCase: + updatecfgDict = {'debugFlag': 143 ,"cDebugFlag":143,"uDebugFlag":143 ,"rpcDebugFlag":143 , "tmrDebugFlag":143 , + "jniDebugFlag":143 ,"simDebugFlag":143,"dDebugFlag":143, "dDebugFlag":143,"vDebugFlag":143,"mDebugFlag":143,"qDebugFlag":143, + "wDebugFlag":143,"sDebugFlag":143,"tsdbDebugFlag":143,"tqDebugFlag":143 ,"fsDebugFlag":143 ,"fnDebugFlag":143, + "maxTablesPerVnode":2 ,"minTablesPerVnode":2,"tableIncStepPerVnode":2 } + + def init(self, conn, logSql): + tdLog.debug("start to execute %s" % __file__) + tdSql.init(conn.cursor()) + self.vnode_disbutes = None + self.ts = 1537146000000 + + + def check_spread_functions(self, tbname , col_name): + + spread_sql = f"select spread({col_name}) from {tbname};" + + same_sql = f"select max({col_name})-min({col_name}) from {tbname}" + + tdSql.query(spread_sql) + spread_result = tdSql.queryResult + + tdSql.query(same_sql) + same_result = tdSql.queryResult + + if spread_result !=same_result: + tdLog.exit(" max function work not as expected, sql : %s "% spread_sql) + else: + tdLog.info(" max function work as expected, sql : %s "% spread_sql) + + + def prepare_datas_of_distribute(self): + + # prepate datas for 20 tables distributed at different vgroups + tdSql.execute("create database if not exists testdb keep 3650 duration 1000 vgroups 5") + tdSql.execute(" use testdb ") + tdSql.execute( + '''create table stb1 + (ts timestamp, c1 int, c2 bigint, c3 smallint, c4 tinyint, c5 float, c6 double, c7 bool, c8 binary(16),c9 nchar(32), c10 timestamp) + tags (t0 timestamp, t1 int, t2 bigint, t3 smallint, t4 tinyint, t5 float, t6 double, t7 bool, t8 binary(16),t9 nchar(32)) + ''' + ) + + tdSql.execute( + ''' + create table t1 + (ts timestamp, c1 int, c2 bigint, c3 smallint, c4 tinyint, c5 float, c6 double, c7 bool, c8 binary(16),c9 nchar(32), c10 timestamp) + ''' + ) + for i in range(20): + tdSql.execute(f'create table ct{i+1} using stb1 tags ( now(), {1*i}, {11111*i}, {111*i}, {11*i}, {1.11*i}, {11.11*i}, {i%2}, "binary{i}", "nchar{i}" )') + + for i in range(9): + tdSql.execute( + f"insert into ct1 values ( now()-{i*10}s, {1*i}, {11111*i}, {111*i}, {11*i}, {1.11*i}, {11.11*i}, {i%2}, 'binary{i}', 'nchar{i}', now()+{1*i}a )" + ) + tdSql.execute( + f"insert into ct4 values ( now()-{i*90}d, {1*i}, {11111*i}, {111*i}, {11*i}, {1.11*i}, {11.11*i}, {i%2}, 'binary{i}', 'nchar{i}', now()+{1*i}a )" + ) + + for i in range(1,21): + if i ==1 or i == 4: + continue + else: + tbname = "ct"+f'{i}' + for j in range(9): + tdSql.execute( + f"insert into {tbname} values ( now()-{(i+j)*10}s, {1*(j+i)}, {11111*(j+i)}, {111*(j+i)}, {11*(j)}, {1.11*(j+i)}, {11.11*(j+i)}, {(j+i)%2}, 'binary{j}', 'nchar{j}', now()+{1*j}a )" + ) + tdSql.execute("insert into ct1 values (now()-45s, 0, 0, 0, 0, 0, 0, 0, 'binary0', 'nchar0', now()+8a )") + tdSql.execute("insert into ct1 values (now()+10s, 9, -99999, -999, -99, -9.99, -99.99, 1, 'binary9', 'nchar9', now()+9a )") + tdSql.execute("insert into ct1 values (now()+15s, 9, -99999, -999, -99, -9.99, NULL, 1, 'binary9', 'nchar9', now()+9a )") + tdSql.execute("insert into ct1 values (now()+20s, 9, -99999, -999, NULL, -9.99, -99.99, 1, 'binary9', 'nchar9', now()+9a )") + + tdSql.execute("insert into ct4 values (now()-810d, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) ") + tdSql.execute("insert into ct4 values (now()-400d, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) ") + tdSql.execute("insert into ct4 values (now()+90d, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) ") + + tdSql.execute( + f'''insert into t1 values + ( '2020-04-21 01:01:01.000', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) + ( '2020-10-21 01:01:01.000', 1, 11111, 111, 11, 1.11, 11.11, 1, "binary1", "nchar1", now()+1a ) + ( '2020-12-31 01:01:01.000', 2, 22222, 222, 22, 2.22, 22.22, 0, "binary2", "nchar2", now()+2a ) + ( '2021-01-01 01:01:06.000', 3, 33333, 333, 33, 3.33, 33.33, 0, "binary3", "nchar3", now()+3a ) + ( '2021-05-07 01:01:10.000', 4, 44444, 444, 44, 4.44, 44.44, 1, "binary4", "nchar4", now()+4a ) + ( '2021-07-21 01:01:01.000', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) + ( '2021-09-30 01:01:16.000', 5, 55555, 555, 55, 5.55, 55.55, 0, "binary5", "nchar5", now()+5a ) + ( '2022-02-01 01:01:20.000', 6, 66666, 666, 66, 6.66, 66.66, 1, "binary6", "nchar6", now()+6a ) + ( '2022-10-28 01:01:26.000', 7, 00000, 000, 00, 0.00, 00.00, 1, "binary7", "nchar7", "1970-01-01 08:00:00.000" ) + ( '2022-12-01 01:01:30.000', 8, -88888, -888, -88, -8.88, -88.88, 0, "binary8", "nchar8", "1969-01-01 01:00:00.000" ) + ( '2022-12-31 01:01:36.000', 9, -99999999999999999, -999, -99, -9.99, -999999999999999999999.99, 1, "binary9", "nchar9", "1900-01-01 00:00:00.000" ) + ( '2023-02-21 01:01:01.000', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) + ''' + ) + + tdLog.info(" prepare data for distributed_aggregate done! ") + + def check_distribute_datas(self): + # get vgroup_ids of all + tdSql.query("show vgroups ") + vgroups = tdSql.queryResult + + vnode_tables={} + + for vgroup_id in vgroups: + vnode_tables[vgroup_id[0]]=[] + + + # check sub_table of per vnode ,make sure sub_table has been distributed + tdSql.query("show tables like 'ct%'") + table_names = tdSql.queryResult + tablenames = [] + for table_name in table_names: + vnode_tables[table_name[6]].append(table_name[0]) + self.vnode_disbutes = vnode_tables + + count = 0 + for k ,v in vnode_tables.items(): + if len(v)>=2: + count+=1 + if count < 2: + tdLog.exit(" the datas of all not satisfy sub_table has been distributed ") + + def check_spread_distribute_diff_vnode(self,col_name): + + vgroup_ids = [] + for k ,v in self.vnode_disbutes.items(): + if len(v)>=2: + vgroup_ids.append(k) + + distribute_tbnames = [] + + for vgroup_id in vgroup_ids: + vnode_tables = self.vnode_disbutes[vgroup_id] + distribute_tbnames.append(random.sample(vnode_tables,1)[0]) + tbname_ins = "" + for tbname in distribute_tbnames: + tbname_ins += "'%s' ,"%tbname + + tbname_filters = tbname_ins[:-1] + + spread_sql = f"select spread({col_name}) from stb1 where tbname in ({tbname_filters})" + + same_sql = f"select max({col_name}) - min({col_name}) from stb1 where tbname in ({tbname_filters})" + + tdSql.query(spread_sql) + spread_result = tdSql.queryResult + + tdSql.query(same_sql) + same_result = tdSql.queryResult + + if spread_result !=same_result: + tdLog.exit(" spread function work not as expected, sql : %s "% spread_sql) + else: + tdLog.info(" spread function work as expected, sql : %s "% spread_sql) + + def check_spread_status(self): + # check max function work status + + tdSql.query("show tables like 'ct%'") + table_names = tdSql.queryResult + tablenames = [] + for table_name in table_names: + tablenames.append(table_name[0]) + + tdSql.query("desc stb1") + col_names = tdSql.queryResult + + colnames = [] + for col_name in col_names: + if col_name[1] in ["INT" ,"BIGINT" ,"SMALLINT" ,"TINYINT" , "FLOAT" ,"DOUBLE"]: + colnames.append(col_name[0]) + + for tablename in tablenames: + for colname in colnames: + self.check_spread_functions(tablename,colname) + + # check max function for different vnode + + for colname in colnames: + if colname.startswith("c"): + self.check_spread_distribute_diff_vnode(colname) + else: + # self.check_spread_distribute_diff_vnode(colname) # bug for tag + pass + + + def distribute_agg_query(self): + # basic filter + tdSql.query("select spread(c1) from stb1 where c1 is null") + tdSql.checkRows(0) + + tdSql.query("select spread(c1) from stb1 where t1=1") + tdSql.checkData(0,0,8.000000000) + + tdSql.query("select spread(c1+c2) from stb1 where c1 =1 ") + tdSql.checkData(0,0,0.000000000) + + tdSql.query("select spread(c1) from stb1 where tbname=\"ct2\"") + tdSql.checkData(0,0,8.000000000) + + tdSql.query("select spread(c1) from stb1 partition by tbname") + tdSql.checkRows(20) + + tdSql.query("select spread(c1) from stb1 where t1> 4 partition by tbname") + tdSql.checkRows(15) + + # union all + tdSql.query("select spread(c1) from stb1 union all select max(c1)-min(c1) from stb1 ") + tdSql.checkRows(2) + tdSql.checkData(0,0,28.000000000) + + # join + + tdSql.execute(" create database if not exists db ") + tdSql.execute(" use db ") + tdSql.execute(" create stable st (ts timestamp , c1 int ,c2 float) tags(t1 int) ") + tdSql.execute(" create table tb1 using st tags(1) ") + tdSql.execute(" create table tb2 using st tags(2) ") + + + for i in range(10): + ts = i*10 + self.ts + tdSql.execute(f" insert into tb1 values({ts},{i},{i}.0)") + tdSql.execute(f" insert into tb2 values({ts},{i},{i}.0)") + + tdSql.query("select spread(tb1.c1), spread(tb2.c2) from tb1, tb2 where tb1.ts=tb2.ts") + tdSql.checkRows(1) + tdSql.checkData(0,0,9.000000000) + tdSql.checkData(0,0,9.00000) + + # group by + tdSql.execute(" use testdb ") + tdSql.query(" select max(c1),c1 from stb1 group by t1 ") + tdSql.checkRows(20) + tdSql.query(" select max(c1),c1 from stb1 group by c1 ") + tdSql.checkRows(30) + tdSql.query(" select max(c1),c2 from stb1 group by c2 ") + tdSql.checkRows(31) + + # partition by tbname or partition by tag + tdSql.query("select spread(c1) from stb1 partition by tbname") + query_data = tdSql.queryResult + + # nest query for support max + tdSql.query("select spread(c2+2)+1 from (select max(c1) c2 from stb1)") + tdSql.checkData(0,0,1.000000000) + tdSql.query("select spread(c1+2)+1 as c2 from (select ts ,c1 ,c2 from stb1)") + tdSql.checkData(0,0,29.000000000) + tdSql.query("select spread(a+2)+1 as c2 from (select ts ,abs(c1) a ,c2 from stb1)") + tdSql.checkData(0,0,29.000000000) + + # mixup with other functions + tdSql.query("select max(c1),count(c1),last(c2,c3),spread(c1) from stb1") + tdSql.checkData(0,0,28) + tdSql.checkData(0,1,184) + tdSql.checkData(0,2,-99999) + tdSql.checkData(0,3,-999) + tdSql.checkData(0,4,28.000000000) + + def run(self): + + self.prepare_datas_of_distribute() + self.check_distribute_datas() + self.check_spread_status() + self.distribute_agg_query() + + + def stop(self): + tdSql.close() + tdLog.success("%s successfully executed" % __file__) + +tdCases.addWindows(__file__, TDTestCase()) +tdCases.addLinux(__file__, TDTestCase()) diff --git a/tests/system-test/2-query/distribute_agg_sum.py b/tests/system-test/2-query/distribute_agg_sum.py new file mode 100644 index 0000000000000000000000000000000000000000..add4d75c6171194f6669d36f39134dbbde6ba846 --- /dev/null +++ b/tests/system-test/2-query/distribute_agg_sum.py @@ -0,0 +1,278 @@ +from util.log import * +from util.cases import * +from util.sql import * +import numpy as np +import random ,os ,sys +import platform + + +class TDTestCase: + updatecfgDict = {'debugFlag': 143 ,"cDebugFlag":143,"uDebugFlag":143 ,"rpcDebugFlag":143 , "tmrDebugFlag":143 , + "jniDebugFlag":143 ,"simDebugFlag":143,"dDebugFlag":143, "dDebugFlag":143,"vDebugFlag":143,"mDebugFlag":143,"qDebugFlag":143, + "wDebugFlag":143,"sDebugFlag":143,"tsdbDebugFlag":143,"tqDebugFlag":143 ,"fsDebugFlag":143 ,"fnDebugFlag":143, + "maxTablesPerVnode":2 ,"minTablesPerVnode":2,"tableIncStepPerVnode":2 } + + def init(self, conn, logSql): + tdLog.debug("start to execute %s" % __file__) + tdSql.init(conn.cursor()) + self.vnode_disbutes = None + self.ts = 1537146000000 + + + def check_sum_functions(self, tbname , col_name): + + sum_sql = f"select sum({col_name}) from {tbname};" + + same_sql = f"select {col_name} from {tbname} where {col_name} is not null " + + tdSql.query(same_sql) + pre_data = np.array(tdSql.queryResult)[np.array(tdSql.queryResult) != None] + if (platform.system().lower() == 'windows' and pre_data.dtype == 'int32'): + pre_data = np.array(pre_data, dtype = 'int64') + pre_sum = np.sum(pre_data) + + tdSql.query(sum_sql) + tdSql.checkData(0,0,pre_sum) + + def prepare_datas_of_distribute(self): + + # prepate datas for 20 tables distributed at different vgroups + tdSql.execute("create database if not exists testdb keep 3650 duration 1000 vgroups 5") + tdSql.execute(" use testdb ") + tdSql.execute( + '''create table stb1 + (ts timestamp, c1 int, c2 bigint, c3 smallint, c4 tinyint, c5 float, c6 double, c7 bool, c8 binary(16),c9 nchar(32), c10 timestamp) + tags (t0 timestamp, t1 int, t2 bigint, t3 smallint, t4 tinyint, t5 float, t6 double, t7 bool, t8 binary(16),t9 nchar(32)) + ''' + ) + + tdSql.execute( + ''' + create table t1 + (ts timestamp, c1 int, c2 bigint, c3 smallint, c4 tinyint, c5 float, c6 double, c7 bool, c8 binary(16),c9 nchar(32), c10 timestamp) + ''' + ) + for i in range(20): + tdSql.execute(f'create table ct{i+1} using stb1 tags ( now(), {1*i}, {11111*i}, {111*i}, {11*i}, {1.11*i}, {11.11*i}, {i%2}, "binary{i}", "nchar{i}" )') + + for i in range(9): + tdSql.execute( + f"insert into ct1 values ( now()-{i*10}s, {1*i}, {11111*i}, {111*i}, {11*i}, {1.11*i}, {11.11*i}, {i%2}, 'binary{i}', 'nchar{i}', now()+{1*i}a )" + ) + tdSql.execute( + f"insert into ct4 values ( now()-{i*90}d, {1*i}, {11111*i}, {111*i}, {11*i}, {1.11*i}, {11.11*i}, {i%2}, 'binary{i}', 'nchar{i}', now()+{1*i}a )" + ) + + for i in range(1,21): + if i ==1 or i == 4: + continue + else: + tbname = "ct"+f'{i}' + for j in range(9): + tdSql.execute( + f"insert into {tbname} values ( now()-{(i+j)*10}s, {1*(j+i)}, {11111*(j+i)}, {111*(j+i)}, {11*(j)}, {1.11*(j+i)}, {11.11*(j+i)}, {(j+i)%2}, 'binary{j}', 'nchar{j}', now()+{1*j}a )" + ) + tdSql.execute("insert into ct1 values (now()-45s, 0, 0, 0, 0, 0, 0, 0, 'binary0', 'nchar0', now()+8a )") + tdSql.execute("insert into ct1 values (now()+10s, 9, -99999, -999, -99, -9.99, -99.99, 1, 'binary9', 'nchar9', now()+9a )") + tdSql.execute("insert into ct1 values (now()+15s, 9, -99999, -999, -99, -9.99, NULL, 1, 'binary9', 'nchar9', now()+9a )") + tdSql.execute("insert into ct1 values (now()+20s, 9, -99999, -999, NULL, -9.99, -99.99, 1, 'binary9', 'nchar9', now()+9a )") + + tdSql.execute("insert into ct4 values (now()-810d, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) ") + tdSql.execute("insert into ct4 values (now()-400d, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) ") + tdSql.execute("insert into ct4 values (now()+90d, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) ") + + tdSql.execute( + f'''insert into t1 values + ( '2020-04-21 01:01:01.000', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) + ( '2020-10-21 01:01:01.000', 1, 11111, 111, 11, 1.11, 11.11, 1, "binary1", "nchar1", now()+1a ) + ( '2020-12-31 01:01:01.000', 2, 22222, 222, 22, 2.22, 22.22, 0, "binary2", "nchar2", now()+2a ) + ( '2021-01-01 01:01:06.000', 3, 33333, 333, 33, 3.33, 33.33, 0, "binary3", "nchar3", now()+3a ) + ( '2021-05-07 01:01:10.000', 4, 44444, 444, 44, 4.44, 44.44, 1, "binary4", "nchar4", now()+4a ) + ( '2021-07-21 01:01:01.000', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) + ( '2021-09-30 01:01:16.000', 5, 55555, 555, 55, 5.55, 55.55, 0, "binary5", "nchar5", now()+5a ) + ( '2022-02-01 01:01:20.000', 6, 66666, 666, 66, 6.66, 66.66, 1, "binary6", "nchar6", now()+6a ) + ( '2022-10-28 01:01:26.000', 7, 00000, 000, 00, 0.00, 00.00, 1, "binary7", "nchar7", "1970-01-01 08:00:00.000" ) + ( '2022-12-01 01:01:30.000', 8, -88888, -888, -88, -8.88, -88.88, 0, "binary8", "nchar8", "1969-01-01 01:00:00.000" ) + ( '2022-12-31 01:01:36.000', 9, -99999999999999999, -999, -99, -9.99, -999999999999999999999.99, 1, "binary9", "nchar9", "1900-01-01 00:00:00.000" ) + ( '2023-02-21 01:01:01.000', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) + ''' + ) + + tdLog.info(" prepare data for distributed_aggregate done! ") + + def check_distribute_datas(self): + # get vgroup_ids of all + tdSql.query("show vgroups ") + vgroups = tdSql.queryResult + + vnode_tables={} + + for vgroup_id in vgroups: + vnode_tables[vgroup_id[0]]=[] + + + # check sub_table of per vnode ,make sure sub_table has been distributed + tdSql.query("show tables like 'ct%'") + table_names = tdSql.queryResult + tablenames = [] + for table_name in table_names: + vnode_tables[table_name[6]].append(table_name[0]) + self.vnode_disbutes = vnode_tables + + count = 0 + for k ,v in vnode_tables.items(): + if len(v)>=2: + count+=1 + if count < 2: + tdLog.exit(" the datas of all not satisfy sub_table has been distributed ") + + def check_sum_distribute_diff_vnode(self,col_name): + + vgroup_ids = [] + for k ,v in self.vnode_disbutes.items(): + if len(v)>=2: + vgroup_ids.append(k) + + distribute_tbnames = [] + + for vgroup_id in vgroup_ids: + vnode_tables = self.vnode_disbutes[vgroup_id] + distribute_tbnames.append(random.sample(vnode_tables,1)[0]) + tbname_ins = "" + for tbname in distribute_tbnames: + tbname_ins += "'%s' ,"%tbname + + tbname_filters = tbname_ins[:-1] + + sum_sql = f"select sum({col_name}) from stb1 where tbname in ({tbname_filters});" + + same_sql = f"select {col_name} from stb1 where tbname in ({tbname_filters}) and {col_name} is not null " + + tdSql.query(same_sql) + pre_data = np.array(tdSql.queryResult)[np.array(tdSql.queryResult) != None] + if (platform.system().lower() == 'windows' and pre_data.dtype == 'int32'): + pre_data = np.array(pre_data, dtype = 'int64') + pre_sum = np.sum(pre_data) + + tdSql.query(sum_sql) + tdSql.checkData(0,0,pre_sum) + + def check_sum_status(self): + # check max function work status + + tdSql.query("show tables like 'ct%'") + table_names = tdSql.queryResult + tablenames = [] + for table_name in table_names: + tablenames.append(table_name[0]) + + tdSql.query("desc stb1") + col_names = tdSql.queryResult + + colnames = [] + for col_name in col_names: + if col_name[1] in ["INT" ,"BIGINT" ,"SMALLINT" ,"TINYINT" , "FLOAT" ,"DOUBLE"]: + colnames.append(col_name[0]) + + for tablename in tablenames: + for colname in colnames: + self.check_sum_functions(tablename,colname) + + # check max function for different vnode + + for colname in colnames: + if colname.startswith("c"): + self.check_sum_distribute_diff_vnode(colname) + else: + # self.check_sum_distribute_diff_vnode(colname) # bug for tag + pass + + + def distribute_agg_query(self): + # basic filter + tdSql.query(" select sum(c1) from stb1 ") + tdSql.checkData(0,0,2592) + + tdSql.query(" select sum(a) from (select sum(c1) a from stb1 partition by tbname) ") + tdSql.checkData(0,0,2592) + + tdSql.query(" select sum(c1) from stb1 where t1=1") + tdSql.checkData(0,0,54) + + tdSql.query("select sum(c1+c2) from stb1 where c1 =1 ") + tdSql.checkData(0,0,22224.000000000) + + tdSql.query("select sum(c1) from stb1 where tbname=\"ct2\"") + tdSql.checkData(0,0,54) + + tdSql.query("select sum(c1) from stb1 partition by tbname") + tdSql.checkRows(20) + + tdSql.query("select sum(c1) from stb1 where t1> 4 partition by tbname") + tdSql.checkRows(15) + + # union all + tdSql.query("select sum(c1) from stb1 union all select sum(c1) from stb1 ") + tdSql.checkRows(2) + tdSql.checkData(0,0,2592) + + tdSql.query("select sum(a) from (select sum(c1) a from stb1 union all select sum(c1) a from stb1)") + tdSql.checkRows(1) + tdSql.checkData(0,0,5184) + + # join + + tdSql.execute(" create database if not exists db ") + tdSql.execute(" use db ") + tdSql.execute(" create stable st (ts timestamp , c1 int ,c2 float) tags(t1 int) ") + tdSql.execute(" create table tb1 using st tags(1) ") + tdSql.execute(" create table tb2 using st tags(2) ") + + + for i in range(10): + ts = i*10 + self.ts + tdSql.execute(f" insert into tb1 values({ts},{i},{i}.0)") + tdSql.execute(f" insert into tb2 values({ts},{i},{i}.0)") + + tdSql.query("select sum(tb1.c1), sum(tb2.c2) from tb1, tb2 where tb1.ts=tb2.ts") + tdSql.checkRows(1) + tdSql.checkData(0,0,45) + tdSql.checkData(0,1,45.000000000) + + # group by + tdSql.execute(" use testdb ") + + # partition by tbname or partition by tag + tdSql.query("select sum(c1) from stb1 partition by tbname") + tdSql.checkRows(20) + + # nest query for support max + tdSql.query("select abs(c2+2)+1 from (select sum(c1) c2 from stb1)") + tdSql.checkData(0,0,2595.000000000) + tdSql.query("select sum(c1+2) as c2 from (select ts ,c1 ,c2 from stb1)") + tdSql.checkData(0,0,2960.000000000) + tdSql.query("select sum(a+2) as c2 from (select ts ,abs(c1) a ,c2 from stb1)") + tdSql.checkData(0,0,2960.000000000) + + # mixup with other functions + tdSql.query("select max(c1),count(c1),last(c2,c3),sum(c1+c2) from stb1") + tdSql.checkData(0,0,28) + tdSql.checkData(0,1,184) + tdSql.checkData(0,2,-99999) + tdSql.checkData(0,3,-999) + tdSql.checkData(0,4,28202310.000000000) + + def run(self): + + self.prepare_datas_of_distribute() + self.check_distribute_datas() + self.check_sum_status() + self.distribute_agg_query() + + + def stop(self): + tdSql.close() + tdLog.success("%s successfully executed" % __file__) + +tdCases.addWindows(__file__, TDTestCase()) +tdCases.addLinux(__file__, TDTestCase()) diff --git a/tests/system-test/2-query/first.py b/tests/system-test/2-query/first.py index 7227d1afb5e22f68af90fb9d2192eb7a4a088c96..e9a8cc950b109ca6a8ec5eff8e323c57ea42e2c2 100644 --- a/tests/system-test/2-query/first.py +++ b/tests/system-test/2-query/first.py @@ -11,8 +11,11 @@ # -*- coding: utf-8 -*- +import random +import string import sys import taos +from util.common import * from util.log import * from util.cases import * from util.sql import * @@ -25,124 +28,159 @@ class TDTestCase: tdSql.init(conn.cursor()) self.rowNum = 10 + self.tbnum = 20 self.ts = 1537146000000 - - def run(self): + self.binary_str = 'taosdata' + self.nchar_str = '涛思数据' + + def first_check_base(self): tdSql.prepare() - - tdSql.execute('''create table test(ts timestamp, col1 tinyint, col2 smallint, col3 int, col4 bigint, col5 float, col6 double, - col7 bool, col8 binary(20), col9 nchar(20), col11 tinyint unsigned, col12 smallint unsigned, col13 int unsigned, col14 bigint unsigned) tags(loc nchar(20))''') - tdSql.execute("create table test1 using test tags('beijing')") - tdSql.execute("insert into test1(ts) values(%d)" % (self.ts - 1)) - - # first verifacation - # bug TD-15957 - tdSql.query("select first(*) from test1") - tdSql.checkRows(1) - tdSql.checkData(0, 1, None) - - tdSql.query("select first(col1) from test1") - tdSql.checkRows(0) - - tdSql.query("select first(col2) from test1") - tdSql.checkRows(0) - - tdSql.query("select first(col3) from test1") - tdSql.checkRows(0) - - tdSql.query("select first(col4) from test1") - tdSql.checkRows(0) - - tdSql.query("select first(col11) from test1") - tdSql.checkRows(0) - - tdSql.query("select first(col12) from test1") - tdSql.checkRows(0) - - tdSql.query("select first(col13) from test1") - tdSql.checkRows(0) - - tdSql.query("select first(col14) from test1") - tdSql.checkRows(0) - - tdSql.query("select first(col5) from test1") - tdSql.checkRows(0) - - tdSql.query("select first(col6) from test1") - tdSql.checkRows(0) - - tdSql.query("select first(col7) from test1") - tdSql.checkRows(0) - - tdSql.query("select first(col8) from test1") - tdSql.checkRows(0) - - tdSql.query("select first(col9) from test1") - tdSql.checkRows(0) - + column_dict = { + 'col1': 'tinyint', + 'col2': 'smallint', + 'col3': 'int', + 'col4': 'bigint', + 'col5': 'tinyint unsigned', + 'col6': 'smallint unsigned', + 'col7': 'int unsigned', + 'col8': 'bigint unsigned', + 'col9': 'float', + 'col10': 'double', + 'col11': 'bool', + 'col12': 'binary(20)', + 'col13': 'nchar(20)' + } + tdSql.execute('''create table stb(ts timestamp, col1 tinyint, col2 smallint, col3 int, col4 bigint, col5 tinyint unsigned, col6 smallint unsigned, + col7 int unsigned, col8 bigint unsigned, col9 float, col10 double, col11 bool, col12 binary(20), col13 nchar(20)) tags(loc nchar(20))''') + tdSql.execute("create table stb_1 using stb tags('beijing')") + tdSql.execute("insert into stb_1(ts) values(%d)" % (self.ts - 1)) + column_list = ['col1','col2','col3','col4','col5','col6','col7','col8','col9','col10','col11','col12','col13'] + for i in ['stb_1','db.stb_1','stb_1','db.stb_1']: + tdSql.query(f"select first(*) from {i}") + tdSql.checkRows(1) + tdSql.checkData(0, 1, None) + #!bug TD-16561 + # for i in ['stb','db.stb']: + # tdSql.query(f"select first(*) from {i}") + # tdSql.checkRows(1) + # tdSql.checkData(0, 1, None) + for i in column_list: + for j in ['stb_1','db.stb_1','stb_1','db.stb_1']: + tdSql.query(f"select first({i}) from {j}") + tdSql.checkRows(0) for i in range(self.rowNum): - tdSql.execute("insert into test1 values(%d, %d, %d, %d, %d, %f, %f, %d, 'taosdata%d', '涛思数据%d', %d, %d, %d, %d)" - % (self.ts + i, i + 1, i + 1, i + 1, i + 1, i + 0.1, i + 0.1, i % 2, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1)) - - tdSql.query("select first(*) from test1") - tdSql.checkRows(1) - tdSql.checkData(0, 1, 1) - - tdSql.query("select first(col1) from test1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 1) - - tdSql.query("select first(col2) from test1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 1) - - tdSql.query("select first(col3) from test1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 1) - - tdSql.query("select first(col4) from test1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 1) - - tdSql.query("select first(col11) from test1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 1) - - tdSql.query("select first(col12) from test1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 1) - - tdSql.query("select first(col13) from test1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 1) - - tdSql.query("select first(col14) from test1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 1) - - tdSql.query("select first(col5) from test1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 0.1) - - tdSql.query("select first(col6) from test1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 0.1) - - tdSql.query("select first(col7) from test1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, False) - - tdSql.query("select first(col8) from test1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 'taosdata1') - - tdSql.query("select first(col9) from test1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, '涛思数据1') + tdSql.execute(f"insert into stb_1 values(%d, %d, %d, %d, %d, %d, %d, %d, %d, %f, %f, %d, '{self.binary_str}%d', '{self.nchar_str}%d')" + % (self.ts + i, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 0.1, i + 0.1, i % 2, i + 1, i + 1)) + for k, v in column_dict.items(): + for j in ['stb_1', 'db.stb_1', 'stb', 'db.stb']: + tdSql.query(f"select first({k}) from {j}") + tdSql.checkRows(1) + # tinyint,smallint,int,bigint,tinyint unsigned,smallint unsigned,int unsigned,bigint unsigned + if v == 'tinyint' or v == 'smallint' or v == 'int' or v == 'bigint' or v == 'tinyint unsigned' or v == 'smallint unsigned'\ + or v == 'int unsigned' or v == 'bigint unsigned': + tdSql.checkData(0, 0, 1) + # float,double + elif v == 'float' or v == 'double': + tdSql.checkData(0, 0, 0.1) + # bool + elif v == 'bool': + tdSql.checkData(0, 0, False) + # binary + elif 'binary' in v: + tdSql.checkData(0, 0, f'{self.binary_str}1') + # nchar + elif 'nchar' in v: + tdSql.checkData(0, 0, f'{self.nchar_str}1') + #!bug TD-16569 + tdSql.query("select first(*),last(*) from stb where ts < 23 interval(1s)") + tdSql.checkRows(0) + tdSql.execute('drop database db') + def first_check_stb_distribute(self): + # prepare data for vgroup 4 + dbname = tdCom.getLongName(10, "letters") + stbname = tdCom.getLongName(5, "letters") + child_table_num = 20 + vgroup = 2 + column_dict = { + 'col1': 'tinyint', + 'col2': 'smallint', + 'col3': 'int', + 'col4': 'bigint', + 'col5': 'tinyint unsigned', + 'col6': 'smallint unsigned', + 'col7': 'int unsigned', + 'col8': 'bigint unsigned', + 'col9': 'float', + 'col10': 'double', + 'col11': 'bool', + 'col12': 'binary(20)', + 'col13': 'nchar(20)' + } + tdSql.execute(f"create database if not exists {dbname} vgroups {vgroup}") + tdSql.execute(f'use {dbname}') + # build 20 child tables,every table insert 10 rows + tdSql.execute(f'''create table {stbname}(ts timestamp, col1 tinyint, col2 smallint, col3 int, col4 bigint, col5 tinyint unsigned, col6 smallint unsigned, + col7 int unsigned, col8 bigint unsigned, col9 float, col10 double, col11 bool, col12 binary(20), col13 nchar(20)) tags(loc nchar(20))''') + for i in range(child_table_num): + tdSql.execute(f"create table {stbname}_{i} using {stbname} tags('beijing')") + tdSql.execute(f"insert into {stbname}_{i}(ts) values(%d)" % (self.ts - 1-i)) + #!bug TD-16561 + # for i in [f'{stbname}', f'{dbname}.{stbname}']: + # tdSql.query(f"select first(*) from {i}") + # tdSql.checkRows(1) + # tdSql.checkData(0, 1, None) + tdSql.query('show tables') + vgroup_list = [] + for i in range(len(tdSql.queryResult)): + vgroup_list.append(tdSql.queryResult[i][6]) + vgroup_list_set = set(vgroup_list) + # print(vgroup_list_set) + # print(vgroup_list) + for i in vgroup_list_set: + vgroups_num = vgroup_list.count(i) + if vgroups_num >=2: + tdLog.info(f'This scene with {vgroups_num} vgroups is ok!') + continue + else: + tdLog.exit('This scene does not meet the requirements with {vgroups_num} vgroup!\n') + for i in range(child_table_num): + for j in range(self.rowNum): + tdSql.execute(f"insert into {stbname}_{i} values(%d, %d, %d, %d, %d, %d, %d, %d, %d, %f, %f, %d, '{self.binary_str}%d', '{self.nchar_str}%d')" + % (self.ts + j + i, j + 1, j + 1, j + 1, j + 1, j + 1, j + 1, j + 1, j + 1, j + 0.1, j + 0.1, j % 2, j + 1, j + 1)) + + for k, v in column_dict.items(): + for j in [f'{stbname}_{i}', f'{dbname}.{stbname}_{i}', f'{stbname}', f'{dbname}.{stbname}']: + tdSql.query(f"select first({k}) from {j}") + tdSql.checkRows(1) + # tinyint,smallint,int,bigint,tinyint unsigned,smallint unsigned,int unsigned,bigint unsigned + if v == 'tinyint' or v == 'smallint' or v == 'int' or v == 'bigint' or v == 'tinyint unsigned' or v == 'smallint unsigned'\ + or v == 'int unsigned' or v == 'bigint unsigned': + tdSql.checkData(0, 0, 1) + # float,double + elif v == 'float' or v == 'double': + tdSql.checkData(0, 0, 0.1) + # bool + elif v == 'bool': + tdSql.checkData(0, 0, False) + # binary + elif 'binary' in v: + tdSql.checkData(0, 0, f'{self.binary_str}1') + # nchar + elif 'nchar' in v: + tdSql.checkData(0, 0, f'{self.nchar_str}1') + #!bug TD-16569 + # tdSql.query(f"select first(*),last(*) from {stbname} where ts < 23 interval(1s)") + # tdSql.checkRows(0) + tdSql.execute(f'drop database {dbname}') + + + + pass + def run(self): + self.first_check_base() + self.first_check_stb_distribute() - tdSql.query("select first(*),last(*) from test1 where ts < 23 interval(1s)") - tdSql.checkRows(0) def stop(self): tdSql.close() diff --git a/tests/system-test/2-query/histogram.py b/tests/system-test/2-query/histogram.py index 0448952be867308e0a8404de9f3bf9b6b43036e4..bc061f8fb793ff716e35fe22fc278b25bca9e257 100644 --- a/tests/system-test/2-query/histogram.py +++ b/tests/system-test/2-query/histogram.py @@ -44,8 +44,7 @@ class TDTestCase: buildPath = root[:len(root) - len("/build/bin")] break return buildPath - - def run(self): + def histogram_check_base(self): print("running {}".format(__file__)) tdSql.execute("drop database if exists db") tdSql.execute("create database if not exists db") @@ -3183,6 +3182,72 @@ class TDTestCase: tdSql.execute('drop database db') + + def histogram_check_distribute(self): + dbname = "db" + stbname = "stb" + row_num = 10 + child_table_num = 20 + vgroups = 2 + user_input_json = "[1,3,5,7]" + ts = 1537146000000 + binary_str = 'taosdata' + nchar_str = '涛思数据' + column_dict = { + 'ts' : 'timestamp', + 'col1' : 'tinyint', + 'col2' : 'smallint', + 'col3' : 'int', + 'col4' : 'bigint', + 'col5' : 'tinyint unsigned', + 'col6' : 'smallint unsigned', + 'col7' : 'int unsigned', + 'col8' : 'bigint unsigned', + 'col9' : 'float', + 'col10': 'double', + 'col11': 'bool', + 'col12': 'binary(20)', + 'col13': 'nchar(20)' + } + tdSql.execute(f"create database if not exists {dbname} vgroups {vgroups}") + tdSql.execute(f'use {dbname}') + # build 20 child tables,every table insert 10 rows + tdSql.execute(f'''create table {stbname}(ts timestamp, col1 tinyint, col2 smallint, col3 int, col4 bigint, col5 tinyint unsigned, col6 smallint unsigned, + col7 int unsigned, col8 bigint unsigned, col9 float, col10 double, col11 bool, col12 binary(20), col13 nchar(20)) tags(loc nchar(20))''') + for i in range(child_table_num): + tdSql.execute(f"create table {stbname}_{i} using {stbname} tags('beijing')") + tdSql.query('show tables') + vgroup_list = [] + for i in range(len(tdSql.queryResult)): + vgroup_list.append(tdSql.queryResult[i][6]) + vgroup_list_set = set(vgroup_list) + for i in vgroup_list_set: + vgroups_num = vgroup_list.count(i) + if vgroups_num >=2: + tdLog.info(f'This scene with {vgroups_num} vgroups is ok!') + continue + else: + tdLog.exit(f'This scene does not meet the requirements with {vgroups_num} vgroup!\n') + for i in range(child_table_num): + for j in range(row_num): + tdSql.execute(f"insert into {stbname}_{i} values(%d, %d, %d, %d, %d, %d, %d, %d, %d, %f, %f, %d, '{binary_str}%d', '{nchar_str}%d')" + % (ts + j + i, j + 1, j + 1, j + 1, j + 1, j + 1, j + 1, j + 1, j + 1, j + 0.1, j + 0.1, j % 2, j + 1, j + 1)) + # user_input + for k,v in column_dict.items(): + if v.lower() == 'tinyint' or v.lower() == 'smallint' or v.lower() == 'int' or v.lower() == 'bigint' or v.lower() =='float' or v.lower() =='double'\ + or v.lower() =='tinyint unsigned' or v.lower() =='smallint unsigned' or v.lower() =='int unsigned' or v.lower() =='bigint unsigned': + tdSql.query(f'select histogram({k}, "user_input", "{user_input_json}", 0) from {stbname}') + tdSql.checkRows(len(user_input_json[1:-1].split(','))-1) + elif 'binary' in v.lower() or 'nchar' in v.lower() or 'bool' == v.lower(): + tdSql.error(f'select histogram({k}, "user_input", "{user_input_json}", 0) from {stbname}') + + tdSql.execute(f'drop database {dbname}') + + + def run(self): + self.histogram_check_base() + self.histogram_check_distribute() + def stop(self): tdSql.close() tdLog.success("%s successfully executed" % __file__) diff --git a/tests/system-test/2-query/hyperloglog.py b/tests/system-test/2-query/hyperloglog.py index 337db140a167fa5c8dfae73d158b011f4a869189..8dd6bd2ddaef22698c66834cc26ec82a3ad16fea 100644 --- a/tests/system-test/2-query/hyperloglog.py +++ b/tests/system-test/2-query/hyperloglog.py @@ -214,6 +214,79 @@ class TDTestCase: for i in range(4): tdSql.execute(f'create table ct{i+1} using stb1 tags ( {i+1} )') { i % 32767 }, { i % 127}, { i * 1.11111 }, { i * 1000.1111 }, { i % 2} + def __create_stable(self,stbname='stb',column_dict={'ts':'timestamp','col1': 'tinyint','col2': 'smallint','col3': 'int', + 'col4': 'bigint','col5': 'tinyint unsigned','col6': 'smallint unsigned','col7': 'int unsigned', + 'col8': 'bigint unsigned','col9': 'float','col10': 'double','col11': 'bool','col12': 'binary(20)','col13': 'nchar(20)'}, + tag_dict={'ts_tag':'timestamp','t1': 'tinyint','t2': 'smallint','t3': 'int', + 't4': 'bigint','t5': 'tinyint unsigned','t6': 'smallint unsigned','t7': 'int unsigned', + 't8': 'bigint unsigned','t9': 'float','t10': 'double','t11': 'bool','t12': 'binary(20)','t13': 'nchar(20)'}): + column_sql = '' + tag_sql = '' + for k,v in column_dict.items(): + column_sql += f"{k} {v}," + for k,v in tag_dict.items(): + tag_sql += f"{k} {v}," + tdSql.execute(f'create table if not exists {stbname} ({column_sql[:-1]}) tags({tag_sql[:-1]})') + + def __insert_data(self): + + pass + + def __hyperloglog_check_distribute(self): + dbname = "dbtest" + stbname = "stb" + childtable_num = 20 + vgroups_num = 4 + row_num = 10 + ts = 1537146000000 + binary_str = 'taosdata' + nchar_str = '涛思数据' + column_dict = { + 'ts':'timestamp', + 'col1': 'tinyint', + 'col2': 'smallint', + 'col3': 'int', + 'col4': 'bigint', + 'col5': 'tinyint unsigned', + 'col6': 'smallint unsigned', + 'col7': 'int unsigned', + 'col8': 'bigint unsigned', + 'col9': 'float', + 'col10': 'double', + 'col11': 'bool', + 'col12': 'binary(20)', + 'col13': 'nchar(20)' + } + tag_dict = { + 'loc':'nchar(20)' + } + tdSql.execute(f"create database if not exists {dbname} vgroups {vgroups_num}") + tdSql.execute(f'use {dbname}') + self.__create_stable(stbname,column_dict,tag_dict) + for i in range(childtable_num): + tdSql.execute(f"create table {stbname}_{i} using {stbname} tags('beijing')") + tdSql.query('show tables') + vgroup_list = [] + for i in range(len(tdSql.queryResult)): + vgroup_list.append(tdSql.queryResult[i][6]) + vgroup_list_set = set(vgroup_list) + for i in vgroup_list_set: + vgroups_num = vgroup_list.count(i) + if vgroups_num >=2: + tdLog.info(f'This scene with {vgroups_num} vgroups is ok!') + continue + else: + tdLog.exit('This scene does not meet the requirements with {vgroups_num} vgroup!\n') + for i in range(row_num): + tdSql.execute(f"insert into stb_1 values(%d, %d, %d, %d, %d, %d, %d, %d, %d, %f, %f, %d, '{binary_str}%d', '{nchar_str}%d')" + % (ts + i, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 0.1, i + 0.1, i % 2, i + 1, i + 1)) + for k in column_dict.keys(): + tdSql.query(f"select hyperloglog({k}) from {stbname}") + tdSql.checkRows(1) + tdSql.query(f"select hyperloglog({k}) from {stbname} group by {k}") + + tdSql.execute(f'drop database {dbname}') + def __insert_data(self, rows): now_time = int(datetime.datetime.timestamp(datetime.datetime.now()) * 1000) @@ -311,6 +384,10 @@ class TDTestCase: tdLog.printNoPrefix("==========step4:after wal, all check again ") self.all_test() + tdLog.printNoPrefix("==========step5: distribute scene check") + self.__hyperloglog_check_distribute() + + def stop(self): tdSql.close() tdLog.success(f"{__file__} successfully executed") diff --git a/tests/system-test/2-query/last.py b/tests/system-test/2-query/last.py index 4ef13e9142f3a2ebc3ef55f6a2316fd6433908f3..ee65d22a22e28fa956d19ddecdcebe54e922ac71 100644 --- a/tests/system-test/2-query/last.py +++ b/tests/system-test/2-query/last.py @@ -1,6 +1,9 @@ +import random +import string from util.log import * from util.cases import * from util.sql import * +from util.common import * import numpy as np @@ -10,416 +13,258 @@ class TDTestCase: tdSql.init(conn.cursor()) self.rowNum = 10 + self.tbnum = 20 self.ts = 1537146000000 - - def run(self): - tdSql.prepare() + self.binary_str = 'taosdata' + self.nchar_str = '涛思数据' - tdSql.execute('''create table stb(ts timestamp, col1 tinyint, col2 smallint, col3 int, col4 bigint, col5 float, col6 double, - col7 bool, col8 binary(20), col9 nchar(20), col11 tinyint unsigned, col12 smallint unsigned, col13 int unsigned, col14 bigint unsigned) tags(loc nchar(20))''') - tdSql.execute("create table stb_1 using stb tags('beijing')") - tdSql.execute("insert into stb_1(ts) values(%d)" % (self.ts - 1)) + def set_create_normaltable_sql(self, ntbname, column_dict): + column_sql = '' + for k, v in column_dict.items(): + column_sql += f"{k} {v}," + create_ntb_sql = f'create table {ntbname} (ts timestamp,{column_sql[:-1]})' + return create_ntb_sql - # last verifacation - tdSql.query("select last(*) from stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 1, None) - tdSql.query("select last(*) from db.stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 1, None) - tdSql.query("select last(col1) from stb_1") - tdSql.checkRows(0) - tdSql.query("select last(col1) from db.stb_1") - tdSql.checkRows(0) - tdSql.query("select last(col2) from stb_1") - tdSql.checkRows(0) - tdSql.query("select last(col2) from db.stb_1") - tdSql.checkRows(0) - tdSql.query("select last(col3) from stb_1") - tdSql.checkRows(0) - tdSql.query("select last(col3) from db.stb_1") - tdSql.checkRows(0) - tdSql.query("select last(col4) from stb_1") - tdSql.checkRows(0) - tdSql.query("select last(col4) from db.stb_1") - tdSql.checkRows(0) - tdSql.query("select last(col11) from stb_1") - tdSql.checkRows(0) - tdSql.query("select last(col11) from db.stb_1") - tdSql.checkRows(0) - tdSql.query("select last(col12) from stb_1") - tdSql.checkRows(0) - tdSql.query("select last(col12) from db.stb_1") - tdSql.checkRows(0) - tdSql.query("select last(col13) from stb_1") - tdSql.checkRows(0) - tdSql.query("select last(col13) from db.stb_1") - tdSql.checkRows(0) - tdSql.query("select last(col14) from stb_1") - tdSql.checkRows(0) - tdSql.query("select last(col14) from db.stb_1") - tdSql.checkRows(0) - tdSql.query("select last(col5) from stb_1") - tdSql.checkRows(0) - tdSql.query("select last(col5) from db.stb_1") - tdSql.checkRows(0) - tdSql.query("select last(col6) from stb_1") - tdSql.checkRows(0) - tdSql.query("select last(col6) from db.stb_1") - tdSql.checkRows(0) - tdSql.query("select last(col7) from stb_1") - tdSql.checkRows(0) - tdSql.query("select last(col7) from db.stb_1") - tdSql.checkRows(0) - tdSql.query("select last(col8) from stb_1") - tdSql.checkRows(0) - tdSql.query("select last(col8) from db.stb_1") - tdSql.checkRows(0) - tdSql.query("select last(col9) from stb_1") - tdSql.checkRows(0) - tdSql.query("select last(col9) from db.stb_1") - tdSql.checkRows(0) - tdSql.query("select count(col1) from stb_1 group by col7") - tdSql.checkRows(1) + def set_create_stable_sql(self,stbname,column_dict,tag_dict): + column_sql = '' + tag_sql = '' + for k,v in column_dict.items(): + column_sql += f"{k} {v}," + for k,v in tag_dict.items(): + tag_sql += f"{k} {v}," + create_stb_sql = f'create table {stbname} (ts timestamp,{column_sql[:-1]}) tags({tag_sql[:-1]})' + return create_stb_sql + + def last_check_stb_tb_base(self): + tdSql.prepare() + stbname = tdCom.getLongName(5, "letters") + column_dict = { + 'col1': 'tinyint', + 'col2': 'smallint', + 'col3': 'int', + 'col4': 'bigint', + 'col5': 'tinyint unsigned', + 'col6': 'smallint unsigned', + 'col7': 'int unsigned', + 'col8': 'bigint unsigned', + 'col9': 'float', + 'col10': 'double', + 'col11': 'bool', + 'col12': 'binary(20)', + 'col13': 'nchar(20)' + } + tag_dict = { + 'loc':'nchar(20)' + } + tdSql.execute(self.set_create_stable_sql(stbname,column_dict,tag_dict)) - for i in range(self.rowNum): - tdSql.execute("insert into stb_1 values(%d, %d, %d, %d, %d, %f, %f, %d, 'taosdata%d', '涛思数据%d', %d, %d, %d, %d)" - % (self.ts + i, i + 1, i + 1, i + 1, i + 1, i + 0.1, i + 0.1, i % 2, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1)) + tdSql.execute(f"create table {stbname}_1 using {stbname} tags('beijing')") + tdSql.execute(f"insert into {stbname}_1(ts) values(%d)" % (self.ts - 1)) - tdSql.query("select last(*) from stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 1, 10) - tdSql.query("select last(*) from db.stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 1, 10) - tdSql.query("select last(col1) from stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col1) from db.stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col2) from stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col2) from db.stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col3) from stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col3) from db.stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col4) from stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col4) from db.stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col11) from stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col11) from db.stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col12) from stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col12) from db.stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col13) from stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col13) from db.stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col14) from stb_1") + for i in [f'{stbname}_1', f'db.{stbname}_1']: + tdSql.query(f"select last(*) from {i}") + tdSql.checkRows(1) + tdSql.checkData(0, 1, None) + #!bug TD-16561 + # for i in ['stb','db.stb','stb','db.stb']: + # tdSql.query(f"select last(*) from {i}") + # tdSql.checkRows(1) + # tdSql.checkData(0, 1, None) + for i in column_dict.keys(): + for j in [f'{stbname}_1', f'db.{stbname}_1', f'{stbname}', f'db.{stbname}']: + tdSql.query(f"select last({i}) from {j}") + tdSql.checkRows(0) + tdSql.query(f"select last({list(column_dict.keys())[0]}) from {stbname}_1 group by {list(column_dict.keys())[-1]}") tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col14) from db.stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col5) from stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 9.1) - tdSql.query("select last(col5) from db.stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 9.1) - tdSql.query("select last(col6) from stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 9.1) - tdSql.query("select last(col6) from db.stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 9.1) - tdSql.query("select last(col7) from stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, True) - tdSql.query("select last(col7) from db.stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, True) - tdSql.query("select last(col8) from stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 'taosdata10') - tdSql.query("select last(col8) from db.stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 'taosdata10') - tdSql.query("select last(col9) from stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, '涛思数据10') - tdSql.query("select last(col9) from db.stb_1") - tdSql.checkRows(1) - tdSql.checkData(0, 0, '涛思数据10') - tdSql.query("select last(col1,col2,col3) from stb_1") - tdSql.checkData(0,2,10) + for i in range(self.rowNum): + tdSql.execute(f"insert into {stbname}_1 values(%d, %d, %d, %d, %d, %d, %d, %d, %d, %f, %f, %d, '{self.binary_str}%d', '{self.nchar_str}%d')" + % (self.ts + i, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 0.1, i + 0.1, i % 2, i + 1, i + 1)) + for i in [f'{stbname}_1', f'db.{stbname}_1', f'{stbname}', f'db.{stbname}']: + tdSql.query(f"select last(*) from {i}") + tdSql.checkRows(1) + tdSql.checkData(0, 1, 10) + for k, v in column_dict.items(): + for j in [f'{stbname}_1', f'db.{stbname}_1', f'{stbname}', f'db.{stbname}']: + tdSql.query(f"select last({k}) from {j}") + tdSql.checkRows(1) + # tinyint,smallint,int,bigint,tinyint unsigned,smallint unsigned,int unsigned,bigint unsigned + if v.lower() == 'tinyint' or v.lower() == 'smallint' or v.lower() == 'int' or v.lower() == 'bigint' or v.lower() == 'tinyint unsigned' or v.lower() == 'smallint unsigned'\ + or v.lower() == 'int unsigned' or v.lower() == 'bigint unsigned': + tdSql.checkData(0, 0, 10) + # float,double + elif v.lower() == 'float' or v.lower() == 'double': + tdSql.checkData(0, 0, 9.1) + # bool + elif v.lower() == 'bool': + tdSql.checkData(0, 0, True) + # binary + elif 'binary' in v.lower(): + tdSql.checkData(0, 0, f'{self.binary_str}{self.rowNum}') + # nchar + elif 'nchar' in v.lower(): + tdSql.checkData(0, 0, f'{self.nchar_str}{self.rowNum}') + for i in [f'{stbname}_1', f'db.{stbname}_1', f'{stbname}', f'db.{stbname}']: + tdSql.query(f"select last({list(column_dict.keys())[0]},{list(column_dict.keys())[1]},{list(column_dict.keys())[2]}) from {stbname}_1") + tdSql.checkData(0, 2, 10) - tdSql.query("select last(*) from stb") - tdSql.checkRows(1) - tdSql.checkData(0, 1, 10) - tdSql.query("select last(*) from db.stb") - tdSql.checkRows(1) - tdSql.checkData(0, 1, 10) - tdSql.query("select last(col1) from stb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col1) from db.stb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col2) from stb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col2) from db.stb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col3) from stb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col3) from db.stb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col4) from stb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col4) from db.stb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col11) from stb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col11) from db.stb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col12) from stb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col12) from db.stb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col13) from stb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col13) from db.stb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col14) from stb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col14) from db.stb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col5) from stb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 9.1) - tdSql.query("select last(col5) from db.stb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 9.1) - tdSql.query("select last(col6) from stb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 9.1) - tdSql.query("select last(col6) from db.stb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 9.1) - tdSql.query("select last(col7) from stb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, True) - tdSql.query("select last(col7) from db.stb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, True) - tdSql.query("select last(col8) from stb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 'taosdata10') - tdSql.query("select last(col8) from db.stb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 'taosdata10') - tdSql.query("select last(col9) from stb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, '涛思数据10') - tdSql.query("select last(col9) from db.stb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, '涛思数据10') - tdSql.query("select last(col1,col2,col3) from stb") - tdSql.checkData(0,2,10) - + tdSql.error(f"select {list(column_dict.keys())[0]} from {stbname} where last({list(column_dict.keys())[12]})='涛思数据10'") + tdSql.error(f"select {list(column_dict.keys())[0]} from {stbname}_1 where last({list(column_dict.keys())[12]})='涛思数据10'") + tdSql.execute('drop database db') - tdSql.execute('''create table ntb(ts timestamp, col1 tinyint, col2 smallint, col3 int, col4 bigint, col5 float, col6 double, - col7 bool, col8 binary(20), col9 nchar(20), col11 tinyint unsigned, col12 smallint unsigned, col13 int unsigned, col14 bigint unsigned)''') - tdSql.execute("insert into ntb(ts) values(%d)" % (self.ts - 1)) - tdSql.query("select last(*) from ntb") + def last_check_ntb_base(self): + tdSql.prepare() + ntbname = tdCom.getLongName(5, "letters") + column_dict = { + 'col1': 'tinyint', + 'col2': 'smallint', + 'col3': 'int', + 'col4': 'bigint', + 'col5': 'tinyint unsigned', + 'col6': 'smallint unsigned', + 'col7': 'int unsigned', + 'col8': 'bigint unsigned', + 'col9': 'float', + 'col10': 'double', + 'col11': 'bool', + 'col12': 'binary(20)', + 'col13': 'nchar(20)' + } + create_ntb_sql = self.set_create_normaltable_sql(ntbname, column_dict) + tdSql.execute(create_ntb_sql) + tdSql.execute(f"insert into {ntbname}(ts) values(%d)" % (self.ts - 1)) + tdSql.query(f"select last(*) from {ntbname}") tdSql.checkRows(1) tdSql.checkData(0, 1, None) - tdSql.query("select last(*) from db.ntb") + tdSql.query(f"select last(*) from db.{ntbname}") tdSql.checkRows(1) tdSql.checkData(0, 1, None) - tdSql.query("select last(col1) from ntb") - tdSql.checkRows(0) - tdSql.query("select last(col1) from db.ntb") - tdSql.checkRows(0) - tdSql.query("select last(col2) from ntb") - tdSql.checkRows(0) - tdSql.query("select last(col2) from db.ntb") - tdSql.checkRows(0) - tdSql.query("select last(col3) from ntb") - tdSql.checkRows(0) - tdSql.query("select last(col3) from db.ntb") - tdSql.checkRows(0) - tdSql.query("select last(col4) from ntb") - tdSql.checkRows(0) - tdSql.query("select last(col4) from db.ntb") - tdSql.checkRows(0) - tdSql.query("select last(col11) from ntb") - tdSql.checkRows(0) - tdSql.query("select last(col11) from db.ntb") - tdSql.checkRows(0) - tdSql.query("select last(col12) from ntb") - tdSql.checkRows(0) - tdSql.query("select last(col12) from db.ntb") - tdSql.checkRows(0) - tdSql.query("select last(col13) from ntb") - tdSql.checkRows(0) - tdSql.query("select last(col13) from db.ntb") - tdSql.checkRows(0) - tdSql.query("select last(col14) from ntb") - tdSql.checkRows(0) - tdSql.query("select last(col14) from db.ntb") - tdSql.checkRows(0) - tdSql.query("select last(col5) from ntb") - tdSql.checkRows(0) - tdSql.query("select last(col5) from db.ntb") - tdSql.checkRows(0) - tdSql.query("select last(col6) from ntb") - tdSql.checkRows(0) - tdSql.query("select last(col6) from db.ntb") - tdSql.checkRows(0) - tdSql.query("select last(col7) from ntb") - tdSql.checkRows(0) - tdSql.query("select last(col7) from db.ntb") - tdSql.checkRows(0) - tdSql.query("select last(col8) from ntb") - tdSql.checkRows(0) - tdSql.query("select last(col8) from db.ntb") - tdSql.checkRows(0) - tdSql.query("select last(col9) from ntb") - tdSql.checkRows(0) - tdSql.query("select last(col9) from db.ntb") - tdSql.checkRows(0) - + for i in column_dict.keys(): + for j in [f'{ntbname}', f'db.{ntbname}']: + tdSql.query(f"select last({i}) from {j}") + tdSql.checkRows(0) for i in range(self.rowNum): - tdSql.execute("insert into ntb values(%d, %d, %d, %d, %d, %f, %f, %d, 'taosdata%d', '涛思数据%d', %d, %d, %d, %d)" - % (self.ts + i, i + 1, i + 1, i + 1, i + 1, i + 0.1, i + 0.1, i % 2, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1)) - - tdSql.query("select last(*) from ntb") + tdSql.execute(f"insert into {ntbname} values(%d, %d, %d, %d, %d, %d, %d, %d, %d, %f, %f, %d, '{self.binary_str}%d', '{self.nchar_str}%d')" + % (self.ts + i, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 0.1, i + 0.1, i % 2, i + 1, i + 1)) + tdSql.query(f"select last(*) from {ntbname}") tdSql.checkRows(1) tdSql.checkData(0, 1, 10) - tdSql.query("select last(*) from db.ntb") + tdSql.query(f"select last(*) from db.{ntbname}") tdSql.checkRows(1) tdSql.checkData(0, 1, 10) - tdSql.query("select last(col1) from ntb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col1) from db.ntb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col2) from ntb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col2) from db.ntb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col3) from ntb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col3) from db.ntb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col4) from ntb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col4) from db.ntb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col11) from ntb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col11) from db.ntb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col12) from ntb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col12) from db.ntb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col13) from ntb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col13) from db.ntb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col14) from ntb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col14) from db.ntb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 10) - tdSql.query("select last(col5) from ntb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 9.1) - tdSql.query("select last(col5) from db.ntb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 9.1) - tdSql.query("select last(col6) from ntb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 9.1) - tdSql.query("select last(col6) from db.ntb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 9.1) - tdSql.query("select last(col7) from ntb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, True) - tdSql.query("select last(col7) from db.ntb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, True) - tdSql.query("select last(col8) from ntb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 'taosdata10') - tdSql.query("select last(col8) from db.ntb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, 'taosdata10') - tdSql.query("select last(col9) from ntb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, '涛思数据10') - tdSql.query("select last(col9) from db.ntb") - tdSql.checkRows(1) - tdSql.checkData(0, 0, '涛思数据10') - tdSql.query("select last(col1,col2,col3) from ntb") - tdSql.checkData(0,2,10) + for k, v in column_dict.items(): + for j in [f'{ntbname}', f'db.{ntbname}']: + tdSql.query(f"select last({k}) from {j}") + tdSql.checkRows(1) + # tinyint,smallint,int,bigint,tinyint unsigned,smallint unsigned,int unsigned,bigint unsigned + if v.lower() == 'tinyint' or v.lower() == 'smallint' or v.lower() == 'int' or v.lower() == 'bigint' or v.lower() == 'tinyint unsigned' or v.lower() == 'smallint unsigned'\ + or v.lower() == 'int unsigned' or v.lower() == 'bigint unsigned': + tdSql.checkData(0, 0, 10) + # float,double + elif v.lower() == 'float' or v.lower() == 'double': + tdSql.checkData(0, 0, 9.1) + # bool + elif v.lower() == 'bool': + tdSql.checkData(0, 0, True) + # binary + elif 'binary' in v.lower(): + tdSql.checkData(0, 0, f'{self.binary_str}{self.rowNum}') + # nchar + elif 'nchar' in v.lower(): + tdSql.checkData(0, 0, f'{self.nchar_str}{self.rowNum}') + + tdSql.error( + f"select {list(column_dict.keys())[0]} from {ntbname} where last({list(column_dict.keys())[9]})='涛思数据10'") + + def last_check_stb_distribute(self): + # prepare data for vgroup 4 + dbname = tdCom.getLongName(10, "letters") + stbname = tdCom.getLongName(5, "letters") + vgroup_num = 4 + column_dict = { + 'col1': 'tinyint', + 'col2': 'smallint', + 'col3': 'int', + 'col4': 'bigint', + 'col5': 'tinyint unsigned', + 'col6': 'smallint unsigned', + 'col7': 'int unsigned', + 'col8': 'bigint unsigned', + 'col9': 'float', + 'col10': 'double', + 'col11': 'bool', + 'col12': 'binary(20)', + 'col13': 'nchar(20)' + } + + tdSql.execute( + f"create database if not exists {dbname} vgroups {vgroup_num}") + tdSql.execute(f'use {dbname}') + + # build 20 child tables,every table insert 10 rows + tdSql.execute(f'''create table {stbname}(ts timestamp, col1 tinyint, col2 smallint, col3 int, col4 bigint, col5 tinyint unsigned, col6 smallint unsigned, + col7 int unsigned, col8 bigint unsigned, col9 float, col10 double, col11 bool, col12 binary(20), col13 nchar(20)) tags(loc nchar(20))''') + for i in range(self.tbnum): + tdSql.execute( + f"create table {stbname}_{i} using {stbname} tags('beijing')") + tdSql.execute( + f"insert into {stbname}_{i}(ts) values(%d)" % (self.ts - 1-i)) + # for i in [f'{stbname}', f'{dbname}.{stbname}']: + # tdSql.query(f"select last(*) from {i}") + # tdSql.checkRows(1) + # tdSql.checkData(0, 1, None) + tdSql.query('show tables') + vgroup_list = [] + for i in range(len(tdSql.queryResult)): + vgroup_list.append(tdSql.queryResult[i][6]) + vgroup_list_set = set(vgroup_list) + for i in vgroup_list_set: + vgroups_num = vgroup_list.count(i) + if vgroups_num >= 2: + tdLog.info(f'This scene with {vgroups_num} vgroups is ok!') + continue + else: + tdLog.exit( + 'This scene does not meet the requirements with {vgroups_num} vgroup!\n') + + for i in range(self.tbnum): + for j in range(self.rowNum): + tdSql.execute(f"insert into {stbname}_{i} values(%d, %d, %d, %d, %d, %d, %d, %d, %d, %f, %f, %d, '{self.binary_str}%d', '{self.nchar_str}%d')" + % (self.ts + j + i, j + 1, j + 1, j + 1, j + 1, j + 1, j + 1, j + 1, j + 1, j + 0.1, j + 0.1, j % 2, j + 1, j + 1)) + for i in [f'{stbname}', f'{dbname}.{stbname}']: + tdSql.query(f"select last(*) from {i}") + tdSql.checkRows(1) + tdSql.checkData(0, 1, 10) + for k, v in column_dict.items(): + for j in [f'{stbname}', f'{dbname}.{stbname}']: + tdSql.query(f"select last({k}) from {j}") + tdSql.checkRows(1) + # tinyint,smallint,int,bigint,tinyint unsigned,smallint unsigned,int unsigned,bigint unsigned + if v.lower() == 'tinyint' or v.lower() == 'smallint' or v.lower() == 'int' or v.lower() == 'bigint' or v.lower() == 'tinyint unsigned' or v.lower() == 'smallint unsigned'\ + or v.lower() == 'int unsigned' or v.lower() == 'bigint unsigned': + tdSql.checkData(0, 0, 10) + # float,double + elif v.lower() == 'float' or v.lower() == 'double': + tdSql.checkData(0, 0, 9.1) + # bool + elif v.lower() == 'bool': + tdSql.checkData(0, 0, True) + # binary + elif 'binary' in v.lower(): + tdSql.checkData(0, 0, f'{self.binary_str}{self.rowNum}') + # nchar + elif 'nchar' in v.lower(): + tdSql.checkData(0, 0, f'{self.nchar_str}{self.rowNum}') + tdSql.execute(f'drop database {dbname}') + + def run(self): + self.last_check_stb_tb_base() + self.last_check_ntb_base() + self.last_check_stb_distribute() - tdSql.error("select col1 from stb where last(col9)='涛思数据10'") - tdSql.error("select col1 from ntb where last(col9)='涛思数据10'") - tdSql.error("select col1 from stb_1 where last(col9)='涛思数据10'") def stop(self): tdSql.close() tdLog.success("%s successfully executed" % __file__) + tdCases.addWindows(__file__, TDTestCase()) -tdCases.addLinux(__file__, TDTestCase()) \ No newline at end of file +tdCases.addLinux(__file__, TDTestCase()) diff --git a/tests/system-test/2-query/max.py b/tests/system-test/2-query/max.py index 5342c7d44988949f4754040b77ea416d20b79658..46426a5a9592fdc0e0b7d8695eecc92b24395c61 100644 --- a/tests/system-test/2-query/max.py +++ b/tests/system-test/2-query/max.py @@ -5,198 +5,213 @@ import numpy as np class TDTestCase: + updatecfgDict = {'debugFlag': 143 ,"cDebugFlag":143,"uDebugFlag":143 ,"rpcDebugFlag":143 , "tmrDebugFlag":143 , + "jniDebugFlag":143 ,"simDebugFlag":143,"dDebugFlag":143, "dDebugFlag":143,"vDebugFlag":143,"mDebugFlag":143,"qDebugFlag":143, + "wDebugFlag":143,"sDebugFlag":143,"tsdbDebugFlag":143,"tqDebugFlag":143 ,"fsDebugFlag":143 ,"fnDebugFlag":143, + "maxTablesPerVnode":2 ,"minTablesPerVnode":2,"tableIncStepPerVnode":2 } def init(self, conn, logSql): tdLog.debug("start to execute %s" % __file__) tdSql.init(conn.cursor()) self.rowNum = 10 self.ts = 1537146000000 - - def prepare_data(self): - - pass - def run(self): + self.binary_str = 'taosdata' + self.nchar_str = '涛思数据' + def max_check_stb_and_tb_base(self): tdSql.prepare() - intData = [] floatData = [] - - tdSql.execute('''create table stb(ts timestamp, col1 tinyint, col2 smallint, col3 int, col4 bigint, col5 float, col6 double, - col7 bool, col8 binary(20), col9 nchar(20), col11 tinyint unsigned, col12 smallint unsigned, col13 int unsigned, col14 bigint unsigned) tags(loc nchar(20))''') + tdSql.execute('''create table stb(ts timestamp, col1 tinyint, col2 smallint, col3 int, col4 bigint, col5 tinyint unsigned, col6 smallint unsigned, + col7 int unsigned, col8 bigint unsigned, col9 float, col10 double, col11 bool, col12 binary(20), col13 nchar(20)) tags(loc nchar(20))''') tdSql.execute("create table stb_1 using stb tags('beijing')") - tdSql.execute('''create table ntb(ts timestamp, col1 tinyint, col2 smallint, col3 int, col4 bigint, col5 float, col6 double, - col7 bool, col8 binary(20), col9 nchar(20), col11 tinyint unsigned, col12 smallint unsigned, col13 int unsigned, col14 bigint unsigned)''') for i in range(self.rowNum): - tdSql.execute("insert into ntb values(%d, %d, %d, %d, %d, %f, %f, %d, 'taosdata%d', '涛思数据%d', %d, %d, %d, %d)" - % (self.ts + i, i + 1, i + 1, i + 1, i + 1, i + 0.1, i + 0.1, i % 2, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1)) + tdSql.execute(f"insert into stb_1 values(%d, %d, %d, %d, %d, %d, %d, %d, %d, %f, %f, %d, '{self.binary_str}%d', '{self.nchar_str}%d')" + % (self.ts + i, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 0.1, i + 0.1, i % 2, i + 1, i + 1)) intData.append(i + 1) floatData.append(i + 0.1) + for i in ['ts','col11','col12','col13']: + for j in ['db.stb','stb','db.stb_1','stb_1']: + tdSql.error(f'select max({i} from {j} )') + + for i in range(1,11): + for j in ['db.stb','stb','db.stb_1','stb_1']: + tdSql.query(f"select max(col{i}) from {j}") + if i<9: + tdSql.checkData(0, 0, np.max(intData)) + elif i>=9: + tdSql.checkData(0, 0, np.max(floatData)) + tdSql.query("select max(col1) from stb_1 where col2<=5") + tdSql.checkData(0,0,5) + tdSql.query("select max(col1) from stb where col2<=5") + tdSql.checkData(0,0,5) + tdSql.execute('drop database db') + + def max_check_ntb_base(self): + tdSql.prepare() + intData = [] + floatData = [] + tdSql.execute('''create table ntb(ts timestamp, col1 tinyint, col2 smallint, col3 int, col4 bigint, col5 tinyint unsigned, col6 smallint unsigned, + col7 int unsigned, col8 bigint unsigned, col9 float, col10 double, col11 bool, col12 binary(20), col13 nchar(20))''') for i in range(self.rowNum): - tdSql.execute("insert into stb_1 values(%d, %d, %d, %d, %d, %f, %f, %d, 'taosdata%d', '涛思数据%d', %d, %d, %d, %d)" - % (self.ts + i, i + 1, i + 1, i + 1, i + 1, i + 0.1, i + 0.1, i % 2, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1)) + tdSql.execute(f"insert into ntb values(%d, %d, %d, %d, %d, %d, %d, %d, %d, %f, %f, %d, '{self.binary_str}%d', '{self.nchar_str}%d')" + % (self.ts + i, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 0.1, i + 0.1, i % 2, i + 1, i + 1)) intData.append(i + 1) - floatData.append(i + 0.1) - - # max verifacation - tdSql.error("select max(ts) from stb_1") - tdSql.error("select max(ts) from db.stb_1") - tdSql.error("select max(col7) from stb_1") - tdSql.error("select max(col7) from db.stb_1") - tdSql.error("select max(col8) from stb_1") - tdSql.error("select max(col8) from db.stb_1") - tdSql.error("select max(col9) from stb_1") - tdSql.error("select max(col9) from db.stb_1") - - tdSql.query("select max(col1) from stb_1") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col1) from db.stb_1") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col2) from stb_1") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col2) from db.stb_1") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col3) from stb_1") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col3) from db.stb_1") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col4) from stb_1") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col4) from db.stb_1") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col11) from stb_1") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col11) from db.stb_1") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col12) from stb_1") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col12) from db.stb_1") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col13) from stb_1") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col13) from db.stb_1") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col14) from stb_1") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col14) from db.stb_1") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col5) from stb_1") - tdSql.checkData(0, 0, np.max(floatData)) - tdSql.query("select max(col5) from db.stb_1") - tdSql.checkData(0, 0, np.max(floatData)) - tdSql.query("select max(col6) from stb_1") - tdSql.checkData(0, 0, np.max(floatData)) - tdSql.query("select max(col6) from db.stb_1") - tdSql.checkData(0, 0, np.max(floatData)) - tdSql.query("select max(col1) from stb_1 where col2<=5") + floatData.append(i + 0.1) + for i in ['ts','col11','col12','col13']: + for j in ['db.ntb','ntb']: + tdSql.error(f'select max({i} from {j} )') + for i in range(1,11): + for j in ['db.ntb','ntb']: + tdSql.query(f"select max(col{i}) from {j}") + if i<9: + tdSql.checkData(0, 0, np.max(intData)) + elif i>=9: + tdSql.checkData(0, 0, np.max(floatData)) + tdSql.query("select max(col1) from ntb where col2<=5") tdSql.checkData(0,0,5) + tdSql.execute('drop database db') + + + def check_max_functions(self, tbname , col_name): + + max_sql = f"select max({col_name}) from {tbname};" + + same_sql = f"select {col_name} from {tbname} order by {col_name} desc limit 1" + + tdSql.query(max_sql) + max_result = tdSql.queryResult + + tdSql.query(same_sql) + same_result = tdSql.queryResult + + if max_result !=same_result: + tdLog.exit(" max function work not as expected, sql : %s "% max_sql) + else: + tdLog.info(" max function work as expected, sql : %s "% max_sql) + + + def support_distributed_aggregate(self): + + # prepate datas for 20 tables distributed at different vgroups + tdSql.execute("create database if not exists testdb keep 3650 duration 1000 vgroups 5") + tdSql.execute(" use testdb ") + tdSql.execute( + '''create table stb1 + (ts timestamp, c1 int, c2 bigint, c3 smallint, c4 tinyint, c5 float, c6 double, c7 bool, c8 binary(16),c9 nchar(32), c10 timestamp) + tags (t0 timestamp, t1 int, t2 bigint, t3 smallint, t4 tinyint, t5 float, t6 double, t7 bool, t8 binary(16),t9 nchar(32)) + ''' + ) + + tdSql.execute( + ''' + create table t1 + (ts timestamp, c1 int, c2 bigint, c3 smallint, c4 tinyint, c5 float, c6 double, c7 bool, c8 binary(16),c9 nchar(32), c10 timestamp) + ''' + ) + for i in range(20): + tdSql.execute(f'create table ct{i+1} using stb1 tags ( now(), {1*i}, {11111*i}, {111*i}, {11*i}, {1.11*i}, {11.11*i}, {i%2}, "binary{i}", "nchar{i}" )') + + for i in range(9): + tdSql.execute( + f"insert into ct1 values ( now()-{i*10}s, {1*i}, {11111*i}, {111*i}, {11*i}, {1.11*i}, {11.11*i}, {i%2}, 'binary{i}', 'nchar{i}', now()+{1*i}a )" + ) + tdSql.execute( + f"insert into ct4 values ( now()-{i*90}d, {1*i}, {11111*i}, {111*i}, {11*i}, {1.11*i}, {11.11*i}, {i%2}, 'binary{i}', 'nchar{i}', now()+{1*i}a )" + ) + + for i in range(1,21): + if i ==1 or i == 4: + continue + else: + tbname = "ct"+f'{i}' + for j in range(9): + tdSql.execute( + f"insert into {tbname} values ( now()-{(i+j)*10}s, {1*(j+i)}, {11111*(j+i)}, {111*(j+i)}, {11*(j)}, {1.11*(j+i)}, {11.11*(j+i)}, {(j+i)%2}, 'binary{j}', 'nchar{j}', now()+{1*j}a )" + ) + tdSql.execute("insert into ct1 values (now()-45s, 0, 0, 0, 0, 0, 0, 0, 'binary0', 'nchar0', now()+8a )") + tdSql.execute("insert into ct1 values (now()+10s, 9, -99999, -999, -99, -9.99, -99.99, 1, 'binary9', 'nchar9', now()+9a )") + tdSql.execute("insert into ct1 values (now()+15s, 9, -99999, -999, -99, -9.99, NULL, 1, 'binary9', 'nchar9', now()+9a )") + tdSql.execute("insert into ct1 values (now()+20s, 9, -99999, -999, NULL, -9.99, -99.99, 1, 'binary9', 'nchar9', now()+9a )") + + tdSql.execute("insert into ct4 values (now()-810d, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) ") + tdSql.execute("insert into ct4 values (now()-400d, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) ") + tdSql.execute("insert into ct4 values (now()+90d, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) ") + + tdSql.execute( + f'''insert into t1 values + ( '2020-04-21 01:01:01.000', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) + ( '2020-10-21 01:01:01.000', 1, 11111, 111, 11, 1.11, 11.11, 1, "binary1", "nchar1", now()+1a ) + ( '2020-12-31 01:01:01.000', 2, 22222, 222, 22, 2.22, 22.22, 0, "binary2", "nchar2", now()+2a ) + ( '2021-01-01 01:01:06.000', 3, 33333, 333, 33, 3.33, 33.33, 0, "binary3", "nchar3", now()+3a ) + ( '2021-05-07 01:01:10.000', 4, 44444, 444, 44, 4.44, 44.44, 1, "binary4", "nchar4", now()+4a ) + ( '2021-07-21 01:01:01.000', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) + ( '2021-09-30 01:01:16.000', 5, 55555, 555, 55, 5.55, 55.55, 0, "binary5", "nchar5", now()+5a ) + ( '2022-02-01 01:01:20.000', 6, 66666, 666, 66, 6.66, 66.66, 1, "binary6", "nchar6", now()+6a ) + ( '2022-10-28 01:01:26.000', 7, 00000, 000, 00, 0.00, 00.00, 1, "binary7", "nchar7", "1970-01-01 08:00:00.000" ) + ( '2022-12-01 01:01:30.000', 8, -88888, -888, -88, -8.88, -88.88, 0, "binary8", "nchar8", "1969-01-01 01:00:00.000" ) + ( '2022-12-31 01:01:36.000', 9, -99999999999999999, -999, -99, -9.99, -999999999999999999999.99, 1, "binary9", "nchar9", "1900-01-01 00:00:00.000" ) + ( '2023-02-21 01:01:01.000', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL ) + ''' + ) + + tdLog.info(" prepare data for distributed_aggregate done! ") + + # get vgroup_ids of all + tdSql.query("show vgroups ") + vgroups = tdSql.queryResult + + vnode_tables={} + + for vgroup_id in vgroups: + vnode_tables[vgroup_id[0]]=[] + + + # check sub_table of per vnode ,make sure sub_table has been distributed + tdSql.query("show tables like 'ct%'") + table_names = tdSql.queryResult + tablenames = [] + for table_name in table_names: + vnode_tables[table_name[6]].append(table_name[0]) + + count = 0 + for k ,v in vnode_tables.items(): + if len(v)>=2: + count+=1 + if count < 2: + tdLog.exit(" the datas of all not satisfy sub_table has been distributed ") + + # check max function work status + + tdSql.query("show tables like 'ct%'") + table_names = tdSql.queryResult + tablenames = [] + for table_name in table_names: + tablenames.append(table_name[0]) + + tdSql.query("desc stb1") + col_names = tdSql.queryResult + + colnames = [] + for col_name in col_names: + if col_name[1] in ["INT" ,"BIGINT" ,"SMALLINT" ,"TINYINT" , "FLOAT" ,"DOUBLE"]: + colnames.append(col_name[0]) + for tablename in tablenames: + for colname in colnames: + self.check_max_functions(tablename,colname) + # max function with basic filter + print(vnode_tables) - tdSql.error("select max(ts) from stb") - tdSql.error("select max(ts) from db.stb") - tdSql.error("select max(col7) from stb") - tdSql.error("select max(col7) from db.stb") - tdSql.error("select max(col8) from stb") - tdSql.error("select max(col8) from db.stb") - tdSql.error("select max(col9) from stb") - tdSql.error("select max(col9) from db.stb") - - tdSql.query("select max(col1) from stb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col1) from db.stb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col2) from stb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col2) from db.stb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col3) from stb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col3) from db.stb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col4) from stb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col4) from db.stb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col11) from stb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col11) from db.stb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col12) from stb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col12) from db.stb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col13) from stb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col13) from db.stb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col14) from stb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col14) from db.stb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col5) from stb") - tdSql.checkData(0, 0, np.max(floatData)) - tdSql.query("select max(col5) from db.stb") - tdSql.checkData(0, 0, np.max(floatData)) - tdSql.query("select max(col6) from stb") - tdSql.checkData(0, 0, np.max(floatData)) - tdSql.query("select max(col6) from db.stb") - tdSql.checkData(0, 0, np.max(floatData)) - tdSql.query("select max(col1) from stb where col2<=5") - tdSql.checkData(0,0,5) + def run(self): + # max verifacation + self.max_check_stb_and_tb_base() + self.max_check_ntb_base() + + self.support_distributed_aggregate() - tdSql.error("select max(ts) from ntb") - tdSql.error("select max(ts) from db.ntb") - tdSql.error("select max(col7) from ntb") - tdSql.error("select max(col7) from db.ntb") - tdSql.error("select max(col8) from ntb") - tdSql.error("select max(col8) from db.ntb") - tdSql.error("select max(col9) from ntb") - tdSql.error("select max(col9) from db.ntb") - - tdSql.query("select max(col1) from ntb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col1) from db.ntb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col2) from ntb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col2) from db.ntb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col3) from ntb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col3) from db.ntb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col4) from ntb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col4) from db.ntb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col11) from ntb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col11) from db.ntb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col12) from ntb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col12) from db.ntb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col13) from ntb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col13) from db.ntb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col14) from ntb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col14) from db.ntb") - tdSql.checkData(0, 0, np.max(intData)) - tdSql.query("select max(col5) from ntb") - tdSql.checkData(0, 0, np.max(floatData)) - tdSql.query("select max(col5) from db.ntb") - tdSql.checkData(0, 0, np.max(floatData)) - tdSql.query("select max(col6) from ntb") - tdSql.checkData(0, 0, np.max(floatData)) - tdSql.query("select max(col6) from db.ntb") - tdSql.checkData(0, 0, np.max(floatData)) - tdSql.query("select max(col1) from stb_1 where col2<=5") - tdSql.checkData(0,0,5) def stop(self): tdSql.close() tdLog.success("%s successfully executed" % __file__) diff --git a/tests/system-test/2-query/tail.py b/tests/system-test/2-query/tail.py index 6039f3effa95f8014b31fd529138a872a6b9bedf..0e2110f9122c318dc0eff34a44dc453469c57bca 100644 --- a/tests/system-test/2-query/tail.py +++ b/tests/system-test/2-query/tail.py @@ -189,6 +189,7 @@ class TDTestCase: def check_tail_table(self , tbname , col_name , tail_rows , offset): tail_sql = f"select tail({col_name} , {tail_rows} , {offset}) from {tbname}" equal_sql = f"select {col_name} from (select ts , {col_name} from {tbname} order by ts desc limit {tail_rows} offset {offset}) order by ts" + #equal_sql = f"select {col_name} from {tbname} order by ts desc limit {tail_rows} offset {offset}" tdSql.query(tail_sql) tail_result = tdSql.queryResult @@ -293,22 +294,22 @@ class TDTestCase: tdSql.checkData(0, 0, None) tdSql.checkData(1, 0, None) - tdSql.query("select tail(c1,3,2) from ct4 where c1 >2 ") - tdSql.checkData(0, 0, 7) + tdSql.query("select tail(c1,3,2) from ct4 where c1 >2 order by 1") + tdSql.checkData(0, 0, 5) tdSql.checkData(1, 0, 6) - tdSql.checkData(2, 0, 5) + tdSql.checkData(2, 0, 7) - tdSql.query("select tail(c1,2,1) from ct4 where c2 between 0 and 99999") - tdSql.checkData(0, 0, 2) - tdSql.checkData(1, 0, 1) + tdSql.query("select tail(c1,2,1) from ct4 where c2 between 0 and 99999 order by 1") + tdSql.checkData(0, 0, 1) + tdSql.checkData(1, 0, 2) # tail with union all tdSql.query("select tail(c1,2,1) from ct4 union all select c1 from ct1") tdSql.checkRows(15) - tdSql.query("select tail(c1,2,1) from ct4 union all select c1 from ct2") + tdSql.query("select tail(c1,2,1) from ct4 union all select c1 from ct2 order by 1") tdSql.checkRows(2) - tdSql.checkData(0, 0, 1) - tdSql.checkData(1, 0, 0) + tdSql.checkData(0, 0, 0) + tdSql.checkData(1, 0, 1) tdSql.query("select tail(c2,2,1) from ct4 union all select abs(c2)/2 from ct4") tdSql.checkRows(14) @@ -334,18 +335,18 @@ class TDTestCase: tdSql.execute(f" insert into ttb1 values({ts_value} , {i})") tdSql.execute(f" insert into ttb2 values({ts_value} , {i})") - tdSql.query("select tail(tb2.num,3,2) from tb1, tb2 where tb1.ts=tb2.ts ") + tdSql.query("select tail(tb2.num,3,2) from tb1, tb2 where tb1.ts=tb2.ts order by 1 desc") tdSql.checkRows(3) - tdSql.checkData(0,0,5) + tdSql.checkData(0,0,7) tdSql.checkData(1,0,6) - tdSql.checkData(2,0,7) + tdSql.checkData(2,0,5) # nest query # tdSql.query("select tail(c1,2) from (select c1 from ct1)") - tdSql.query("select c1 from (select tail(c1,2) c1 from ct4)") + tdSql.query("select c1 from (select tail(c1,2) c1 from ct4) order by 1 nulls first") tdSql.checkRows(2) - tdSql.checkData(0, 0, 0) - tdSql.checkData(1, 0, None) + tdSql.checkData(0, 0, None) + tdSql.checkData(1, 0, 0) tdSql.query("select sum(c1) from (select tail(c1,2) c1 from ct1)") tdSql.checkRows(1) diff --git a/tests/system-test/2-query/top.py b/tests/system-test/2-query/top.py index 146bb349377ef539cfab4e294337621a807cfa27..83f535856ee8b9456d7759128063d5149df82ea1 100644 --- a/tests/system-test/2-query/top.py +++ b/tests/system-test/2-query/top.py @@ -11,6 +11,9 @@ # -*- coding: utf-8 -*- +import random +import string +from util.common import * from util.log import * from util.cases import * from util.sql import * @@ -22,82 +25,89 @@ class TDTestCase: tdSql.init(conn.cursor()) self.rowNum = 10 + self.tbnum = 20 self.ts = 1537146000000 - - def run(self): + self.binary_str = 'taosdata' + self.nchar_str = '涛思数据' + def top_check_base(self): tdSql.prepare() - - - - tdSql.execute('''create table test(ts timestamp, col1 tinyint, col2 smallint, col3 int, col4 bigint, col5 float, col6 double, - col7 bool, col8 binary(20), col9 nchar(20), col11 tinyint unsigned, col12 smallint unsigned, col13 int unsigned, col14 bigint unsigned) tags(loc nchar(20))''') - tdSql.execute("create table test1 using test tags('beijing')") + tdSql.execute('''create table stb(ts timestamp, col1 tinyint, col2 smallint, col3 int, col4 bigint, col5 tinyint unsigned, col6 smallint unsigned, + col7 int unsigned, col8 bigint unsigned, col9 float, col10 double, col11 bool, col12 binary(20), col13 nchar(20)) tags(loc nchar(20))''') + tdSql.execute("create table stb_1 using stb tags('beijing')") for i in range(self.rowNum): - tdSql.execute("insert into test1 values(%d, %d, %d, %d, %d, %f, %f, %d, 'taosdata%d', '涛思数据%d', %d, %d, %d, %d)" - % (self.ts + i, i + 1, i + 1, i + 1, i + 1, i + 0.1, i + 0.1, i % 2, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1)) - - - # top verifacation - tdSql.error("select top(ts, 10) from test") - tdSql.error("select top(col1, 0) from test") - tdSql.error("select top(col1, 101) from test") - tdSql.error("select top(col2, 0) from test") - tdSql.error("select top(col2, 101) from test") - tdSql.error("select top(col3, 0) from test") - tdSql.error("select top(col3, 101) from test") - tdSql.error("select top(col4, 0) from test") - tdSql.error("select top(col4, 101) from test") - tdSql.error("select top(col5, 0) from test") - tdSql.error("select top(col5, 101) from test") - tdSql.error("select top(col6, 0) from test") - tdSql.error("select top(col6, 101) from test") - tdSql.error("select top(col7, 10) from test") - tdSql.error("select top(col8, 10) from test") - tdSql.error("select top(col9, 10) from test") - tdSql.error("select top(col11, 0) from test") - tdSql.error("select top(col11, 101) from test") - tdSql.error("select top(col12, 0) from test") - tdSql.error("select top(col12, 101) from test") - tdSql.error("select top(col13, 0) from test") - tdSql.error("select top(col13, 101) from test") - tdSql.error("select top(col14, 0) from test") - tdSql.error("select top(col14, 101) from test") - - tdSql.query("select top(col1, 2) from test") - tdSql.checkRows(2) - tdSql.checkEqual(tdSql.queryResult,[(9,),(10,)]) - tdSql.query("select top(col2, 2) from test") - tdSql.checkRows(2) - tdSql.checkEqual(tdSql.queryResult,[(9,),(10,)]) - tdSql.query("select top(col3, 2) from test") - tdSql.checkRows(2) - tdSql.checkEqual(tdSql.queryResult,[(9,),(10,)]) - tdSql.query("select top(col4, 2) from test") + tdSql.execute(f"insert into stb_1 values(%d, %d, %d, %d, %d, %d, %d, %d, %d, %f, %f, %d, '{self.binary_str}%d', '{self.nchar_str}%d')" + % (self.ts + i, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 0.1, i + 0.1, i % 2, i + 1, i + 1)) + column_list = ['col1','col2','col3','col4','col5','col6','col7','col8'] + error_column_list = ['col11','col12','col13'] + error_param_list = [0,101] + for i in column_list: + tdSql.query(f'select top({i},2) from stb_1') + tdSql.checkRows(2) + tdSql.checkEqual(tdSql.queryResult,[(9,),(10,)]) + for j in error_param_list: + tdSql.error(f'select top({i},{j}) from stb_1') + for i in error_column_list: + tdSql.error(f'select top({i},10) from stb_1') + tdSql.query("select ts,top(col1, 2),ts from stb_1 group by tbname") tdSql.checkRows(2) - tdSql.checkEqual(tdSql.queryResult,[(9,),(10,)]) - tdSql.query("select top(col11, 2) from test") - tdSql.checkRows(2) - tdSql.checkEqual(tdSql.queryResult,[(9,),(10,)]) - tdSql.query("select top(col12, 2) from test") - tdSql.checkRows(2) - tdSql.checkEqual(tdSql.queryResult,[(9,),(10,)]) - tdSql.query("select top(col13, 2) from test") - tdSql.checkRows(2) - tdSql.checkEqual(tdSql.queryResult,[(9,),(10,)]) - tdSql.query("select top(col14, 2) from test") - tdSql.checkRows(2) - tdSql.checkEqual(tdSql.queryResult,[(9,),(10,)]) - tdSql.query("select ts,top(col1, 2),ts from test1") - tdSql.checkRows(2) - tdSql.query("select top(col14, 100) from test") - tdSql.checkRows(10) - tdSql.query("select ts,top(col1, 2),ts from test group by tbname") - tdSql.checkRows(2) - tdSql.query('select top(col2,1) from test interval(1y) order by col2') + tdSql.query('select top(col2,1) from stb_1 interval(1y) order by col2') tdSql.checkData(0,0,10) + tdSql.error("select * from stb_1 where top(col2,1)=1") + tdSql.execute('drop database db') + def top_check_stb_distribute(self): + # prepare data for vgroup 4 + dbname = tdCom.getLongName(10, "letters") + stbname = tdCom.getLongName(5, "letters") + tdSql.execute(f"create database if not exists {dbname} vgroups 2") + tdSql.execute(f'use {dbname}') + # build 20 child tables,every table insert 10 rows + tdSql.execute(f'''create table {stbname}(ts timestamp, col1 tinyint, col2 smallint, col3 int, col4 bigint, col5 tinyint unsigned, col6 smallint unsigned, + col7 int unsigned, col8 bigint unsigned, col9 float, col10 double, col11 bool, col12 binary(20), col13 nchar(20)) tags(loc nchar(20))''') + for i in range(self.tbnum): + tdSql.execute(f"create table {stbname}_{i} using {stbname} tags('beijing')") + tdSql.execute(f"insert into {stbname}_{i}(ts) values(%d)" % (self.ts - 1-i)) + column_list = ['col1','col2','col3','col4','col5','col6','col7','col8'] + for i in [f'{stbname}', f'{dbname}.{stbname}']: + for j in column_list: + tdSql.query(f"select top({j},1) from {i}") + tdSql.checkRows(0) + tdSql.query('show tables') + vgroup_list = [] + for i in range(len(tdSql.queryResult)): + vgroup_list.append(tdSql.queryResult[i][6]) + vgroup_list_set = set(vgroup_list) + + for i in vgroup_list_set: + vgroups_num = vgroup_list.count(i) + if vgroups_num >=2: + tdLog.info(f'This scene with {vgroups_num} vgroups is ok!') + continue + else: + tdLog.exit(f'This scene does not meet the requirements with {vgroups_num} vgroup!\n') + for i in range(self.rowNum): + for j in range(self.tbnum): + tdSql.execute(f"insert into {stbname}_{j} values(%d, %d, %d, %d, %d, %d, %d, %d, %d, %f, %f, %d, '{self.binary_str}%d', '{self.nchar_str}%d')" + % (self.ts + i, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, i + 0.1, i + 0.1, i % 2, i + 1, i + 1)) + + error_column_list = ['col11','col12','col13'] + error_param_list = [0,101] + for i in column_list: + tdSql.query(f'select top({i},2) from {stbname}') + tdSql.checkRows(2) + tdSql.checkEqual(tdSql.queryResult,[(10,),(10,)]) + for j in error_param_list: + tdSql.error(f'select top({i},{j}) from {stbname}') + for i in error_column_list: + tdSql.error(f'select top({i},10) from {stbname}') - tdSql.error("select * from test where bottom(col2,1)=1") - tdSql.error("select top(col14, 0) from test;") + tdSql.query(f"select ts,top(col1, 2),ts from {stbname} group by tbname") + tdSql.checkRows(2*self.tbnum) + tdSql.query(f'select top(col2,1) from {stbname} interval(1y) order by col2') + tdSql.checkData(0,0,10) + tdSql.error(f"select * from {stbname} where top(col2,1)=1") + def run(self): + self.top_check_base() + self.top_check_stb_distribute() def stop(self): tdSql.close() tdLog.success("%s successfully executed" % __file__) diff --git a/tests/system-test/2-query/unique.py b/tests/system-test/2-query/unique.py index 227efa6f9ceda24df73830bd46838fd657b67d48..f910ff14393395ec658522c1c0dad2a56ad7946a 100644 --- a/tests/system-test/2-query/unique.py +++ b/tests/system-test/2-query/unique.py @@ -93,8 +93,8 @@ class TDTestCase: "select unique(c1) , min(c1) from t1", "select unique(c1) , spread(c1) from t1", "select unique(c1) , diff(c1) from t1", - "select unique(c1) , abs(c1) from t1", - "select unique(c1) , c1 from t1", + #"select unique(c1) , abs(c1) from t1", # support + #"select unique(c1) , c1 from t1", "select unique from stb1 partition by tbname", "select unique(123--123)==1 from stb1 partition by tbname", "select unique(123) from stb1 partition by tbname", @@ -104,21 +104,21 @@ class TDTestCase: "select unique(c1 ,c2 ) from stb1 partition by tbname", "select unique(c1 ,NULL) from stb1 partition by tbname", "select unique(,) from stb1 partition by tbname;", - "select unique(floor(c1) ab from stb1 partition by tbname)", - "select unique(c1) as int from stb1 partition by tbname", + #"select unique(floor(c1) ab from stb1 partition by tbname)", # support + #"select unique(c1) as int from stb1 partition by tbname", "select unique('c1') from stb1 partition by tbname", "select unique(NULL) from stb1 partition by tbname", "select unique('') from stb1 partition by tbname", "select unique(c%) from stb1 partition by tbname", - #"select unique(t1) from stb1 partition by tbname", + #"select unique(t1) from stb1 partition by tbname", # support "select unique(True) from stb1 partition by tbname", "select unique(c1) , count(c1) from stb1 partition by tbname", "select unique(c1) , avg(c1) from stb1 partition by tbname", "select unique(c1) , min(c1) from stb1 partition by tbname", "select unique(c1) , spread(c1) from stb1 partition by tbname", "select unique(c1) , diff(c1) from stb1 partition by tbname", - "select unique(c1) , abs(c1) from stb1 partition by tbname", - "select unique(c1) , c1 from stb1 partition by tbname" + #"select unique(c1) , abs(c1) from stb1 partition by tbname", # support + #"select unique(c1) , c1 from stb1 partition by tbname" # support ] for error_sql in error_sql_lists: @@ -198,7 +198,7 @@ class TDTestCase: unique_datas = [] for elem in unique_result: unique_datas.append(elem[0]) - + unique_datas.sort(key=lambda x: (x is None, x)) tdSql.query(origin_sql) origin_result = tdSql.queryResult @@ -212,6 +212,7 @@ class TDTestCase: continue else: pre_unique.append(elem) + pre_unique.sort(key=lambda x: (x is None, x)) if pre_unique == unique_datas: tdLog.info(" unique query check pass , unique sql is: %s" %unique_sql) @@ -266,16 +267,16 @@ class TDTestCase: tdSql.checkRows(10) tdSql.error("select unique(c1),tbname from ct1") - tdSql.error("select unique(c1),t1 from ct1") + #tdSql.error("select unique(c1),t1 from ct1") #support # unique with common col - tdSql.error("select unique(c1) ,ts from ct1") - tdSql.error("select unique(c1) ,c1 from ct1") + #tdSql.error("select unique(c1) ,ts from ct1") + #tdSql.error("select unique(c1) ,c1 from ct1") # unique with scalar function - tdSql.error("select unique(c1) ,abs(c1) from ct1") + #tdSql.error("select unique(c1) ,abs(c1) from ct1") tdSql.error("select unique(c1) , unique(c2) from ct1") - tdSql.error("select unique(c1) , abs(c2)+2 from ct1") + #tdSql.error("select unique(c1) , abs(c2)+2 from ct1") # unique with aggregate function @@ -288,13 +289,13 @@ class TDTestCase: tdSql.query("select unique(c1) from ct4 where c1 is null") tdSql.checkData(0, 0, None) - tdSql.query("select unique(c1) from ct4 where c1 >2 ") - tdSql.checkData(0, 0, 8) - tdSql.checkData(1, 0, 7) - tdSql.checkData(2, 0, 6) - tdSql.checkData(5, 0, 3) + tdSql.query("select unique(c1) from ct4 where c1 >2 order by 1") + tdSql.checkData(0, 0, 3) + tdSql.checkData(1, 0, 4) + tdSql.checkData(2, 0, 5) + tdSql.checkData(5, 0, 8) - tdSql.query("select unique(c1) from ct4 where c2 between 0 and 99999") + tdSql.query("select unique(c1) from ct4 where c2 between 0 and 99999 order by 1 desc") tdSql.checkData(0, 0, 8) tdSql.checkData(1, 0, 7) tdSql.checkData(2, 0, 6) @@ -335,23 +336,23 @@ class TDTestCase: tdSql.execute(f" insert into ttb1 values({ts_value} , {i})") tdSql.execute(f" insert into ttb2 values({ts_value} , {i})") - tdSql.query("select unique(tb2.num) from tb1, tb2 where tb1.ts=tb2.ts ") + tdSql.query("select unique(tb2.num) from tb1, tb2 where tb1.ts=tb2.ts order by 1") tdSql.checkRows(10) tdSql.checkData(0,0,0) tdSql.checkData(1,0,1) tdSql.checkData(2,0,2) tdSql.checkData(9,0,9) - tdSql.query("select unique(tb2.num) from tb1, tb2 where tb1.ts=tb2.ts union all select unique(tb1.num) from tb1, tb2 where tb1.ts=tb2.ts ") + tdSql.query("select unique(tb2.num) from tb1, tb2 where tb1.ts=tb2.ts union all select unique(tb1.num) from tb1, tb2 where tb1.ts=tb2.ts order by 1") tdSql.checkRows(20) tdSql.checkData(0,0,0) - tdSql.checkData(1,0,1) - tdSql.checkData(2,0,2) - tdSql.checkData(9,0,9) + tdSql.checkData(2,0,1) + tdSql.checkData(4,0,2) + tdSql.checkData(18,0,9) # nest query # tdSql.query("select unique(c1) from (select c1 from ct1)") - tdSql.query("select c1 from (select unique(c1) c1 from ct4)") + tdSql.query("select c1 from (select unique(c1) c1 from ct4) order by 1 desc nulls first") tdSql.checkRows(10) tdSql.checkData(0, 0, None) tdSql.checkData(1, 0, 8) @@ -366,7 +367,7 @@ class TDTestCase: tdSql.checkData(0, 0, 45) tdSql.checkData(1, 0, 45) - tdSql.query("select 1-abs(c1) from (select unique(c1) c1 from ct4)") + tdSql.query("select 1-abs(c1) from (select unique(c1) c1 from ct4) order by 1 nulls first") tdSql.checkRows(10) tdSql.checkData(0, 0, None) tdSql.checkData(1, 0, -7.000000000) @@ -421,7 +422,7 @@ class TDTestCase: f"insert into sub1_bound values ( now()+1s, 2147483648, 9223372036854775808, 32768, 128, 3.40E+38, 1.7e+308, True, 'binary_tb1', 'nchar_tb1', now() )" ) - tdSql.query("select unique(c2) from sub1_bound") + tdSql.query("select unique(c2) from sub1_bound order by 1 desc") tdSql.checkRows(5) tdSql.checkData(0,0,9223372036854775807) diff --git a/tests/system-test/6-cluster/5dnode3mnodeDrop.py b/tests/system-test/6-cluster/5dnode3mnodeDrop.py index f999a16b05ee654767f89f3627eb2664b0084bce..b98134f5e094b96cb5b49c697a910daa1d60349d 100644 --- a/tests/system-test/6-cluster/5dnode3mnodeDrop.py +++ b/tests/system-test/6-cluster/5dnode3mnodeDrop.py @@ -269,7 +269,8 @@ class TDTestCase: tdSql.query("show dnodes;") print(tdSql.queryResult) - # drop and follower of mnode + + # drop follower of mnode dropcount =0 while dropcount <= 10: for i in range(1,3): diff --git a/tests/system-test/6-cluster/5dnode3mnodeDropInsert.py b/tests/system-test/6-cluster/5dnode3mnodeDropInsert.py new file mode 100644 index 0000000000000000000000000000000000000000..cfa39206048b6579238ddf55d74cd10e351e0edc --- /dev/null +++ b/tests/system-test/6-cluster/5dnode3mnodeDropInsert.py @@ -0,0 +1,428 @@ +from ssl import ALERT_DESCRIPTION_CERTIFICATE_UNOBTAINABLE +import taos +import sys +import time +import os + +from util.log import * +from util.sql import * +from util.cases import * +from util.dnodes import TDDnodes +from util.dnodes import TDDnode +import time +import socket +import subprocess +from multiprocessing import Process +import threading +import time +import inspect +import ctypes +class MyDnodes(TDDnodes): + def __init__(self ,dnodes_lists): + super(MyDnodes,self).__init__() + self.dnodes = dnodes_lists # dnode must be TDDnode instance + self.simDeployed = False + + +class TDTestCase: + + def init(self,conn ,logSql): + tdLog.debug(f"start to excute {__file__}") + self.TDDnodes = None + self.ts = 1500000000000 + + def buildcluster(self,dnodenumber): + self.depoly_cluster(dnodenumber) + self.master_dnode = self.TDDnodes.dnodes[0] + self.host=self.master_dnode.cfgDict["fqdn"] + conn1 = taos.connect(self.master_dnode.cfgDict["fqdn"] , config=self.master_dnode.cfgDir) + tdSql.init(conn1.cursor()) + + + def getBuildPath(self): + selfPath = os.path.dirname(os.path.realpath(__file__)) + + if ("community" in selfPath): + projPath = selfPath[:selfPath.find("community")] + else: + projPath = selfPath[:selfPath.find("tests")] + + for root, dirs, files in os.walk(projPath): + if ("taosd" in files): + rootRealPath = os.path.dirname(os.path.realpath(root)) + if ("packaging" not in rootRealPath): + buildPath = root[:len(root) - len("/build/bin")] + break + return buildPath + + def _async_raise(self, tid, exctype): + """raises the exception, performs cleanup if needed""" + if not inspect.isclass(exctype): + exctype = type(exctype) + res = ctypes.pythonapi.PyThreadState_SetAsyncExc(tid, ctypes.py_object(exctype)) + if res == 0: + raise ValueError("invalid thread id") + elif res != 1: + # """if it returns a number greater than one, you're in trouble, + # and you should call it again with exc=NULL to revert the effect""" + ctypes.pythonapi.PyThreadState_SetAsyncExc(tid, None) + raise SystemError("PyThreadState_SetAsyncExc failed") + + def stop_thread(self,thread): + self._async_raise(thread.ident, SystemExit) + + + def createDbTbale(self,dbcountStart,dbcountStop,stbname,chilCount): + # fisrt add data : db\stable\childtable\general table + + for couti in range(dbcountStart,dbcountStop): + tdLog.debug("drop database if exists db%d" %couti) + tdSql.execute("drop database if exists db%d" %couti) + print("create database if not exists db%d replica 1 duration 300" %couti) + tdSql.execute("create database if not exists db%d replica 1 duration 300" %couti) + tdSql.execute("use db%d" %couti) + tdSql.execute( + '''create table %s + (ts timestamp, c1 int, c2 bigint,c3 binary(16), c4 timestamp) + tags (t1 int) + '''%stbname + ) + tdSql.execute( + ''' + create table t1 + (ts timestamp, c1 int, c2 bigint, c3 smallint, c4 tinyint, c5 float, c6 double, c7 bool, c8 binary(16),c9 nchar(32), c10 timestamp) + ''' + ) + for i in range(chilCount): + tdSql.execute(f'create table {stbname}_{i+1} using {stbname} tags ( {i+1} )') + + def insertTabaleData(self,dbcountStart,dbcountStop,stbname,chilCount,ts_start,rowCount): + # insert data : create childtable and data + + for couti in range(dbcountStart,dbcountStop): + tdSql.execute("use db%d" %couti) + pre_insert = "insert into " + sql = pre_insert + chilCount=int(chilCount) + allRows=chilCount*rowCount + tdLog.debug("doing insert data into stable-index:%s rows:%d ..."%(stbname, allRows)) + exeStartTime=time.time() + for i in range(0,chilCount): + sql += " %s_%d values "%(stbname,i) + for j in range(rowCount): + sql += "(%d, %d, %d,'taos_%d',%d) "%(ts_start + j*1000, j, j, j, ts_start + j*1000) + if j >0 and j%4000 == 0: + # print(sql) + tdSql.execute(sql) + sql = "insert into %s_%d values " %(stbname,i) + # end sql + if sql != pre_insert: + # print(sql) + print(len(sql)) + tdSql.execute(sql) + exeEndTime=time.time() + spendTime=exeEndTime-exeStartTime + speedInsert=allRows/spendTime + tdLog.debug("spent %.2fs to INSERT %d rows into %s , insert rate is %.2f rows/s... [OK]"% (spendTime,allRows,stbname,speedInsert)) + + def checkData(self,dbname,stbname,stableCount,CtableCount,rowsPerSTable,): + tdSql.execute("use %s"%dbname) + tdSql.query("show stables") + tdSql.checkRows(stableCount) + tdSql.query("show tables") + tdSql.checkRows(CtableCount) + for i in range(stableCount): + tdSql.query("select count(*) from %s%d"%(stbname,i)) + tdSql.checkData(0,0,rowsPerSTable) + return + + + def depoly_cluster(self ,dnodes_nums): + + testCluster = False + valgrind = 0 + hostname = socket.gethostname() + dnodes = [] + start_port = 6030 + start_port_sec = 6130 + for num in range(1, dnodes_nums+1): + dnode = TDDnode(num) + dnode.addExtraCfg("firstEp", f"{hostname}:{start_port}") + dnode.addExtraCfg("fqdn", f"{hostname}") + dnode.addExtraCfg("serverPort", f"{start_port + (num-1)*100}") + dnode.addExtraCfg("monitorFqdn", hostname) + dnode.addExtraCfg("monitorPort", 7043) + dnode.addExtraCfg("secondEp", f"{hostname}:{start_port_sec}") + dnodes.append(dnode) + + self.TDDnodes = MyDnodes(dnodes) + self.TDDnodes.init("") + self.TDDnodes.setTestCluster(testCluster) + self.TDDnodes.setValgrind(valgrind) + self.TDDnodes.stopAll() + for dnode in self.TDDnodes.dnodes: + self.TDDnodes.deploy(dnode.index,{}) + + for dnode in self.TDDnodes.dnodes: + self.TDDnodes.starttaosd(dnode.index) + + # create cluster + for dnode in self.TDDnodes.dnodes[1:]: + # print(dnode.cfgDict) + dnode_id = dnode.cfgDict["fqdn"] + ":" +dnode.cfgDict["serverPort"] + dnode_first_host = dnode.cfgDict["firstEp"].split(":")[0] + dnode_first_port = dnode.cfgDict["firstEp"].split(":")[-1] + cmd = f" taos -h {dnode_first_host} -P {dnode_first_port} -s ' create dnode \"{dnode_id} \" ' ;" + print(cmd) + os.system(cmd) + + time.sleep(2) + tdLog.info(" create cluster with %d dnode done! " %dnodes_nums) + + def checkdnodes(self,dnodenumber): + count=0 + while count < 10: + time.sleep(1) + statusReadyBumber=0 + tdSql.query("show dnodes;") + if tdSql.checkRows(dnodenumber) : + print("dnode is %d nodes"%dnodenumber) + for i in range(dnodenumber): + if tdSql.queryResult[i][4] !='ready' : + status=tdSql.queryResult[i][4] + print("dnode:%d status is %s "%(i,status)) + break + else: + statusReadyBumber+=1 + print(statusReadyBumber) + if statusReadyBumber == dnodenumber : + print("all of %d mnodes is ready in 10s "%dnodenumber) + return True + break + count+=1 + else: + print("%d mnodes is not ready in 10s "%dnodenumber) + return False + + + def check3mnode(self): + count=0 + while count < 10: + time.sleep(1) + tdSql.query("show mnodes;") + if tdSql.checkRows(3) : + print("mnode is three nodes") + if tdSql.queryResult[0][2]=='leader' : + if tdSql.queryResult[1][2]=='follower': + if tdSql.queryResult[2][2]=='follower': + print("three mnodes is ready in 10s") + break + elif tdSql.queryResult[0][2]=='follower' : + if tdSql.queryResult[1][2]=='leader': + if tdSql.queryResult[2][2]=='follower': + print("three mnodes is ready in 10s") + break + elif tdSql.queryResult[0][2]=='follower' : + if tdSql.queryResult[1][2]=='follower': + if tdSql.queryResult[2][2]=='leader': + print("three mnodes is ready in 10s") + break + count+=1 + else: + print("three mnodes is not ready in 10s ") + return -1 + + tdSql.query("show mnodes;") + tdSql.checkRows(3) + tdSql.checkData(0,1,'%s:6030'%self.host) + tdSql.checkData(0,3,'ready') + tdSql.checkData(1,1,'%s:6130'%self.host) + tdSql.checkData(1,3,'ready') + tdSql.checkData(2,1,'%s:6230'%self.host) + tdSql.checkData(2,3,'ready') + + def check3mnode1off(self): + count=0 + while count < 10: + time.sleep(1) + tdSql.query("show mnodes;") + if tdSql.checkRows(3) : + print("mnode is three nodes") + if tdSql.queryResult[0][2]=='offline' : + if tdSql.queryResult[1][2]=='leader': + if tdSql.queryResult[2][2]=='follower': + print("stop mnodes on dnode 2 successfully in 10s") + break + elif tdSql.queryResult[1][2]=='follower': + if tdSql.queryResult[2][2]=='leader': + print("stop mnodes on dnode 2 successfully in 10s") + break + count+=1 + else: + print("stop mnodes on dnode 2 failed in 10s ") + return -1 + tdSql.error("drop mnode on dnode 1;") + + tdSql.query("show mnodes;") + tdSql.checkRows(3) + tdSql.checkData(0,1,'%s:6030'%self.host) + tdSql.checkData(0,2,'offline') + tdSql.checkData(0,3,'ready') + tdSql.checkData(1,1,'%s:6130'%self.host) + tdSql.checkData(1,3,'ready') + tdSql.checkData(2,1,'%s:6230'%self.host) + tdSql.checkData(2,3,'ready') + + def check3mnode2off(self): + count=0 + while count < 40: + time.sleep(1) + tdSql.query("show mnodes;") + if tdSql.checkRows(3) : + print("mnode is three nodes") + if tdSql.queryResult[0][2]=='leader' : + if tdSql.queryResult[1][2]=='offline': + if tdSql.queryResult[2][2]=='follower': + print("stop mnodes on dnode 2 successfully in 10s") + break + count+=1 + else: + print("stop mnodes on dnode 2 failed in 10s ") + return -1 + tdSql.error("drop mnode on dnode 2;") + + tdSql.query("show mnodes;") + tdSql.checkRows(3) + tdSql.checkData(0,1,'%s:6030'%self.host) + tdSql.checkData(0,2,'leader') + tdSql.checkData(0,3,'ready') + tdSql.checkData(1,1,'%s:6130'%self.host) + tdSql.checkData(1,2,'offline') + tdSql.checkData(1,3,'ready') + tdSql.checkData(2,1,'%s:6230'%self.host) + tdSql.checkData(2,2,'follower') + tdSql.checkData(2,3,'ready') + + def check3mnode3off(self): + count=0 + while count < 10: + time.sleep(1) + tdSql.query("show mnodes;") + if tdSql.checkRows(3) : + print("mnode is three nodes") + if tdSql.queryResult[0][2]=='leader' : + if tdSql.queryResult[2][2]=='offline': + if tdSql.queryResult[1][2]=='follower': + print("stop mnodes on dnode 3 successfully in 10s") + break + count+=1 + else: + print("stop mnodes on dnode 3 failed in 10s") + return -1 + tdSql.error("drop mnode on dnode 3;") + tdSql.query("show mnodes;") + tdSql.checkRows(3) + tdSql.checkData(0,1,'%s:6030'%self.host) + tdSql.checkData(0,2,'leader') + tdSql.checkData(0,3,'ready') + tdSql.checkData(1,1,'%s:6130'%self.host) + tdSql.checkData(1,2,'follower') + tdSql.checkData(1,3,'ready') + tdSql.checkData(2,1,'%s:6230'%self.host) + tdSql.checkData(2,2,'offline') + tdSql.checkData(2,3,'ready') + + def five_dnode_three_mnode(self,dnodenumber): + # testcase parameters + vgroups=1 + dbcountStart=0 + dbcountStop=1 + dbname="db" + stbname="stb" + tablesPerStb=1000 + rowsPerTable=100 + startTs=1640966400000 # 2022-01-01 00:00:00.000 + + tdSql.query("show dnodes;") + tdSql.checkData(0,1,'%s:6030'%self.host) + tdSql.checkData(4,1,'%s:6430'%self.host) + tdSql.checkData(0,4,'ready') + tdSql.checkData(4,4,'ready') + tdSql.query("show mnodes;") + tdSql.checkRows(1) + tdSql.checkData(0,1,'%s:6030'%self.host) + tdSql.checkData(0,2,'leader') + tdSql.checkData(0,3,'ready') + + # fisr add three mnodes; + tdSql.execute("create mnode on dnode 2") + tdSql.execute("create mnode on dnode 3") + + # fisrt check statut ready + self.check3mnode() + + tdSql.error("create mnode on dnode 2") + tdSql.query("show dnodes;") + print(tdSql.queryResult) + tdLog.debug("stop all of mnode ") + + # drop follower of mnode and insert data + self.createDbTbale(dbcountStart, dbcountStop,stbname,tablesPerStb) + #(method) insertTabaleData: (dbcountStart: Any, dbcountStop: Any, stbname: Any, chilCount: Any, ts_start: Any, rowCount: Any) -> None + threads=threading.Thread(target=self.insertTabaleData, args=( + dbcountStart, + dbcountStop, + stbname, + tablesPerStb, + startTs, + rowsPerTable)) + + threads.start() + dropcount =0 + while dropcount <= 10: + for i in range(1,3): + tdLog.debug("drop mnode on dnode %d"%(i+1)) + tdSql.execute("drop mnode on dnode %d"%(i+1)) + tdSql.query("show mnodes;") + count=0 + while count<10: + time.sleep(1) + tdSql.query("show mnodes;") + if tdSql.checkRows(2): + print("drop mnode %d successfully"%(i+1)) + break + count+=1 + tdLog.debug("create mnode on dnode %d"%(i+1)) + tdSql.execute("create mnode on dnode %d"%(i+1)) + count=0 + while count<10: + time.sleep(1) + tdSql.query("show mnodes;") + if tdSql.checkRows(3): + print("drop mnode %d successfully"%(i+1)) + break + count+=1 + dropcount+=1 + threads.join() + self.check3mnode() + + + + def getConnection(self, dnode): + host = dnode.cfgDict["fqdn"] + port = dnode.cfgDict["serverPort"] + config_dir = dnode.cfgDir + return taos.connect(host=host, port=int(port), config=config_dir) + + + def run(self): + # print(self.master_dnode.cfgDict) + self.buildcluster(5) + self.five_dnode_three_mnode(5) + + def stop(self): + tdSql.close() + tdLog.success(f"{__file__} successfully executed") + +tdCases.addLinux(__file__, TDTestCase()) +tdCases.addWindows(__file__, TDTestCase()) \ No newline at end of file diff --git a/tests/system-test/6-cluster/5dnode3mnodeSeperate1VnodeStopInsert.py b/tests/system-test/6-cluster/5dnode3mnodeSeperate1VnodeStopInsert.py new file mode 100644 index 0000000000000000000000000000000000000000..1739db09af6f784eb22c7bfbe20aef4b9190720d --- /dev/null +++ b/tests/system-test/6-cluster/5dnode3mnodeSeperate1VnodeStopInsert.py @@ -0,0 +1,377 @@ +from ssl import ALERT_DESCRIPTION_CERTIFICATE_UNOBTAINABLE +import taos +import sys +import time +import os + +from util.log import * +from util.sql import * +from util.cases import * +from util.dnodes import TDDnodes +from util.dnodes import TDDnode +import time +import socket +import subprocess +from multiprocessing import Process +import threading +import time +import inspect +import ctypes +class MyDnodes(TDDnodes): + def __init__(self ,dnodes_lists): + super(MyDnodes,self).__init__() + self.dnodes = dnodes_lists # dnode must be TDDnode instance + self.simDeployed = False + + +class TDTestCase: + + def init(self,conn ,logSql): + tdLog.debug(f"start to excute {__file__}") + self.TDDnodes = None + + def buildcluster(self,dnodenumber): + self.depoly_cluster(dnodenumber) + self.master_dnode = self.TDDnodes.dnodes[0] + self.host=self.master_dnode.cfgDict["fqdn"] + conn1 = taos.connect(self.master_dnode.cfgDict["fqdn"] , config=self.master_dnode.cfgDir) + tdSql.init(conn1.cursor()) + + + def getBuildPath(self): + selfPath = os.path.dirname(os.path.realpath(__file__)) + + if ("community" in selfPath): + projPath = selfPath[:selfPath.find("community")] + else: + projPath = selfPath[:selfPath.find("tests")] + + for root, dirs, files in os.walk(projPath): + if ("taosd" in files): + rootRealPath = os.path.dirname(os.path.realpath(root)) + if ("packaging" not in rootRealPath): + buildPath = root[:len(root) - len("/build/bin")] + break + return buildPath + + def _async_raise(self, tid, exctype): + """raises the exception, performs cleanup if needed""" + if not inspect.isclass(exctype): + exctype = type(exctype) + res = ctypes.pythonapi.PyThreadState_SetAsyncExc(tid, ctypes.py_object(exctype)) + if res == 0: + raise ValueError("invalid thread id") + elif res != 1: + # """if it returns a number greater than one, you're in trouble, + # and you should call it again with exc=NULL to revert the effect""" + ctypes.pythonapi.PyThreadState_SetAsyncExc(tid, None) + raise SystemError("PyThreadState_SetAsyncExc failed") + + def stop_thread(self,thread): + self._async_raise(thread.ident, SystemExit) + + + def insert_data(self,countstart,countstop): + # fisrt add data : db\stable\childtable\general table + + for couti in range(countstart,countstop): + tdLog.debug("drop database if exists db%d" %couti) + tdSql.execute("drop database if exists db%d" %couti) + print("create database if not exists db%d replica 1 duration 300" %couti) + tdSql.execute("create database if not exists db%d replica 1 duration 300" %couti) + tdSql.execute("use db%d" %couti) + tdSql.execute( + '''create table stb1 + (ts timestamp, c1 int, c2 bigint, c3 smallint, c4 tinyint, c5 float, c6 double, c7 bool, c8 binary(16),c9 nchar(32), c10 timestamp) + tags (t1 int) + ''' + ) + tdSql.execute( + ''' + create table t1 + (ts timestamp, c1 int, c2 bigint, c3 smallint, c4 tinyint, c5 float, c6 double, c7 bool, c8 binary(16),c9 nchar(32), c10 timestamp) + ''' + ) + for i in range(4): + tdSql.execute(f'create table ct{i+1} using stb1 tags ( {i+1} )') + + def checkData(self,dbname,stbname,stableCount,CtableCount,rowsPerSTable,): + tdSql.execute("use %s"%dbname) + tdSql.query("show stables") + tdSql.checkRows(stableCount) + tdSql.query("show tables") + tdSql.checkRows(CtableCount) + for i in range(stableCount): + tdSql.query("select count(*) from %s%d"%(stbname,i)) + tdSql.checkData(0,0,rowsPerSTable) + return + + def depoly_cluster(self ,dnodes_nums=5,independent=True): + + testCluster = False + valgrind = 0 + hostname = socket.gethostname() + dnodes = [] + start_port = 6030 + start_port_sec = 6130 + for num in range(1, dnodes_nums+1): + dnode = TDDnode(num) + dnode.addExtraCfg("firstEp", f"{hostname}:{start_port}") + dnode.addExtraCfg("fqdn", f"{hostname}") + dnode.addExtraCfg("serverPort", f"{start_port + (num-1)*100}") + dnode.addExtraCfg("monitorFqdn", hostname) + dnode.addExtraCfg("monitorPort", 7043) + dnode.addExtraCfg("secondEp", f"{hostname}:{start_port_sec}") + # configure three dnoe don't support vnodes + if independent and (num < 4): + dnode.addExtraCfg("supportVnodes", 0) + + dnodes.append(dnode) + + self.TDDnodes = MyDnodes(dnodes) + self.TDDnodes.init("") + self.TDDnodes.setTestCluster(testCluster) + self.TDDnodes.setValgrind(valgrind) + self.TDDnodes.stopAll() + for dnode in self.TDDnodes.dnodes: + self.TDDnodes.deploy(dnode.index,{}) + + for dnode in self.TDDnodes.dnodes: + self.TDDnodes.starttaosd(dnode.index) + + # create cluster + for dnode in self.TDDnodes.dnodes[1:]: + # print(dnode.cfgDict) + dnode_id = dnode.cfgDict["fqdn"] + ":" +dnode.cfgDict["serverPort"] + dnode_first_host = dnode.cfgDict["firstEp"].split(":")[0] + dnode_first_port = dnode.cfgDict["firstEp"].split(":")[-1] + cmd = f" taos -h {dnode_first_host} -P {dnode_first_port} -s ' create dnode \"{dnode_id} \" ' ;" + print(cmd) + os.system(cmd) + + time.sleep(2) + tdLog.info(" create cluster with %d dnode done! " %dnodes_nums) + + def checkdnodes(self,dnodenumber): + count=0 + while count < 100: + time.sleep(1) + statusReadyBumber=0 + tdSql.query("show dnodes;") + if tdSql.checkRows(dnodenumber) : + print("dnode is %d nodes"%dnodenumber) + for i in range(dnodenumber): + if tdSql.queryResult[i][4] !='ready' : + status=tdSql.queryResult[i][4] + print("dnode:%d status is %s "%(i,status)) + break + else: + statusReadyBumber+=1 + print(statusReadyBumber) + if statusReadyBumber == dnodenumber : + print("all of %d mnodes is ready in 10s "%dnodenumber) + return True + break + count+=1 + else: + print("%d mnodes is not ready in 10s "%dnodenumber) + return False + + + def check3mnode(self): + count=0 + while count < 10: + time.sleep(1) + tdSql.query("show mnodes;") + if tdSql.checkRows(3) : + print("mnode is three nodes") + if tdSql.queryResult[0][2]=='leader' : + if tdSql.queryResult[1][2]=='follower': + if tdSql.queryResult[2][2]=='follower': + print("three mnodes is ready in 10s") + break + elif tdSql.queryResult[0][2]=='follower' : + if tdSql.queryResult[1][2]=='leader': + if tdSql.queryResult[2][2]=='follower': + print("three mnodes is ready in 10s") + break + elif tdSql.queryResult[0][2]=='follower' : + if tdSql.queryResult[1][2]=='follower': + if tdSql.queryResult[2][2]=='leader': + print("three mnodes is ready in 10s") + break + count+=1 + else: + print("three mnodes is not ready in 10s ") + return -1 + + tdSql.query("show mnodes;") + tdSql.checkRows(3) + tdSql.checkData(0,1,'%s:6030'%self.host) + tdSql.checkData(0,3,'ready') + tdSql.checkData(1,1,'%s:6130'%self.host) + tdSql.checkData(1,3,'ready') + tdSql.checkData(2,1,'%s:6230'%self.host) + tdSql.checkData(2,3,'ready') + + def check3mnode1off(self): + count=0 + while count < 10: + time.sleep(1) + tdSql.query("show mnodes;") + if tdSql.checkRows(3) : + print("mnode is three nodes") + if tdSql.queryResult[0][2]=='offline' : + if tdSql.queryResult[1][2]=='leader': + if tdSql.queryResult[2][2]=='follower': + print("stop mnodes on dnode 2 successfully in 10s") + break + elif tdSql.queryResult[1][2]=='follower': + if tdSql.queryResult[2][2]=='leader': + print("stop mnodes on dnode 2 successfully in 10s") + break + count+=1 + else: + print("stop mnodes on dnode 2 failed in 10s ") + return -1 + tdSql.error("drop mnode on dnode 1;") + + tdSql.query("show mnodes;") + tdSql.checkRows(3) + tdSql.checkData(0,1,'%s:6030'%self.host) + tdSql.checkData(0,2,'offline') + tdSql.checkData(0,3,'ready') + tdSql.checkData(1,1,'%s:6130'%self.host) + tdSql.checkData(1,3,'ready') + tdSql.checkData(2,1,'%s:6230'%self.host) + tdSql.checkData(2,3,'ready') + + def check3mnode2off(self): + count=0 + while count < 40: + time.sleep(1) + tdSql.query("show mnodes;") + if tdSql.checkRows(3) : + print("mnode is three nodes") + if tdSql.queryResult[0][2]=='leader' : + if tdSql.queryResult[1][2]=='offline': + if tdSql.queryResult[2][2]=='follower': + print("stop mnodes on dnode 2 successfully in 10s") + break + count+=1 + else: + print("stop mnodes on dnode 2 failed in 10s ") + return -1 + tdSql.error("drop mnode on dnode 2;") + + tdSql.query("show mnodes;") + tdSql.checkRows(3) + tdSql.checkData(0,1,'%s:6030'%self.host) + tdSql.checkData(0,2,'leader') + tdSql.checkData(0,3,'ready') + tdSql.checkData(1,1,'%s:6130'%self.host) + tdSql.checkData(1,2,'offline') + tdSql.checkData(1,3,'ready') + tdSql.checkData(2,1,'%s:6230'%self.host) + tdSql.checkData(2,2,'follower') + tdSql.checkData(2,3,'ready') + + def check3mnode3off(self): + count=0 + while count < 10: + time.sleep(1) + tdSql.query("show mnodes;") + if tdSql.checkRows(3) : + print("mnode is three nodes") + if tdSql.queryResult[0][2]=='leader' : + if tdSql.queryResult[2][2]=='offline': + if tdSql.queryResult[1][2]=='follower': + print("stop mnodes on dnode 3 successfully in 10s") + break + count+=1 + else: + print("stop mnodes on dnode 3 failed in 10s") + return -1 + tdSql.error("drop mnode on dnode 3;") + tdSql.query("show mnodes;") + tdSql.checkRows(3) + tdSql.checkData(0,1,'%s:6030'%self.host) + tdSql.checkData(0,2,'leader') + tdSql.checkData(0,3,'ready') + tdSql.checkData(1,1,'%s:6130'%self.host) + tdSql.checkData(1,2,'follower') + tdSql.checkData(1,3,'ready') + tdSql.checkData(2,1,'%s:6230'%self.host) + tdSql.checkData(2,2,'offline') + tdSql.checkData(2,3,'ready') + + def five_dnode_three_mnode(self,dnodenumber): + tdSql.query("show dnodes;") + tdSql.checkData(0,1,'%s:6030'%self.host) + tdSql.checkData(4,1,'%s:6430'%self.host) + tdSql.checkData(0,4,'ready') + tdSql.checkData(4,4,'ready') + tdSql.query("show mnodes;") + tdSql.checkRows(1) + tdSql.checkData(0,1,'%s:6030'%self.host) + tdSql.checkData(0,2,'leader') + tdSql.checkData(0,3,'ready') + + # fisr add three mnodes; + tdSql.execute("create mnode on dnode 2") + tdSql.execute("create mnode on dnode 3") + + # fisrt check statut ready + self.check3mnode() + + tdSql.error("create mnode on dnode 2") + tdSql.query("show dnodes;") + print(tdSql.queryResult) + tdLog.debug("stop all of mnode ") + + # seperate vnode and mnode in different dnodes. + # create database and stable + stopcount =0 + while stopcount < 2: + for i in range(dnodenumber): + # threads=[] + # threads = MyThreadFunc(self.insert_data(i*2,i*2+2)) + threads=threading.Thread(target=self.insert_data, args=(i,i+1)) + threads.start() + self.TDDnodes.stoptaosd(i+1) + self.TDDnodes.starttaosd(i+1) + + if self.checkdnodes(5): + print("123") + threads.join() + else: + print("456") + self.stop_thread(threads) + assert 1 == 2 ,"some dnode started failed" + return False + # self.check3mnode() + self.check3mnode() + + + stopcount+=1 + self.check3mnode() + + + def getConnection(self, dnode): + host = dnode.cfgDict["fqdn"] + port = dnode.cfgDict["serverPort"] + config_dir = dnode.cfgDir + return taos.connect(host=host, port=int(port), config=config_dir) + + + def run(self): + # print(self.master_dnode.cfgDict) + self.buildcluster(5) + self.five_dnode_three_mnode(5) + + def stop(self): + tdSql.close() + tdLog.success(f"{__file__} successfully executed") + +tdCases.addLinux(__file__, TDTestCase()) +tdCases.addWindows(__file__, TDTestCase()) \ No newline at end of file diff --git a/tests/system-test/7-tmq/basic5.py b/tests/system-test/7-tmq/basic5.py index e44f327995facc02ca01aab6e92e9e41ecac36f4..4ed3be967eb1611b3cddd967b5d141ef770e19d3 100644 --- a/tests/system-test/7-tmq/basic5.py +++ b/tests/system-test/7-tmq/basic5.py @@ -192,7 +192,7 @@ class TDTestCase: time.sleep(1) tdLog.info("start consume processor") - pollDelay = 100 + pollDelay = 20 showMsg = 1 showRow = 1 @@ -208,7 +208,7 @@ class TDTestCase: os.system(shellCmd) # wait for data ready - prepareEnvThread.join() + # prepareEnvThread.join() tdLog.info("insert process end, and start to check consume result") while 1: diff --git a/tests/system-test/7-tmq/subscribeDb0.py b/tests/system-test/7-tmq/subscribeDb0.py index c9f256ed7495850bd1a88b9a6690f23a85cb68fa..4e8fb04517dc228329235994174c482d59868383 100644 --- a/tests/system-test/7-tmq/subscribeDb0.py +++ b/tests/system-test/7-tmq/subscribeDb0.py @@ -322,176 +322,6 @@ class TDTestCase: tdLog.printNoPrefix("======== test case 5 end ...... ") - def tmqCase6(self, cfgPath, buildPath): - tdLog.printNoPrefix("======== test case 6: Produce while one consumers to subscribe tow topic, Each contains one db") - tdLog.info("step 1: create database, stb, ctb and insert data") - # create and start thread - parameterDict = {'cfg': '', \ - 'dbName': 'db60', \ - 'vgroups': 4, \ - 'stbName': 'stb', \ - 'ctbNum': 10, \ - 'rowsPerTbl': 5000, \ - 'batchNum': 100, \ - 'startTs': 1640966400000} # 2022-01-01 00:00:00.000 - parameterDict['cfg'] = cfgPath - - self.initConsumerTable() - - tdSql.execute("create database if not exists %s vgroups %d" %(parameterDict['dbName'], parameterDict['vgroups'])) - - prepareEnvThread = threading.Thread(target=self.prepareEnv, kwargs=parameterDict) - prepareEnvThread.start() - - parameterDict2 = {'cfg': '', \ - 'dbName': 'db61', \ - 'vgroups': 4, \ - 'stbName': 'stb2', \ - 'ctbNum': 10, \ - 'rowsPerTbl': 5000, \ - 'batchNum': 100, \ - 'startTs': 1640966400000} # 2022-01-01 00:00:00.000 - parameterDict['cfg'] = cfgPath - - tdSql.execute("create database if not exists %s vgroups %d" %(parameterDict2['dbName'], parameterDict2['vgroups'])) - - prepareEnvThread2 = threading.Thread(target=self.prepareEnv, kwargs=parameterDict2) - prepareEnvThread2.start() - - tdLog.info("create topics from db") - topicName1 = 'topic_db60' - topicName2 = 'topic_db61' - - tdSql.execute("create topic %s as database %s" %(topicName1, parameterDict['dbName'])) - tdSql.execute("create topic %s as database %s" %(topicName2, parameterDict2['dbName'])) - - consumerId = 0 - expectrowcnt = parameterDict["rowsPerTbl"] * parameterDict["ctbNum"] + parameterDict2["rowsPerTbl"] * parameterDict2["ctbNum"] - topicList = topicName1 + ',' + topicName2 - ifcheckdata = 0 - ifManualCommit = 0 - keyList = 'group.id:cgrp1,\ - enable.auto.commit:false,\ - auto.commit.interval.ms:6000,\ - auto.offset.reset:earliest' - self.insertConsumerInfo(consumerId, expectrowcnt,topicList,keyList,ifcheckdata,ifManualCommit) - - #consumerId = 1 - #self.insertConsumerInfo(consumerId, expectrowcnt,topicList,keyList,ifcheckdata,ifManualCommit) - - event.wait() - - tdLog.info("start consume processor") - pollDelay = 100 - showMsg = 1 - showRow = 1 - self.startTmqSimProcess(buildPath,cfgPath,pollDelay,parameterDict["dbName"],showMsg, showRow) - - # wait for data ready - prepareEnvThread.join() - prepareEnvThread2.join() - - tdLog.info("insert process end, and start to check consume result") - expectRows = 1 - resultList = self.selectConsumeResult(expectRows) - totalConsumeRows = 0 - for i in range(expectRows): - totalConsumeRows += resultList[i] - - if totalConsumeRows != expectrowcnt: - tdLog.info("act consume rows: %d, expect consume rows: %d"%(totalConsumeRows, expectrowcnt)) - tdLog.exit("tmq consume rows error!") - - tdSql.query("drop topic %s"%topicName1) - tdSql.query("drop topic %s"%topicName2) - - tdLog.printNoPrefix("======== test case 6 end ...... ") - - def tmqCase7(self, cfgPath, buildPath): - tdLog.printNoPrefix("======== test case 7: Produce while two consumers to subscribe tow topic, Each contains one db") - tdLog.info("step 1: create database, stb, ctb and insert data") - # create and start thread - parameterDict = {'cfg': '', \ - 'dbName': 'db70', \ - 'vgroups': 4, \ - 'stbName': 'stb', \ - 'ctbNum': 10, \ - 'rowsPerTbl': 5000, \ - 'batchNum': 100, \ - 'startTs': 1640966400000} # 2022-01-01 00:00:00.000 - parameterDict['cfg'] = cfgPath - - self.initConsumerTable() - - tdSql.execute("create database if not exists %s vgroups %d" %(parameterDict['dbName'], parameterDict['vgroups'])) - - prepareEnvThread = threading.Thread(target=self.prepareEnv, kwargs=parameterDict) - prepareEnvThread.start() - - parameterDict2 = {'cfg': '', \ - 'dbName': 'db71', \ - 'vgroups': 4, \ - 'stbName': 'stb2', \ - 'ctbNum': 10, \ - 'rowsPerTbl': 5000, \ - 'batchNum': 100, \ - 'startTs': 1640966400000} # 2022-01-01 00:00:00.000 - parameterDict['cfg'] = cfgPath - - tdSql.execute("create database if not exists %s vgroups %d" %(parameterDict2['dbName'], parameterDict2['vgroups'])) - - prepareEnvThread2 = threading.Thread(target=self.prepareEnv, kwargs=parameterDict2) - prepareEnvThread2.start() - - tdLog.info("create topics from db") - topicName1 = 'topic_db60' - topicName2 = 'topic_db61' - - tdSql.execute("create topic %s as database %s" %(topicName1, parameterDict['dbName'])) - tdSql.execute("create topic %s as database %s" %(topicName2, parameterDict2['dbName'])) - - consumerId = 0 - expectrowcnt = parameterDict["rowsPerTbl"] * parameterDict["ctbNum"] + parameterDict2["rowsPerTbl"] * parameterDict2["ctbNum"] - topicList = topicName1 + ',' + topicName2 - ifcheckdata = 0 - ifManualCommit = 1 - keyList = 'group.id:cgrp1,\ - enable.auto.commit:false,\ - auto.commit.interval.ms:6000,\ - auto.offset.reset:earliest' - self.insertConsumerInfo(consumerId, expectrowcnt,topicList,keyList,ifcheckdata,ifManualCommit) - - consumerId = 1 - self.insertConsumerInfo(consumerId, expectrowcnt,topicList,keyList,ifcheckdata,ifManualCommit) - - event.wait() - - tdLog.info("start consume processor") - pollDelay = 100 - showMsg = 1 - showRow = 1 - self.startTmqSimProcess(buildPath,cfgPath,pollDelay,parameterDict["dbName"],showMsg, showRow) - - # wait for data ready - prepareEnvThread.join() - prepareEnvThread2.join() - - tdLog.info("insert process end, and start to check consume result") - expectRows = 2 - resultList = self.selectConsumeResult(expectRows) - totalConsumeRows = 0 - for i in range(expectRows): - totalConsumeRows += resultList[i] - - if totalConsumeRows != expectrowcnt: - tdLog.info("act consume rows: %d, expect consume rows: %d"%(totalConsumeRows, expectrowcnt)) - tdLog.exit("tmq consume rows error!") - - tdSql.query("drop topic %s"%topicName1) - tdSql.query("drop topic %s"%topicName2) - - tdLog.printNoPrefix("======== test case 7 end ...... ") - def run(self): tdSql.prepare() @@ -505,8 +335,6 @@ class TDTestCase: self.tmqCase4(cfgPath, buildPath) self.tmqCase5(cfgPath, buildPath) - self.tmqCase6(cfgPath, buildPath) - self.tmqCase7(cfgPath, buildPath) def stop(self): diff --git a/tests/system-test/7-tmq/subscribeDb1.py b/tests/system-test/7-tmq/subscribeDb1.py index ed92a429aee5cc10cfc2b5964a215225f813c908..28a341f8f353cd16120420fa098ca46b1a458edb 100644 --- a/tests/system-test/7-tmq/subscribeDb1.py +++ b/tests/system-test/7-tmq/subscribeDb1.py @@ -72,10 +72,10 @@ class TDTestCase: if tdSql.getRows() == expectRows: break else: - time.sleep(5) - + time.sleep(5) + for i in range(expectRows): - tdLog.info ("ts: %s, consume id: %d, consume msgs: %d, consume rows: %d"%(tdSql.getData(i , 0), tdSql.getData(i , 1), tdSql.getData(i , 2), tdSql.getData(i , 3))) + tdLog.info ("consume id: %d, consume msgs: %d, consume rows: %d"%(tdSql.getData(i , 1), tdSql.getData(i , 2), tdSql.getData(i , 3))) resultList.append(tdSql.getData(i , 3)) return resultList @@ -85,7 +85,7 @@ class TDTestCase: logFile = cfgPath + '/../log/valgrind-tmq.log' shellCmd = 'nohup valgrind --log-file=' + logFile shellCmd += '--tool=memcheck --leak-check=full --show-reachable=no --track-origins=yes --show-leak-kinds=all --num-callers=20 -v --workaround-gcc296-bugs=yes ' - + if (platform.system().lower() == 'windows'): shellCmd = 'mintty -h never -w hide ' + buildPath + '\\build\\bin\\tmq_sim.exe -c ' + cfgPath shellCmd += " -y %d -d %s -g %d -r %d -w %s "%(pollDelay, dbName, showMsg, showRow, cdbName) @@ -97,7 +97,7 @@ class TDTestCase: tdLog.info(shellCmd) os.system(shellCmd) - def create_tables(self,tsql, dbName,vgroups,stbName,ctbNum,rowsPerTbl): + def create_tables(self,tsql, dbName,vgroups,stbName,ctbNum): tsql.execute("create database if not exists %s vgroups %d"%(dbName, vgroups)) tsql.execute("use %s" %dbName) tsql.execute("create table if not exists %s (ts timestamp, c1 bigint, c2 binary(16)) tags(t1 int)"%stbName) @@ -151,8 +151,7 @@ class TDTestCase: parameterDict["dbName"],\ parameterDict["vgroups"],\ parameterDict["stbName"],\ - parameterDict["ctbNum"],\ - parameterDict["rowsPerTbl"]) + parameterDict["ctbNum"]) self.insert_data(tsql,\ parameterDict["dbName"],\ @@ -163,16 +162,16 @@ class TDTestCase: parameterDict["startTs"]) return - def tmqCase8(self, cfgPath, buildPath): - tdLog.printNoPrefix("======== test case 8: Produce while one consume to subscribe one db, inclue 1 stb") + def tmqCase6(self, cfgPath, buildPath): + tdLog.printNoPrefix("======== test case 6: Produce while one consumers to subscribe tow topic, Each contains one db") tdLog.info("step 1: create database, stb, ctb and insert data") # create and start thread parameterDict = {'cfg': '', \ - 'dbName': 'db8', \ + 'dbName': 'db60', \ 'vgroups': 4, \ 'stbName': 'stb', \ 'ctbNum': 10, \ - 'rowsPerTbl': 10000, \ + 'rowsPerTbl': 5000, \ 'batchNum': 100, \ 'startTs': 1640966400000} # 2022-01-01 00:00:00.000 parameterDict['cfg'] = cfgPath @@ -183,100 +182,43 @@ class TDTestCase: prepareEnvThread = threading.Thread(target=self.prepareEnv, kwargs=parameterDict) prepareEnvThread.start() - - tdLog.info("create topics from db") - topicName1 = 'topic_db1' - - tdSql.execute("create topic %s as database %s" %(topicName1, parameterDict['dbName'])) - consumerId = 0 - expectrowcnt = parameterDict["rowsPerTbl"] * parameterDict["ctbNum"] / 2 - topicList = topicName1 - ifcheckdata = 0 - ifManualCommit = 0 - keyList = 'group.id:cgrp1,\ - enable.auto.commit:false,\ - auto.commit.interval.ms:6000,\ - auto.offset.reset:earliest' - self.insertConsumerInfo(consumerId, expectrowcnt,topicList,keyList,ifcheckdata,ifManualCommit) - - event.wait() - tdLog.info("start consume processor") - pollDelay = 100 - showMsg = 1 - showRow = 1 - self.startTmqSimProcess(buildPath,cfgPath,pollDelay,parameterDict["dbName"],showMsg, showRow) - - # wait for data ready - prepareEnvThread.join() - - tdLog.info("insert process end, and start to check consume result") - expectRows = 1 - resultList = self.selectConsumeResult(expectRows) - totalConsumeRows = 0 - for i in range(expectRows): - totalConsumeRows += resultList[i] - - if totalConsumeRows != expectrowcnt: - tdLog.info("act consume rows: %d, expect consume rows: %d"%(totalConsumeRows, expectrowcnt)) - tdLog.exit("tmq consume rows error!") - - - tdLog.info("again start consume processer") - self.initConsumerTable() - expectrowcnt = parameterDict["rowsPerTbl"] * parameterDict["ctbNum"] - self.insertConsumerInfo(consumerId, expectrowcnt,topicList,keyList,ifcheckdata,ifManualCommit) - self.startTmqSimProcess(buildPath,cfgPath,pollDelay,parameterDict["dbName"],showMsg, showRow) - expectRows = 1 - resultList = self.selectConsumeResult(expectRows) - totalConsumeRows = 0 - for i in range(expectRows): - totalConsumeRows += resultList[i] - - if totalConsumeRows != expectrowcnt: - tdLog.info("act consume rows: %d, expect consume rows: %d"%(totalConsumeRows, expectrowcnt)) - tdLog.exit("tmq consume rows error!") - - tdSql.query("drop topic %s"%topicName1) - - tdLog.printNoPrefix("======== test case 8 end ...... ") - - def tmqCase9(self, cfgPath, buildPath): - tdLog.printNoPrefix("======== test case 9: Produce while one consume to subscribe one db, inclue 1 stb") - tdLog.info("step 1: create database, stb, ctb and insert data") - # create and start thread - parameterDict = {'cfg': '', \ - 'dbName': 'db9', \ + parameterDict2 = {'cfg': '', \ + 'dbName': 'db61', \ 'vgroups': 4, \ - 'stbName': 'stb', \ + 'stbName': 'stb2', \ 'ctbNum': 10, \ - 'rowsPerTbl': 10000, \ + 'rowsPerTbl': 5000, \ 'batchNum': 100, \ 'startTs': 1640966400000} # 2022-01-01 00:00:00.000 parameterDict['cfg'] = cfgPath - self.initConsumerTable() + tdSql.execute("create database if not exists %s vgroups %d" %(parameterDict2['dbName'], parameterDict2['vgroups'])) - tdSql.execute("create database if not exists %s vgroups %d" %(parameterDict['dbName'], parameterDict['vgroups'])) + prepareEnvThread2 = threading.Thread(target=self.prepareEnv, kwargs=parameterDict2) + prepareEnvThread2.start() - prepareEnvThread = threading.Thread(target=self.prepareEnv, kwargs=parameterDict) - prepareEnvThread.start() - tdLog.info("create topics from db") - topicName1 = 'topic_db1' + topicName1 = 'topic_db60' + topicName2 = 'topic_db61' + + tdSql.execute("create topic %s as database %s" %(topicName1, parameterDict['dbName'])) + tdSql.execute("create topic %s as database %s" %(topicName2, parameterDict2['dbName'])) - tdSql.execute("create topic %s as database %s" %(topicName1, parameterDict['dbName'])) consumerId = 0 - expectrowcnt = parameterDict["rowsPerTbl"] * parameterDict["ctbNum"] / 2 - topicList = topicName1 + expectrowcnt = parameterDict["rowsPerTbl"] * parameterDict["ctbNum"] + parameterDict2["rowsPerTbl"] * parameterDict2["ctbNum"] + topicList = topicName1 + ',' + topicName2 ifcheckdata = 0 - ifManualCommit = 1 + ifManualCommit = 0 keyList = 'group.id:cgrp1,\ enable.auto.commit:false,\ auto.commit.interval.ms:6000,\ auto.offset.reset:earliest' self.insertConsumerInfo(consumerId, expectrowcnt,topicList,keyList,ifcheckdata,ifManualCommit) + #consumerId = 1 + #self.insertConsumerInfo(consumerId, expectrowcnt,topicList,keyList,ifcheckdata,ifManualCommit) + event.wait() tdLog.info("start consume processor") @@ -286,7 +228,8 @@ class TDTestCase: self.startTmqSimProcess(buildPath,cfgPath,pollDelay,parameterDict["dbName"],showMsg, showRow) # wait for data ready - prepareEnvThread.join() + prepareEnvThread.join() + prepareEnvThread2.join() tdLog.info("insert process end, and start to check consume result") expectRows = 1 @@ -295,44 +238,25 @@ class TDTestCase: for i in range(expectRows): totalConsumeRows += resultList[i] - tdSql.query("select count(*) from %s.%s" %(parameterDict['dbName'], parameterDict['stbName'])) - countOfStb = tdSql.getData(0,0) - print ("====total rows of stb: %d"%countOfStb) - - tdLog.info("act consume rows: %d, expect consume rows: %d"%(totalConsumeRows, expectrowcnt)) - if totalConsumeRows < expectrowcnt: - tdLog.exit("tmq consume rows error!") - - tdLog.info("again start consume processer") - self.initConsumerTable() - expectrowcnt = parameterDict["rowsPerTbl"] * parameterDict["ctbNum"] - self.insertConsumerInfo(consumerId, expectrowcnt,topicList,keyList,ifcheckdata,ifManualCommit) - self.startTmqSimProcess(buildPath,cfgPath,pollDelay,parameterDict["dbName"],showMsg, showRow) - expectRows = 1 - resultList = self.selectConsumeResult(expectRows) - totalConsumeRows2 = 0 - for i in range(expectRows): - totalConsumeRows2 += resultList[i] - - tdLog.info("firstly act consume rows: %d"%(totalConsumeRows)) - tdLog.info("secondly act consume rows: %d, expect consume rows: %d"%(totalConsumeRows2, expectrowcnt)) - if totalConsumeRows + totalConsumeRows2 != expectrowcnt: + if totalConsumeRows != expectrowcnt: + tdLog.info("act consume rows: %d, expect consume rows: %d"%(totalConsumeRows, expectrowcnt)) tdLog.exit("tmq consume rows error!") tdSql.query("drop topic %s"%topicName1) + tdSql.query("drop topic %s"%topicName2) - tdLog.printNoPrefix("======== test case 9 end ...... ") + tdLog.printNoPrefix("======== test case 6 end ...... ") - def tmqCase10(self, cfgPath, buildPath): - tdLog.printNoPrefix("======== test case 10: Produce while one consume to subscribe one db, inclue 1 stb") + def tmqCase7(self, cfgPath, buildPath): + tdLog.printNoPrefix("======== test case 7: Produce while two consumers to subscribe tow topic, Each contains one db") tdLog.info("step 1: create database, stb, ctb and insert data") # create and start thread parameterDict = {'cfg': '', \ - 'dbName': 'db10', \ + 'dbName': 'db70', \ 'vgroups': 4, \ 'stbName': 'stb', \ 'ctbNum': 10, \ - 'rowsPerTbl': 10000, \ + 'rowsPerTbl': 5000, \ 'batchNum': 100, \ 'startTs': 1640966400000} # 2022-01-01 00:00:00.000 parameterDict['cfg'] = cfgPath @@ -343,135 +267,70 @@ class TDTestCase: prepareEnvThread = threading.Thread(target=self.prepareEnv, kwargs=parameterDict) prepareEnvThread.start() - - tdLog.info("create topics from db") - topicName1 = 'topic_db1' - - tdSql.execute("create topic %s as database %s" %(topicName1, parameterDict['dbName'])) - consumerId = 0 - expectrowcnt = parameterDict["rowsPerTbl"] * parameterDict["ctbNum"] - topicList = topicName1 - ifcheckdata = 0 - ifManualCommit = 1 - keyList = 'group.id:cgrp1,\ - enable.auto.commit:false,\ - auto.commit.interval.ms:6000,\ - auto.offset.reset:earliest' - self.insertConsumerInfo(consumerId, expectrowcnt,topicList,keyList,ifcheckdata,ifManualCommit) - - event.wait() - tdLog.info("start consume processor") - pollDelay = 100 - showMsg = 1 - showRow = 1 - self.startTmqSimProcess(buildPath,cfgPath,pollDelay,parameterDict["dbName"],showMsg, showRow) - - time.sleep(2) - tdLog.info("pkill consume processor") - if (platform.system().lower() == 'windows'): - os.system("TASKKILL /F /IM tmq_sim.exe") - else: - os.system('pkill tmq_sim') - expectRows = 0 - resultList = self.selectConsumeResult(expectRows) - - # wait for data ready - prepareEnvThread.join() - tdLog.info("insert process end, and start to check consume result") - - tdLog.info("again start consume processer") - self.startTmqSimProcess(buildPath,cfgPath,pollDelay,parameterDict["dbName"],showMsg, showRow) - - expectRows = 1 - resultList = self.selectConsumeResult(expectRows) - totalConsumeRows = 0 - for i in range(expectRows): - totalConsumeRows += resultList[i] - - if totalConsumeRows != expectrowcnt: - tdLog.info("act consume rows: %d, expect consume rows: %d"%(totalConsumeRows, expectrowcnt)) - tdLog.exit("tmq consume rows error!") - - time.sleep(15) - tdSql.query("drop topic %s"%topicName1) - - tdLog.printNoPrefix("======== test case 10 end ...... ") - - def tmqCase11(self, cfgPath, buildPath): - tdLog.printNoPrefix("======== test case 11: Produce while one consume to subscribe one db, inclue 1 stb") - tdLog.info("step 1: create database, stb, ctb and insert data") - # create and start thread - parameterDict = {'cfg': '', \ - 'dbName': 'db11', \ + parameterDict2 = {'cfg': '', \ + 'dbName': 'db71', \ 'vgroups': 4, \ - 'stbName': 'stb', \ + 'stbName': 'stb2', \ 'ctbNum': 10, \ - 'rowsPerTbl': 10000, \ + 'rowsPerTbl': 5000, \ 'batchNum': 100, \ 'startTs': 1640966400000} # 2022-01-01 00:00:00.000 parameterDict['cfg'] = cfgPath - self.initConsumerTable() + tdSql.execute("create database if not exists %s vgroups %d" %(parameterDict2['dbName'], parameterDict2['vgroups'])) - tdSql.execute("create database if not exists %s vgroups %d" %(parameterDict['dbName'], parameterDict['vgroups'])) + prepareEnvThread2 = threading.Thread(target=self.prepareEnv, kwargs=parameterDict2) + prepareEnvThread2.start() - prepareEnvThread = threading.Thread(target=self.prepareEnv, kwargs=parameterDict) - prepareEnvThread.start() - tdLog.info("create topics from db") - topicName1 = 'topic_db1' + topicName1 = 'topic_db60' + topicName2 = 'topic_db61' + + tdSql.execute("create topic %s as database %s" %(topicName1, parameterDict['dbName'])) + tdSql.execute("create topic %s as database %s" %(topicName2, parameterDict2['dbName'])) - tdSql.execute("create topic %s as database %s" %(topicName1, parameterDict['dbName'])) consumerId = 0 - expectrowcnt = parameterDict["rowsPerTbl"] * parameterDict["ctbNum"] - topicList = topicName1 + expectrowcnt = parameterDict["rowsPerTbl"] * parameterDict["ctbNum"] + parameterDict2["rowsPerTbl"] * parameterDict2["ctbNum"] + topicList = topicName1 + ',' + topicName2 ifcheckdata = 0 ifManualCommit = 1 keyList = 'group.id:cgrp1,\ - enable.auto.commit:true,\ - auto.commit.interval.ms:1000,\ + enable.auto.commit:false,\ + auto.commit.interval.ms:6000,\ auto.offset.reset:earliest' self.insertConsumerInfo(consumerId, expectrowcnt,topicList,keyList,ifcheckdata,ifManualCommit) + consumerId = 1 + self.insertConsumerInfo(consumerId, expectrowcnt,topicList,keyList,ifcheckdata,ifManualCommit) + event.wait() tdLog.info("start consume processor") - pollDelay = 20 + pollDelay = 100 showMsg = 1 showRow = 1 self.startTmqSimProcess(buildPath,cfgPath,pollDelay,parameterDict["dbName"],showMsg, showRow) - time.sleep(3) - tdLog.info("pkill consume processor") - if (platform.system().lower() == 'windows'): - os.system("TASKKILL /F /IM tmq_sim.exe") - else: - os.system('pkill tmq_sim') - expectRows = 0 - resultList = self.selectConsumeResult(expectRows) - # wait for data ready prepareEnvThread.join() + prepareEnvThread2.join() + tdLog.info("insert process end, and start to check consume result") - - tdLog.info("again start consume processer") - self.startTmqSimProcess(buildPath,cfgPath,pollDelay,parameterDict["dbName"],showMsg, showRow) - - expectRows = 1 + expectRows = 2 resultList = self.selectConsumeResult(expectRows) totalConsumeRows = 0 for i in range(expectRows): totalConsumeRows += resultList[i] - if totalConsumeRows >= expectrowcnt or totalConsumeRows <= 0: - tdLog.info("act consume rows: %d, expect consume rows between %d and 0"%(totalConsumeRows, expectrowcnt)) + if totalConsumeRows != expectrowcnt: + tdLog.info("act consume rows: %d, expect consume rows: %d"%(totalConsumeRows, expectrowcnt)) tdLog.exit("tmq consume rows error!") - time.sleep(15) tdSql.query("drop topic %s"%topicName1) + tdSql.query("drop topic %s"%topicName2) - tdLog.printNoPrefix("======== test case 11 end ...... ") + tdLog.printNoPrefix("======== test case 7 end ...... ") def run(self): tdSql.prepare() @@ -484,10 +343,9 @@ class TDTestCase: cfgPath = buildPath + "/../sim/psim/cfg" tdLog.info("cfgPath: %s" % cfgPath) - self.tmqCase8(cfgPath, buildPath) - self.tmqCase9(cfgPath, buildPath) - self.tmqCase10(cfgPath, buildPath) - self.tmqCase11(cfgPath, buildPath) + self.tmqCase6(cfgPath, buildPath) + self.tmqCase7(cfgPath, buildPath) + def stop(self): tdSql.close() diff --git a/tests/system-test/7-tmq/subscribeDb2.py b/tests/system-test/7-tmq/subscribeDb2.py new file mode 100644 index 0000000000000000000000000000000000000000..af31e802b31c6c682a1b120ba01b1e25cf6750f9 --- /dev/null +++ b/tests/system-test/7-tmq/subscribeDb2.py @@ -0,0 +1,347 @@ + +import taos +import sys +import time +import socket +import os +import threading + +from util.log import * +from util.sql import * +from util.cases import * +from util.dnodes import * + +class TDTestCase: + hostname = socket.gethostname() + #rpcDebugFlagVal = '143' + #clientCfgDict = {'serverPort': '', 'firstEp': '', 'secondEp':'', 'rpcDebugFlag':'135', 'fqdn':''} + #clientCfgDict["rpcDebugFlag"] = rpcDebugFlagVal + #updatecfgDict = {'clientCfg': {}, 'serverPort': '', 'firstEp': '', 'secondEp':'', 'rpcDebugFlag':'135', 'fqdn':''} + #updatecfgDict["rpcDebugFlag"] = rpcDebugFlagVal + #print ("===================: ", updatecfgDict) + + def init(self, conn, logSql): + tdLog.debug(f"start to excute {__file__}") + tdSql.init(conn.cursor()) + #tdSql.init(conn.cursor(), logSql) # output sql.txt file + + def getBuildPath(self): + selfPath = os.path.dirname(os.path.realpath(__file__)) + + if ("community" in selfPath): + projPath = selfPath[:selfPath.find("community")] + else: + projPath = selfPath[:selfPath.find("tests")] + + for root, dirs, files in os.walk(projPath): + if ("taosd" in files or "taosd.exe" in files): + rootRealPath = os.path.dirname(os.path.realpath(root)) + if ("packaging" not in rootRealPath): + buildPath = root[:len(root) - len("/build/bin")] + break + return buildPath + + def newcur(self,cfg,host,port): + user = "root" + password = "taosdata" + con=taos.connect(host=host, user=user, password=password, config=cfg ,port=port) + cur=con.cursor() + print(cur) + return cur + + def initConsumerTable(self,cdbName='cdb'): + tdLog.info("create consume database, and consume info table, and consume result table") + tdSql.query("create database if not exists %s vgroups 1"%(cdbName)) + tdSql.query("drop table if exists %s.consumeinfo "%(cdbName)) + tdSql.query("drop table if exists %s.consumeresult "%(cdbName)) + + tdSql.query("create table %s.consumeinfo (ts timestamp, consumerid int, topiclist binary(1024), keylist binary(1024), expectmsgcnt bigint, ifcheckdata int, ifmanualcommit int)"%cdbName) + tdSql.query("create table %s.consumeresult (ts timestamp, consumerid int, consummsgcnt bigint, consumrowcnt bigint, checkresult int)"%cdbName) + + def insertConsumerInfo(self,consumerId, expectrowcnt,topicList,keyList,ifcheckdata,ifmanualcommit,cdbName='cdb'): + sql = "insert into %s.consumeinfo values "%cdbName + sql += "(now, %d, '%s', '%s', %d, %d, %d)"%(consumerId, topicList, keyList, expectrowcnt, ifcheckdata, ifmanualcommit) + tdLog.info("consume info sql: %s"%sql) + tdSql.query(sql) + + def selectConsumeResult(self,expectRows,cdbName='cdb'): + resultList=[] + while 1: + tdSql.query("select * from %s.consumeresult"%cdbName) + #tdLog.info("row: %d, %l64d, %l64d"%(tdSql.getData(0, 1),tdSql.getData(0, 2),tdSql.getData(0, 3)) + if tdSql.getRows() == expectRows: + break + else: + time.sleep(5) + + for i in range(expectRows): + tdLog.info ("ts: %s, consume id: %d, consume msgs: %d, consume rows: %d"%(tdSql.getData(i , 0), tdSql.getData(i , 1), tdSql.getData(i , 2), tdSql.getData(i , 3))) + resultList.append(tdSql.getData(i , 3)) + + return resultList + + def startTmqSimProcess(self,buildPath,cfgPath,pollDelay,dbName,showMsg=1,showRow=1,cdbName='cdb',valgrind=0): + if valgrind == 1: + logFile = cfgPath + '/../log/valgrind-tmq.log' + shellCmd = 'nohup valgrind --log-file=' + logFile + shellCmd += '--tool=memcheck --leak-check=full --show-reachable=no --track-origins=yes --show-leak-kinds=all --num-callers=20 -v --workaround-gcc296-bugs=yes ' + + if (platform.system().lower() == 'windows'): + shellCmd = 'mintty -h never -w hide ' + buildPath + '\\build\\bin\\tmq_sim.exe -c ' + cfgPath + shellCmd += " -y %d -d %s -g %d -r %d -w %s "%(pollDelay, dbName, showMsg, showRow, cdbName) + shellCmd += "> nul 2>&1 &" + else: + shellCmd = 'nohup ' + buildPath + '/build/bin/tmq_sim -c ' + cfgPath + shellCmd += " -y %d -d %s -g %d -r %d -w %s "%(pollDelay, dbName, showMsg, showRow, cdbName) + shellCmd += "> /dev/null 2>&1 &" + tdLog.info(shellCmd) + os.system(shellCmd) + + def create_tables(self,tsql, dbName,vgroups,stbName,ctbNum,rowsPerTbl): + tsql.execute("create database if not exists %s vgroups %d"%(dbName, vgroups)) + tsql.execute("use %s" %dbName) + tsql.execute("create table if not exists %s (ts timestamp, c1 bigint, c2 binary(16)) tags(t1 int)"%stbName) + pre_create = "create table" + sql = pre_create + #tdLog.debug("doing create one stable %s and %d child table in %s ..." %(stbname, count ,dbname)) + for i in range(ctbNum): + sql += " %s_%d using %s tags(%d)"%(stbName,i,stbName,i+1) + if (i > 0) and (i%100 == 0): + tsql.execute(sql) + sql = pre_create + if sql != pre_create: + tsql.execute(sql) + + event.set() + tdLog.debug("complete to create database[%s], stable[%s] and %d child tables" %(dbName, stbName, ctbNum)) + return + + def insert_data(self,tsql,dbName,stbName,ctbNum,rowsPerTbl,batchNum,startTs): + tdLog.debug("start to insert data ............") + tsql.execute("use %s" %dbName) + pre_insert = "insert into " + sql = pre_insert + + t = time.time() + startTs = int(round(t * 1000)) + #tdLog.debug("doing insert data into stable:%s rows:%d ..."%(stbName, allRows)) + for i in range(ctbNum): + sql += " %s_%d values "%(stbName,i) + for j in range(rowsPerTbl): + sql += "(%d, %d, 'tmqrow_%d') "%(startTs + j, j, j) + if (j > 0) and ((j%batchNum == 0) or (j == rowsPerTbl - 1)): + tsql.execute(sql) + if j < rowsPerTbl - 1: + sql = "insert into %s_%d values " %(stbName,i) + else: + sql = "insert into " + #end sql + if sql != pre_insert: + #print("insert sql:%s"%sql) + tsql.execute(sql) + tdLog.debug("insert data ............ [OK]") + return + + def prepareEnv(self, **parameterDict): + print ("input parameters:") + print (parameterDict) + # create new connector for my thread + tsql=self.newcur(parameterDict['cfg'], 'localhost', 6030) + self.create_tables(tsql,\ + parameterDict["dbName"],\ + parameterDict["vgroups"],\ + parameterDict["stbName"],\ + parameterDict["ctbNum"],\ + parameterDict["rowsPerTbl"]) + + self.insert_data(tsql,\ + parameterDict["dbName"],\ + parameterDict["stbName"],\ + parameterDict["ctbNum"],\ + parameterDict["rowsPerTbl"],\ + parameterDict["batchNum"],\ + parameterDict["startTs"]) + return + + def tmqCase8(self, cfgPath, buildPath): + tdLog.printNoPrefix("======== test case 8: Produce while one consume to subscribe one db, inclue 1 stb") + tdLog.info("step 1: create database, stb, ctb and insert data") + # create and start thread + parameterDict = {'cfg': '', \ + 'dbName': 'db8', \ + 'vgroups': 4, \ + 'stbName': 'stb', \ + 'ctbNum': 10, \ + 'rowsPerTbl': 10000, \ + 'batchNum': 100, \ + 'startTs': 1640966400000} # 2022-01-01 00:00:00.000 + parameterDict['cfg'] = cfgPath + + self.initConsumerTable() + + tdSql.execute("create database if not exists %s vgroups %d" %(parameterDict['dbName'], parameterDict['vgroups'])) + + prepareEnvThread = threading.Thread(target=self.prepareEnv, kwargs=parameterDict) + prepareEnvThread.start() + + tdLog.info("create topics from db") + topicName1 = 'topic_db1' + + tdSql.execute("create topic %s as database %s" %(topicName1, parameterDict['dbName'])) + consumerId = 0 + expectrowcnt = parameterDict["rowsPerTbl"] * parameterDict["ctbNum"] / 2 + topicList = topicName1 + ifcheckdata = 0 + ifManualCommit = 0 + keyList = 'group.id:cgrp1,\ + enable.auto.commit:false,\ + auto.commit.interval.ms:6000,\ + auto.offset.reset:earliest' + self.insertConsumerInfo(consumerId, expectrowcnt,topicList,keyList,ifcheckdata,ifManualCommit) + + event.wait() + + tdLog.info("start consume processor") + pollDelay = 100 + showMsg = 1 + showRow = 1 + self.startTmqSimProcess(buildPath,cfgPath,pollDelay,parameterDict["dbName"],showMsg, showRow) + + # wait for data ready + prepareEnvThread.join() + + tdLog.info("insert process end, and start to check consume result") + expectRows = 1 + resultList = self.selectConsumeResult(expectRows) + totalConsumeRows = 0 + for i in range(expectRows): + totalConsumeRows += resultList[i] + + if totalConsumeRows != expectrowcnt: + tdLog.info("act consume rows: %d, expect consume rows: %d"%(totalConsumeRows, expectrowcnt)) + tdLog.exit("tmq consume rows error!") + + + tdLog.info("again start consume processer") + self.initConsumerTable() + expectrowcnt = parameterDict["rowsPerTbl"] * parameterDict["ctbNum"] + self.insertConsumerInfo(consumerId, expectrowcnt,topicList,keyList,ifcheckdata,ifManualCommit) + self.startTmqSimProcess(buildPath,cfgPath,pollDelay,parameterDict["dbName"],showMsg, showRow) + expectRows = 1 + resultList = self.selectConsumeResult(expectRows) + totalConsumeRows = 0 + for i in range(expectRows): + totalConsumeRows += resultList[i] + + if totalConsumeRows != expectrowcnt: + tdLog.info("act consume rows: %d, expect consume rows: %d"%(totalConsumeRows, expectrowcnt)) + tdLog.exit("tmq consume rows error!") + + tdSql.query("drop topic %s"%topicName1) + + tdLog.printNoPrefix("======== test case 8 end ...... ") + + def tmqCase9(self, cfgPath, buildPath): + tdLog.printNoPrefix("======== test case 9: Produce while one consume to subscribe one db, inclue 1 stb") + tdLog.info("step 1: create database, stb, ctb and insert data") + # create and start thread + parameterDict = {'cfg': '', \ + 'dbName': 'db9', \ + 'vgroups': 4, \ + 'stbName': 'stb', \ + 'ctbNum': 10, \ + 'rowsPerTbl': 10000, \ + 'batchNum': 100, \ + 'startTs': 1640966400000} # 2022-01-01 00:00:00.000 + parameterDict['cfg'] = cfgPath + + self.initConsumerTable() + + tdSql.execute("create database if not exists %s vgroups %d" %(parameterDict['dbName'], parameterDict['vgroups'])) + + prepareEnvThread = threading.Thread(target=self.prepareEnv, kwargs=parameterDict) + prepareEnvThread.start() + + tdLog.info("create topics from db") + topicName1 = 'topic_db1' + + tdSql.execute("create topic %s as database %s" %(topicName1, parameterDict['dbName'])) + consumerId = 0 + expectrowcnt = parameterDict["rowsPerTbl"] * parameterDict["ctbNum"] / 2 + topicList = topicName1 + ifcheckdata = 0 + ifManualCommit = 1 + keyList = 'group.id:cgrp1,\ + enable.auto.commit:false,\ + auto.commit.interval.ms:6000,\ + auto.offset.reset:earliest' + self.insertConsumerInfo(consumerId, expectrowcnt,topicList,keyList,ifcheckdata,ifManualCommit) + + event.wait() + + tdLog.info("start consume processor") + pollDelay = 100 + showMsg = 1 + showRow = 1 + self.startTmqSimProcess(buildPath,cfgPath,pollDelay,parameterDict["dbName"],showMsg, showRow) + + # wait for data ready + prepareEnvThread.join() + + tdLog.info("insert process end, and start to check consume result") + expectRows = 1 + resultList = self.selectConsumeResult(expectRows) + totalConsumeRows = 0 + for i in range(expectRows): + totalConsumeRows += resultList[i] + + tdSql.query("select count(*) from %s.%s" %(parameterDict['dbName'], parameterDict['stbName'])) + countOfStb = tdSql.getData(0,0) + print ("====total rows of stb: %d"%countOfStb) + + tdLog.info("act consume rows: %d, expect consume rows: %d"%(totalConsumeRows, expectrowcnt)) + if totalConsumeRows < expectrowcnt: + tdLog.exit("tmq consume rows error!") + + tdLog.info("again start consume processer") + self.initConsumerTable() + expectrowcnt = parameterDict["rowsPerTbl"] * parameterDict["ctbNum"] + self.insertConsumerInfo(consumerId, expectrowcnt,topicList,keyList,ifcheckdata,ifManualCommit) + self.startTmqSimProcess(buildPath,cfgPath,pollDelay,parameterDict["dbName"],showMsg, showRow) + expectRows = 1 + resultList = self.selectConsumeResult(expectRows) + totalConsumeRows2 = 0 + for i in range(expectRows): + totalConsumeRows2 += resultList[i] + + tdLog.info("firstly act consume rows: %d"%(totalConsumeRows)) + tdLog.info("secondly act consume rows: %d, expect consume rows: %d"%(totalConsumeRows2, expectrowcnt)) + if totalConsumeRows + totalConsumeRows2 != expectrowcnt: + tdLog.exit("tmq consume rows error!") + + tdSql.query("drop topic %s"%topicName1) + + tdLog.printNoPrefix("======== test case 9 end ...... ") + + def run(self): + tdSql.prepare() + + buildPath = self.getBuildPath() + if (buildPath == ""): + tdLog.exit("taosd not found!") + else: + tdLog.info("taosd found in %s" % buildPath) + cfgPath = buildPath + "/../sim/psim/cfg" + tdLog.info("cfgPath: %s" % cfgPath) + + self.tmqCase8(cfgPath, buildPath) + self.tmqCase9(cfgPath, buildPath) + + def stop(self): + tdSql.close() + tdLog.success(f"{__file__} successfully executed") + +event = threading.Event() + +tdCases.addLinux(__file__, TDTestCase()) +tdCases.addWindows(__file__, TDTestCase()) diff --git a/tests/system-test/7-tmq/subscribeDb3.py b/tests/system-test/7-tmq/subscribeDb3.py new file mode 100644 index 0000000000000000000000000000000000000000..6973f4c51fdc4db250898b15898f23782e613090 --- /dev/null +++ b/tests/system-test/7-tmq/subscribeDb3.py @@ -0,0 +1,337 @@ + +import taos +import sys +import time +import socket +import os +import threading + +from util.log import * +from util.sql import * +from util.cases import * +from util.dnodes import * + +class TDTestCase: + hostname = socket.gethostname() + #rpcDebugFlagVal = '143' + #clientCfgDict = {'serverPort': '', 'firstEp': '', 'secondEp':'', 'rpcDebugFlag':'135', 'fqdn':''} + #clientCfgDict["rpcDebugFlag"] = rpcDebugFlagVal + #updatecfgDict = {'clientCfg': {}, 'serverPort': '', 'firstEp': '', 'secondEp':'', 'rpcDebugFlag':'135', 'fqdn':''} + #updatecfgDict["rpcDebugFlag"] = rpcDebugFlagVal + #print ("===================: ", updatecfgDict) + + def init(self, conn, logSql): + tdLog.debug(f"start to excute {__file__}") + tdSql.init(conn.cursor()) + #tdSql.init(conn.cursor(), logSql) # output sql.txt file + + def getBuildPath(self): + selfPath = os.path.dirname(os.path.realpath(__file__)) + + if ("community" in selfPath): + projPath = selfPath[:selfPath.find("community")] + else: + projPath = selfPath[:selfPath.find("tests")] + + for root, dirs, files in os.walk(projPath): + if ("taosd" in files or "taosd.exe" in files): + rootRealPath = os.path.dirname(os.path.realpath(root)) + if ("packaging" not in rootRealPath): + buildPath = root[:len(root) - len("/build/bin")] + break + return buildPath + + def newcur(self,cfg,host,port): + user = "root" + password = "taosdata" + con=taos.connect(host=host, user=user, password=password, config=cfg ,port=port) + cur=con.cursor() + print(cur) + return cur + + def initConsumerTable(self,cdbName='cdb'): + tdLog.info("create consume database, and consume info table, and consume result table") + tdSql.query("create database if not exists %s vgroups 1"%(cdbName)) + tdSql.query("drop table if exists %s.consumeinfo "%(cdbName)) + tdSql.query("drop table if exists %s.consumeresult "%(cdbName)) + + tdSql.query("create table %s.consumeinfo (ts timestamp, consumerid int, topiclist binary(1024), keylist binary(1024), expectmsgcnt bigint, ifcheckdata int, ifmanualcommit int)"%cdbName) + tdSql.query("create table %s.consumeresult (ts timestamp, consumerid int, consummsgcnt bigint, consumrowcnt bigint, checkresult int)"%cdbName) + + def insertConsumerInfo(self,consumerId, expectrowcnt,topicList,keyList,ifcheckdata,ifmanualcommit,cdbName='cdb'): + sql = "insert into %s.consumeinfo values "%cdbName + sql += "(now, %d, '%s', '%s', %d, %d, %d)"%(consumerId, topicList, keyList, expectrowcnt, ifcheckdata, ifmanualcommit) + tdLog.info("consume info sql: %s"%sql) + tdSql.query(sql) + + def selectConsumeResult(self,expectRows,cdbName='cdb'): + resultList=[] + while 1: + tdSql.query("select * from %s.consumeresult"%cdbName) + #tdLog.info("row: %d, %l64d, %l64d"%(tdSql.getData(0, 1),tdSql.getData(0, 2),tdSql.getData(0, 3)) + if tdSql.getRows() == expectRows: + break + else: + time.sleep(5) + + for i in range(expectRows): + tdLog.info ("ts: %s, consume id: %d, consume msgs: %d, consume rows: %d"%(tdSql.getData(i , 0), tdSql.getData(i , 1), tdSql.getData(i , 2), tdSql.getData(i , 3))) + resultList.append(tdSql.getData(i , 3)) + + return resultList + + def startTmqSimProcess(self,buildPath,cfgPath,pollDelay,dbName,showMsg=1,showRow=1,cdbName='cdb',valgrind=0): + if valgrind == 1: + logFile = cfgPath + '/../log/valgrind-tmq.log' + shellCmd = 'nohup valgrind --log-file=' + logFile + shellCmd += '--tool=memcheck --leak-check=full --show-reachable=no --track-origins=yes --show-leak-kinds=all --num-callers=20 -v --workaround-gcc296-bugs=yes ' + + if (platform.system().lower() == 'windows'): + shellCmd = 'mintty -h never -w hide ' + buildPath + '\\build\\bin\\tmq_sim.exe -c ' + cfgPath + shellCmd += " -y %d -d %s -g %d -r %d -w %s "%(pollDelay, dbName, showMsg, showRow, cdbName) + shellCmd += "> nul 2>&1 &" + else: + shellCmd = 'nohup ' + buildPath + '/build/bin/tmq_sim -c ' + cfgPath + shellCmd += " -y %d -d %s -g %d -r %d -w %s "%(pollDelay, dbName, showMsg, showRow, cdbName) + shellCmd += "> /dev/null 2>&1 &" + tdLog.info(shellCmd) + os.system(shellCmd) + + def create_tables(self,tsql, dbName,vgroups,stbName,ctbNum,rowsPerTbl): + tsql.execute("create database if not exists %s vgroups %d"%(dbName, vgroups)) + tsql.execute("use %s" %dbName) + tsql.execute("create table if not exists %s (ts timestamp, c1 bigint, c2 binary(16)) tags(t1 int)"%stbName) + pre_create = "create table" + sql = pre_create + #tdLog.debug("doing create one stable %s and %d child table in %s ..." %(stbname, count ,dbname)) + for i in range(ctbNum): + sql += " %s_%d using %s tags(%d)"%(stbName,i,stbName,i+1) + if (i > 0) and (i%100 == 0): + tsql.execute(sql) + sql = pre_create + if sql != pre_create: + tsql.execute(sql) + + event.set() + tdLog.debug("complete to create database[%s], stable[%s] and %d child tables" %(dbName, stbName, ctbNum)) + return + + def insert_data(self,tsql,dbName,stbName,ctbNum,rowsPerTbl,batchNum,startTs): + tdLog.debug("start to insert data ............") + tsql.execute("use %s" %dbName) + pre_insert = "insert into " + sql = pre_insert + + t = time.time() + startTs = int(round(t * 1000)) + #tdLog.debug("doing insert data into stable:%s rows:%d ..."%(stbName, allRows)) + for i in range(ctbNum): + sql += " %s_%d values "%(stbName,i) + for j in range(rowsPerTbl): + sql += "(%d, %d, 'tmqrow_%d') "%(startTs + j, j, j) + if (j > 0) and ((j%batchNum == 0) or (j == rowsPerTbl - 1)): + tsql.execute(sql) + if j < rowsPerTbl - 1: + sql = "insert into %s_%d values " %(stbName,i) + else: + sql = "insert into " + #end sql + if sql != pre_insert: + #print("insert sql:%s"%sql) + tsql.execute(sql) + tdLog.debug("insert data ............ [OK]") + return + + def prepareEnv(self, **parameterDict): + print ("input parameters:") + print (parameterDict) + # create new connector for my thread + tsql=self.newcur(parameterDict['cfg'], 'localhost', 6030) + self.create_tables(tsql,\ + parameterDict["dbName"],\ + parameterDict["vgroups"],\ + parameterDict["stbName"],\ + parameterDict["ctbNum"],\ + parameterDict["rowsPerTbl"]) + + self.insert_data(tsql,\ + parameterDict["dbName"],\ + parameterDict["stbName"],\ + parameterDict["ctbNum"],\ + parameterDict["rowsPerTbl"],\ + parameterDict["batchNum"],\ + parameterDict["startTs"]) + return + + def tmqCase10(self, cfgPath, buildPath): + tdLog.printNoPrefix("======== test case 10: Produce while one consume to subscribe one db, inclue 1 stb") + tdLog.info("step 1: create database, stb, ctb and insert data") + # create and start thread + parameterDict = {'cfg': '', \ + 'dbName': 'db10', \ + 'vgroups': 4, \ + 'stbName': 'stb', \ + 'ctbNum': 10, \ + 'rowsPerTbl': 10000, \ + 'batchNum': 100, \ + 'startTs': 1640966400000} # 2022-01-01 00:00:00.000 + parameterDict['cfg'] = cfgPath + + self.initConsumerTable() + + tdSql.execute("create database if not exists %s vgroups %d" %(parameterDict['dbName'], parameterDict['vgroups'])) + + prepareEnvThread = threading.Thread(target=self.prepareEnv, kwargs=parameterDict) + prepareEnvThread.start() + + tdLog.info("create topics from db") + topicName1 = 'topic_db1' + + tdSql.execute("create topic %s as database %s" %(topicName1, parameterDict['dbName'])) + consumerId = 0 + expectrowcnt = parameterDict["rowsPerTbl"] * parameterDict["ctbNum"] + topicList = topicName1 + ifcheckdata = 0 + ifManualCommit = 1 + keyList = 'group.id:cgrp1,\ + enable.auto.commit:false,\ + auto.commit.interval.ms:6000,\ + auto.offset.reset:earliest' + self.insertConsumerInfo(consumerId, expectrowcnt,topicList,keyList,ifcheckdata,ifManualCommit) + + event.wait() + + tdLog.info("start consume processor") + pollDelay = 100 + showMsg = 1 + showRow = 1 + self.startTmqSimProcess(buildPath,cfgPath,pollDelay,parameterDict["dbName"],showMsg, showRow) + + time.sleep(2) + tdLog.info("pkill consume processor") + if (platform.system().lower() == 'windows'): + os.system("TASKKILL /F /IM tmq_sim.exe") + else: + os.system('pkill tmq_sim') + expectRows = 0 + resultList = self.selectConsumeResult(expectRows) + + # wait for data ready + prepareEnvThread.join() + tdLog.info("insert process end, and start to check consume result") + + tdLog.info("again start consume processer") + self.startTmqSimProcess(buildPath,cfgPath,pollDelay,parameterDict["dbName"],showMsg, showRow) + + expectRows = 1 + resultList = self.selectConsumeResult(expectRows) + totalConsumeRows = 0 + for i in range(expectRows): + totalConsumeRows += resultList[i] + + if totalConsumeRows != expectrowcnt: + tdLog.info("act consume rows: %d, expect consume rows: %d"%(totalConsumeRows, expectrowcnt)) + tdLog.exit("tmq consume rows error!") + + time.sleep(15) + tdSql.query("drop topic %s"%topicName1) + + tdLog.printNoPrefix("======== test case 10 end ...... ") + + def tmqCase11(self, cfgPath, buildPath): + tdLog.printNoPrefix("======== test case 11: Produce while one consume to subscribe one db, inclue 1 stb") + tdLog.info("step 1: create database, stb, ctb and insert data") + # create and start thread + parameterDict = {'cfg': '', \ + 'dbName': 'db11', \ + 'vgroups': 4, \ + 'stbName': 'stb', \ + 'ctbNum': 10, \ + 'rowsPerTbl': 10000, \ + 'batchNum': 100, \ + 'startTs': 1640966400000} # 2022-01-01 00:00:00.000 + parameterDict['cfg'] = cfgPath + + self.initConsumerTable() + + tdSql.execute("create database if not exists %s vgroups %d" %(parameterDict['dbName'], parameterDict['vgroups'])) + + prepareEnvThread = threading.Thread(target=self.prepareEnv, kwargs=parameterDict) + prepareEnvThread.start() + + tdLog.info("create topics from db") + topicName1 = 'topic_db1' + + tdSql.execute("create topic %s as database %s" %(topicName1, parameterDict['dbName'])) + consumerId = 0 + expectrowcnt = parameterDict["rowsPerTbl"] * parameterDict["ctbNum"] + topicList = topicName1 + ifcheckdata = 0 + ifManualCommit = 1 + keyList = 'group.id:cgrp1,\ + enable.auto.commit:true,\ + auto.commit.interval.ms:1000,\ + auto.offset.reset:earliest' + self.insertConsumerInfo(consumerId, expectrowcnt,topicList,keyList,ifcheckdata,ifManualCommit) + + event.wait() + + tdLog.info("start consume processor") + pollDelay = 20 + showMsg = 1 + showRow = 1 + self.startTmqSimProcess(buildPath,cfgPath,pollDelay,parameterDict["dbName"],showMsg, showRow) + + time.sleep(6) + tdLog.info("pkill consume processor") + if (platform.system().lower() == 'windows'): + os.system("TASKKILL /F /IM tmq_sim.exe") + else: + os.system('pkill tmq_sim') + expectRows = 0 + resultList = self.selectConsumeResult(expectRows) + + # wait for data ready + prepareEnvThread.join() + tdLog.info("insert process end, and start to check consume result") + + tdLog.info("again start consume processer") + self.startTmqSimProcess(buildPath,cfgPath,pollDelay,parameterDict["dbName"],showMsg, showRow) + + expectRows = 1 + resultList = self.selectConsumeResult(expectRows) + totalConsumeRows = 0 + for i in range(expectRows): + totalConsumeRows += resultList[i] + + if totalConsumeRows >= expectrowcnt or totalConsumeRows <= 0: + tdLog.info("act consume rows: %d, expect consume rows between %d and 0"%(totalConsumeRows, expectrowcnt)) + tdLog.exit("tmq consume rows error!") + + time.sleep(15) + tdSql.query("drop topic %s"%topicName1) + + tdLog.printNoPrefix("======== test case 11 end ...... ") + + def run(self): + tdSql.prepare() + + buildPath = self.getBuildPath() + if (buildPath == ""): + tdLog.exit("taosd not found!") + else: + tdLog.info("taosd found in %s" % buildPath) + cfgPath = buildPath + "/../sim/psim/cfg" + tdLog.info("cfgPath: %s" % cfgPath) + + self.tmqCase10(cfgPath, buildPath) + self.tmqCase11(cfgPath, buildPath) + + def stop(self): + tdSql.close() + tdLog.success(f"{__file__} successfully executed") + +event = threading.Event() + +tdCases.addLinux(__file__, TDTestCase()) +tdCases.addWindows(__file__, TDTestCase()) diff --git a/tests/system-test/fulltest.sh b/tests/system-test/fulltest.sh index b9f7e0f2eb39f1c5837ed103ad335de2ed417084..b5004d41ce2f2a663eda05af3673ab73f60d6d65 100755 --- a/tests/system-test/fulltest.sh +++ b/tests/system-test/fulltest.sh @@ -98,17 +98,25 @@ python3 ./test.py -f 2-query/stateduration.py python3 ./test.py -f 2-query/function_stateduration.py python3 ./test.py -f 2-query/statecount.py python3 ./test.py -f 2-query/tail.py +python3 ./test.py -f 2-query/distribute_agg_count.py +python3 ./test.py -f 2-query/distribute_agg_max.py +python3 ./test.py -f 2-query/distribute_agg_min.py +python3 ./test.py -f 2-query/distribute_agg_sum.py +python3 ./test.py -f 2-query/distribute_agg_spread.py +python3 ./test.py -f 2-query/distribute_agg_apercentile.py python3 ./test.py -f 6-cluster/5dnode1mnode.py python3 ./test.py -f 6-cluster/5dnode2mnode.py #python3 ./test.py -f 6-cluster/5dnode3mnodeStop.py -python3 ./test.py -f 6-cluster/5dnode3mnodeDrop.py +#python3 ./test.py -f 6-cluster/5dnode3mnodeDrop.py # BUG python3 ./test.py -f 6-cluster/5dnode3mnodeStopInsert.py python3 ./test.py -f 7-tmq/basic5.py python3 ./test.py -f 7-tmq/subscribeDb.py python3 ./test.py -f 7-tmq/subscribeDb0.py python3 ./test.py -f 7-tmq/subscribeDb1.py +python3 ./test.py -f 7-tmq/subscribeDb2.py +python3 ./test.py -f 7-tmq/subscribeDb3.py python3 ./test.py -f 7-tmq/subscribeStb.py python3 ./test.py -f 7-tmq/subscribeStb0.py python3 ./test.py -f 7-tmq/subscribeStb1.py diff --git a/tests/test/c/sdbDump.c b/tests/test/c/sdbDump.c index e5986cf4dddaa7191e7047e76b51305baefc55c1..612b870b7e0590713c373379b0c0053abeeeef02 100644 --- a/tests/test/c/sdbDump.c +++ b/tests/test/c/sdbDump.c @@ -283,7 +283,8 @@ void dumpTrans(SSdb *pSdb, SJson *json) { tjsonAddIntegerToObject(item, "conflict", pObj->conflict); tjsonAddIntegerToObject(item, "exec", pObj->exec); tjsonAddStringToObject(item, "createdTime", i642str(pObj->createdTime)); - tjsonAddStringToObject(item, "dbname", pObj->dbname); + tjsonAddStringToObject(item, "dbname1", pObj->dbname1); + tjsonAddStringToObject(item, "dbname2", pObj->dbname2); tjsonAddIntegerToObject(item, "commitLogNum", taosArrayGetSize(pObj->commitActions)); tjsonAddIntegerToObject(item, "redoActionNum", taosArrayGetSize(pObj->redoActions)); tjsonAddIntegerToObject(item, "undoActionNum", taosArrayGetSize(pObj->undoActions)); @@ -294,8 +295,9 @@ void dumpTrans(SSdb *pSdb, SJson *json) { void dumpHeader(SSdb *pSdb, SJson *json) { tjsonAddIntegerToObject(json, "sver", 1); - tjsonAddStringToObject(json, "curVer", i642str(pSdb->curVer)); - tjsonAddStringToObject(json, "curTerm", i642str(pSdb->curTerm)); + tjsonAddStringToObject(json, "applyIndex", i642str(pSdb->applyIndex)); + tjsonAddStringToObject(json, "applyTerm", i642str(pSdb->applyTerm)); + tjsonAddStringToObject(json, "applyConfig", i642str(pSdb->applyConfig)); SJson *maxIdsJson = tjsonCreateObject(); tjsonAddItemToObject(json, "maxIds", maxIdsJson); diff --git a/tests/test/c/tmqDemo.c b/tests/test/c/tmqDemo.c index 5a972e071a5e28708cfb561ef1015d72ba830219..02fd3c13962eb5474deee298dca67dd3d07a7f49 100644 --- a/tests/test/c/tmqDemo.c +++ b/tests/test/c/tmqDemo.c @@ -353,8 +353,8 @@ tmq_list_t* build_topic_list() { void sync_consume_loop(tmq_t* tmq, tmq_list_t* topics) { static const int MIN_COMMIT_COUNT = 1000; - int msg_count = 0; - tmq_resp_err_t err; + int msg_count = 0; + int32_t err; if ((err = tmq_subscribe(tmq, topics))) { fprintf(stderr, "%% Failed to start consuming topics: %s\n", tmq_err2str(err)); @@ -379,7 +379,7 @@ void sync_consume_loop(tmq_t* tmq, tmq_list_t* topics) { } void perf_loop(tmq_t* tmq, tmq_list_t* topics, int32_t totalMsgs, int64_t walLogSize) { - tmq_resp_err_t err; + int32_t err; if ((err = tmq_subscribe(tmq, topics))) { fprintf(stderr, "%% Failed to start consuming topics: %s\n", tmq_err2str(err)); diff --git a/tests/test/c/tmqSim.c b/tests/test/c/tmqSim.c index 8455bd9890533c6f3ee34c782f595db2b390209b..0f78a003d6acba40079a0d28c367f11b3966e2ae 100644 --- a/tests/test/c/tmqSim.c +++ b/tests/test/c/tmqSim.c @@ -62,7 +62,7 @@ typedef struct { tmq_t* tmq; tmq_list_t* topicList; - + int32_t numOfVgroups; int32_t rowsOfPerVgroups[MAX_VGROUP_CNT][2]; // [i][0]: vgroup id, [i][1]: rows of consume int64_t ts; @@ -74,7 +74,7 @@ typedef struct { char cdbName[32]; char dbName[32]; int32_t showMsgFlag; - int32_t showRowFlag; + int32_t showRowFlag; int32_t saveRowFlag; int32_t consumeDelay; // unit s int32_t numOfThread; @@ -108,26 +108,20 @@ static void printHelp() { } char* getCurrentTimeString(char* timeString) { - time_t tTime = taosGetTimestampSec(); + time_t tTime = taosGetTimestampSec(); struct tm tm = *taosLocalTime(&tTime, NULL); - sprintf(timeString, "%d-%02d-%02d %02d:%02d:%02d", - tm.tm_year + 1900, - tm.tm_mon + 1, - tm.tm_mday, - tm.tm_hour, - tm.tm_min, - tm.tm_sec); + sprintf(timeString, "%d-%02d-%02d %02d:%02d:%02d", tm.tm_year + 1900, tm.tm_mon + 1, tm.tm_mday, tm.tm_hour, + tm.tm_min, tm.tm_sec); return timeString; } - void initLogFile() { char filename[256]; - char tmpString[128]; + char tmpString[128]; - sprintf(filename,"%s/../log/tmqlog_%s.txt", configDir, getCurrentTimeString(tmpString)); - //sprintf(filename, "%s/../log/tmqlog.txt", configDir); + sprintf(filename, "%s/../log/tmqlog_%s.txt", configDir, getCurrentTimeString(tmpString)); + // sprintf(filename, "%s/../log/tmqlog.txt", configDir); #ifdef WINDOWS for (int i = 2; i < sizeof(filename); i++) { if (filename[i] == ':') filename[i] = '-'; @@ -249,17 +243,18 @@ void addRowsToVgroupId(SThreadInfo* pInfo, int32_t vgroupId, int32_t rows) { for (i = 0; i < pInfo->numOfVgroups; i++) { if (vgroupId == pInfo->rowsOfPerVgroups[i][0]) { pInfo->rowsOfPerVgroups[i][1] += rows; - return; - } + return; + } } pInfo->rowsOfPerVgroups[pInfo->numOfVgroups][0] = vgroupId; pInfo->rowsOfPerVgroups[pInfo->numOfVgroups][1] += rows; pInfo->numOfVgroups++; - + taosFprintfFile(g_fp, "consume id %d, add one new vogroup id: %d\n", pInfo->consumerId, vgroupId); if (pInfo->numOfVgroups > MAX_VGROUP_CNT) { - taosFprintfFile(g_fp, "====consume id %d, vgroup num %d over than 32. new vgroupId: %d\n", pInfo->consumerId, pInfo->numOfVgroups, vgroupId); + taosFprintfFile(g_fp, "====consume id %d, vgroup num %d over than 32. new vgroupId: %d\n", pInfo->consumerId, + pInfo->numOfVgroups, vgroupId); taosCloseFile(&g_fp); exit(-1); } @@ -277,7 +272,8 @@ int32_t saveConsumeContentToTbl(SThreadInfo* pInfo, char* buf) { TAOS* pConn = taos_connect(NULL, "root", "taosdata", NULL, 0); assert(pConn != NULL); - sprintf(sqlStr, "insert into %s.content_%d values (%"PRId64", \'%s\')", g_stConfInfo.cdbName, pInfo->consumerId, pInfo->ts++, buf); + sprintf(sqlStr, "insert into %s.content_%d values (%" PRId64 ", \'%s\')", g_stConfInfo.cdbName, pInfo->consumerId, + pInfo->ts++, buf); TAOS_RES* pRes = taos_query(pConn, sqlStr); if (taos_errno(pRes) != 0) { pError("error in insert consume result, reason:%s\n", taos_errstr(pRes)); @@ -295,12 +291,13 @@ int32_t saveConsumeContentToTbl(SThreadInfo* pInfo, char* buf) { static int32_t msg_process(TAOS_RES* msg, SThreadInfo* pInfo, int32_t msgIndex) { char buf[1024]; int32_t totalRows = 0; - + // printf("topic: %s\n", tmq_get_topic_name(msg)); int32_t vgroupId = tmq_get_vgroup_id(msg); - + taosFprintfFile(g_fp, "msg index:%" PRId64 ", consumerId: %d\n", msgIndex, pInfo->consumerId); - //taosFprintfFile(g_fp, "topic: %s, vgroupId: %d, tableName: %s\n", tmq_get_topic_name(msg), vgroupId, tmq_get_table_name(msg)); + // taosFprintfFile(g_fp, "topic: %s, vgroupId: %d, tableName: %s\n", tmq_get_topic_name(msg), vgroupId, + // tmq_get_table_name(msg)); taosFprintfFile(g_fp, "topic: %s, vgroupId: %d\n", tmq_get_topic_name(msg), vgroupId); while (1) { @@ -316,9 +313,9 @@ static int32_t msg_process(TAOS_RES* msg, SThreadInfo* pInfo, int32_t msgIndex) const char* tbName = tmq_get_table_name(msg); if (0 != g_stConfInfo.showRowFlag) { - taosFprintfFile(g_fp, "tbname:%s, rows[%d]: %s\n", (tbName != NULL ? tbName:"null table"), totalRows, buf); - if (0 != g_stConfInfo.saveRowFlag) { - saveConsumeContentToTbl(pInfo, buf); + taosFprintfFile(g_fp, "tbname:%s, rows[%d]: %s\n", (tbName != NULL ? tbName : "null table"), totalRows, buf); + if (0 != g_stConfInfo.saveRowFlag) { + saveConsumeContentToTbl(pInfo, buf); } } @@ -326,7 +323,7 @@ static int32_t msg_process(TAOS_RES* msg, SThreadInfo* pInfo, int32_t msgIndex) } addRowsToVgroupId(pInfo, vgroupId, totalRows); - + return totalRows; } @@ -342,8 +339,8 @@ int queryDB(TAOS* taos, char* command) { return 0; } -static void tmq_commit_cb_print(tmq_t* tmq, tmq_resp_err_t resp, tmq_topic_vgroup_list_t* offsets, void* param) { - pError("tmq_commit_cb_print() commit %d\n", resp); +static void tmq_commit_cb_print(tmq_t* tmq, int32_t code, void* param) { + pError("tmq_commit_cb_print() commit %d\n", code); } void build_consumer(SThreadInfo* pInfo) { @@ -401,16 +398,11 @@ int32_t saveConsumeResult(SThreadInfo* pInfo) { int64_t now = taosGetTimestampMs(); // schema: ts timestamp, consumerid int, consummsgcnt bigint, checkresult int - sprintf(sqlStr, "insert into %s.consumeresult values (%"PRId64", %d, %" PRId64 ", %" PRId64 ", %d)", - g_stConfInfo.cdbName, - now, - pInfo->consumerId, - pInfo->consumeMsgCnt, - pInfo->consumeRowCnt, - pInfo->checkresult); + sprintf(sqlStr, "insert into %s.consumeresult values (%" PRId64 ", %d, %" PRId64 ", %" PRId64 ", %d)", + g_stConfInfo.cdbName, now, pInfo->consumerId, pInfo->consumeMsgCnt, pInfo->consumeRowCnt, pInfo->checkresult); char tmpString[128]; - taosFprintfFile(g_fp, "%s, consume id %d result: %s\n", getCurrentTimeString(tmpString), pInfo->consumerId ,sqlStr); + taosFprintfFile(g_fp, "%s, consume id %d result: %s\n", getCurrentTimeString(tmpString), pInfo->consumerId, sqlStr); TAOS_RES* pRes = taos_query(pConn, sqlStr); if (taos_errno(pRes) != 0) { @@ -421,7 +413,7 @@ int32_t saveConsumeResult(SThreadInfo* pInfo) { taos_free_result(pRes); - #if 0 +#if 0 // vgroups for (i = 0; i < pInfo->numOfVgroups; i++) { // schema: ts timestamp, consumerid int, consummsgcnt bigint, checkresult int @@ -445,19 +437,20 @@ int32_t saveConsumeResult(SThreadInfo* pInfo) { taos_free_result(pRes); } - #endif +#endif return 0; } void loop_consume(SThreadInfo* pInfo) { - tmq_resp_err_t err; + int32_t code; int64_t totalMsgs = 0; int64_t totalRows = 0; char tmpString[128]; - taosFprintfFile(g_fp, "%s consumer id %d start to loop pull msg\n", getCurrentTimeString(tmpString), pInfo->consumerId); + taosFprintfFile(g_fp, "%s consumer id %d start to loop pull msg\n", getCurrentTimeString(tmpString), + pInfo->consumerId); pInfo->ts = taosGetTimestampMs(); @@ -473,7 +466,7 @@ void loop_consume(SThreadInfo* pInfo) { totalMsgs++; if (totalRows >= pInfo->expectMsgCnt) { - char tmpString[128]; + char tmpString[128]; taosFprintfFile(g_fp, "%s over than expect rows, so break consume\n", getCurrentTimeString(tmpString)); break; } @@ -503,8 +496,8 @@ void* consumeThreadFunc(void* param) { return NULL; } - tmq_resp_err_t err = tmq_subscribe(pInfo->tmq, pInfo->topicList); - if (err) { + int32_t err = tmq_subscribe(pInfo->tmq, pInfo->topicList); + if (err != 0) { pError("tmq_subscribe() fail, reason: %s\n", tmq_err2str(err)); exit(-1); } @@ -519,17 +512,19 @@ void* consumeThreadFunc(void* param) { pPrint("tmq_commit() manual commit when consume end.\n"); /*tmq_commit(pInfo->tmq, NULL, 0);*/ tmq_commit_sync(pInfo->tmq, NULL); + taosFprintfFile(g_fp, "tmq_commit() manual commit over.\n"); + pPrint("tmq_commit() manual commit over.\n"); } err = tmq_unsubscribe(pInfo->tmq); - if (err) { + if (err != 0) { pError("tmq_unsubscribe() fail, reason: %s\n", tmq_err2str(err)); /*pInfo->consumeMsgCnt = -1;*/ /*return NULL;*/ } err = tmq_consumer_close(pInfo->tmq); - if (err) { + if (err != 0) { pError("tmq_consumer_close() fail, reason: %s\n", tmq_err2str(err)); /*exit(-1);*/ } diff --git a/tests/tsim/src/simExe.c b/tests/tsim/src/simExe.c index dbc3c2c4602f1424d997a07fb28ac92e1ea1ff0e..f97a13d2c5867ef35ceda35e716461400544371c 100644 --- a/tests/tsim/src/simExe.c +++ b/tests/tsim/src/simExe.c @@ -458,11 +458,17 @@ bool simExecuteSystemContentCmd(SScript *script, char *option) { char buf[4096] = {0}; char buf1[4096 + 512] = {0}; char filename[400] = {0}; - sprintf(filename, "%s/%s.tmp", simScriptDir, script->fileName); + sprintf(filename, "%s" TD_DIRSEP "%s.tmp", simScriptDir, script->fileName); +#ifdef WINDOWS + sprintf(buf, "cd %s && ", simScriptDir); + simVisuallizeOption(script, option, buf + strlen(buf)); + sprintf(buf1, "%s > %s 2>nul", buf, filename); +#else sprintf(buf, "cd %s; ", simScriptDir); simVisuallizeOption(script, option, buf + strlen(buf)); sprintf(buf1, "%s > %s 2>/dev/null", buf, filename); +#endif sprintf(script->system_exit_code, "%d", system(buf1)); simStoreSystemContentResult(script, filename); diff --git a/tests/tsim/src/simParse.c b/tests/tsim/src/simParse.c index 638c4a1ccb57da6b8d4f29521f931783cff13ba7..5b6dda4dae008364ed11c352e6fd7c8adfa0fc76 100644 --- a/tests/tsim/src/simParse.c +++ b/tests/tsim/src/simParse.c @@ -206,7 +206,7 @@ SScript *simParseScript(char *fileName) { for (int32_t i = 0; i < cmdlen; ++i) { if (buffer[i] == '\r' || buffer[i] == '\n') { - buffer[i] = ' '; + buffer[i] = '\0'; } } diff --git a/tools/shell/src/shellEngine.c b/tools/shell/src/shellEngine.c index 8ed0e9ddcf43a490c905ae9db5ada09f202d3899..1f29237d38b0441507bef05a39586d8a3b80ca56 100644 --- a/tools/shell/src/shellEngine.c +++ b/tools/shell/src/shellEngine.c @@ -855,8 +855,7 @@ void shellGetGrantInfo() { if (code == TSDB_CODE_OPS_NOT_SUPPORT) { fprintf(stdout, "Server is Community Edition, %s\n\n", sinfo); } else { - fprintf(stderr, "Failed to check Server Edition, Reason:0x%04x:%s\n\n", taos_errno(shell.conn), - taos_errstr(shell.conn)); + fprintf(stderr, "Failed to check Server Edition, Reason:0x%04x:%s\n\n", code, taos_errstr(tres)); } return; } diff --git a/tools/taos-tools b/tools/taos-tools index 3d5aa76f8c718dcffa100b45e4cbf313d499c356..28a49b447f71c4f014ebbac858b7215b897d57fd 160000 --- a/tools/taos-tools +++ b/tools/taos-tools @@ -1 +1 @@ -Subproject commit 3d5aa76f8c718dcffa100b45e4cbf313d499c356 +Subproject commit 28a49b447f71c4f014ebbac858b7215b897d57fd diff --git a/tools/taosadapter b/tools/taosadapter new file mode 160000 index 0000000000000000000000000000000000000000..29926478edd87533a043f91c1a9ed0e27671e626 --- /dev/null +++ b/tools/taosadapter @@ -0,0 +1 @@ +Subproject commit 29926478edd87533a043f91c1a9ed0e27671e626