diff --git a/docs/en/07-develop/03-insert-data/05-high-volume.md b/docs/en/07-develop/03-insert-data/05-high-volume.md new file mode 100644 index 0000000000000000000000000000000000000000..fc4f071997caeecfda6e314bd7f60278d8cf672c --- /dev/null +++ b/docs/en/07-develop/03-insert-data/05-high-volume.md @@ -0,0 +1,444 @@ +--- +sidebar_label: High Performance Writing +title: High Performance Writing +--- + +import Tabs from "@theme/Tabs"; +import TabItem from "@theme/TabItem"; + +This chapter introduces how to write data into TDengine with high throughput. + +## How to achieve high performance data writing + +To achieve high performance writing, there are a few aspects to consider. In the following sections we will describe these important factors in achieving high performance writing. + +### Application Program + +From the perspective of application program, you need to consider: + +1. The data size of each single write, also known as batch size. Generally speaking, higher batch size generates better writing performance. However, once the batch size is over a specific value, you will not get any additional benefit anymore. When using SQL to write into TDengine, it's better to put as much as possible data in single SQL. The maximum SQL length supported by TDengine is 1,048,576 bytes, i.e. 1 MB. It can be configured by parameter `maxSQLLength` on client side, and the default value is 65,480. + +2. The number of concurrent connections. Normally more connections can get better result. However, once the number of connections exceeds the processing ability of the server side, the performance may downgrade. + +3. The distribution of data to be written across tables or sub-tables. Writing to single table in one batch is more efficient than writing to multiple tables in one batch. + +4. Data Writing Protocol. + - Prameter binding mode is more efficient than SQL because it doesn't have the cost of parsing SQL. + - Writing to known existing tables is more efficient than wirting to uncertain tables in automatic creating mode because the later needs to check whether the table exists or not before actually writing data into it + - Writing in SQL is more efficient than writing in schemaless mode because schemaless writing creats table automatically and may alter table schema + +Application programs need to take care of the above factors and try to take advantage of them. The application progam should write to single table in each write batch. The batch size needs to be tuned to a proper value on a specific system. The number of concurrent connections needs to be tuned to a proper value too to achieve the best writing throughput. + +### Data Source + +Application programs need to read data from data source then write into TDengine. If you meet one or more of below situations, you need to setup message queues between the threads for reading from data source and the threads for writing into TDengine. + +1. There are multiple data sources, the data generation speed of each data source is much slower than the speed of single writing thread. In this case, the purpose of message queues is to consolidate the data from multiple data sources together to increase the batch size of single write. +2. The speed of data generation from single data source is much higher than the speed of single writing thread. The purpose of message queue in this case is to provide buffer so that data is not lost and multiple writing threads can get data from the buffer. +3. The data for single table are from multiple data source. In this case the purpose of message queues is to combine the data for single table together to improve the write efficiency. + +If the data source is Kafka, then the appication program is a consumer of Kafka, you can benefit from some kafka features to achieve high performance writing: + +1. Put the data for a table in single partition of single topic so that it's easier to put the data for each table together and write in batch +2. Subscribe multiple topics to accumulate data together. +3. Add more consumers to gain more concurrency and throughput. +4. Incrase the size of single fetch to increase the size of write batch. + +### Tune TDengine + +TDengine is a distributed and high performance time series database, there are also some ways to tune TDengine to get better writing performance. + +1. Set proper number of `vgroups` according to available CPU cores. Normally, we recommend 2 \* number_of_cores as a starting point. If the verification result shows this is not enough to utilize CPU resources, you can use a higher value. +2. Set proper `minTablesPerVnode`, `tableIncStepPerVnode`, and `maxVgroupsPerDb` according to the number of tables so that tables are distributed even across vgroups. The purpose is to balance the workload among all vnodes so that system resources can be utilized better to get higher performance. + +For more performance tuning tips, please refer to [Performance Optimization](../../operation/optimize) and [Configuration Parameters](../../reference/config). + +## Sample Programs + +This section will introduce the sample programs to demonstrate how to write into TDengine with high performance. + +### Scenario + +Below are the scenario for the sample programs of high performance wrting. + +- Application program reads data from data source, the sample program simulates a data source by generating data +- The speed of single writing thread is much slower than the speed of generating data, so the program starts multiple writing threads while each thread establish a connection to TDengine and each thread has a message queue of fixed size. +- Application program maps the received data to different writing threads based on table name to make sure all the data for each table is always processed by a specific writing thread. +- Each writing thread writes the received data into TDengine once the message queue becomes empty or the read data meets a threshold. + +![Thread Model of High Performance Writing into TDengine](highvolume.webp) + +### Sample Programs + +The sample programs listed in this section are based on the scenario described previously. If your scenarios is different, please try to adjust the code based on the principles described in this chapter. + +The sample programs assume the source data is for all the different sub tables in same super table (meters). The super table has been created before the sample program starts to writing data. Sub tables are created automatically according to received data. If there are multiple super tables in your case, please try to adjust the part of creating table automatically. + + + + +**Program Inventory** + +| Class | Description | +| ---------------- | ----------------------------------------------------------------------------------------------------- | +| FastWriteExample | Main Program | +| ReadTask | Read data from simulated data source and put into a queue according to the hash value of table name | +| WriteTask | Read data from Queue, compose a wirte batch and write into TDengine | +| MockDataSource | Generate data for some sub tables of super table meters | +| SQLWriter | WriteTask uses this class to compose SQL, create table automatically, check SQL length and write data | +| StmtWriter | Write in Parameter binding mode (Not finished yet) | +| DataBaseMonitor | Calculate the writing speed and output on console every 10 seconds | + +Below is the list of complete code of the classes in above table and more detailed description. + +
+FastWriteExample +The main Program is responsible for: + +1. Create message queues +2. Start writing threads +3. Start reading threads +4. Otuput writing speed every 10 seconds + +The main program provides 4 parameters for tuning： + +1. The number of reading threads, default value is 1 +2. The number of writing threads, default alue is 2 +3. The total number of tables in the generated data, default value is 1000. These tables are distributed evenly across all writing threads. If the number of tables is very big, it will cost much time to firstly create these tables. +4. The batch size of single write, default value is 3,000 + +The capacity of message queue also impacts performance and can be tuned by modifying program. Normally it's always better to have a larger message queue. A larger message queue means lower possibility of being blocked when enqueueing and higher throughput. But a larger message queue consumes more memory space. The default value used in the sample programs is already big enoug. + +```java +{{#include docs/examples/java/src/main/java/com/taos/example/highvolume/FastWriteExample.java}} +``` + +
+ +
+ReadTask + +ReadTask reads data from data source. Each ReadTask is associated with a simulated data source, each data source generates data for a group of specific tables, and the data of any table is only generated from a single specific data source. + +ReadTask puts data in message queue in blocking mode. That means, the putting operation is blocked if the message queue is full. + +```java +{{#include docs/examples/java/src/main/java/com/taos/example/highvolume/ReadTask.java}} +``` + +
+ +
+WriteTask + +```java +{{#include docs/examples/java/src/main/java/com/taos/example/highvolume/WriteTask.java}} +``` + +
+ +
+ +MockDataSource + +```java +{{#include docs/examples/java/src/main/java/com/taos/example/highvolume/MockDataSource.java}} +``` + +
+ +
+ +SQLWriter + +SQLWriter class encapsulates the logic of composing SQL and writing data. Please be noted that the tables have not been created before writing, but are created automatically when catching the exception of table doesn't exist. For other exceptions caught, the SQL which caused the exception are logged for you to debug. + +```java +{{#include docs/examples/java/src/main/java/com/taos/example/highvolume/SQLWriter.java}} +``` + +
+ +
+ +DataBaseMonitor + +```java +{{#include docs/examples/java/src/main/java/com/taos/example/highvolume/DataBaseMonitor.java}} +``` + +
+ +**Steps to Launch** + +
+Launch Java Sample Program + +You need to set environment variable `TDENGINE_JDBC_URL` before launching the program. If TDengine Server is setup on localhost, then the default value for user name, password and port can be used, like below: + +``` +TDENGINE_JDBC_URL="jdbc:TAOS://localhost:6030?user=root&password=taosdata" +``` + +**Launch in IDE** + +1. Clone TDengine repolitory + ``` + git clone git@github.com:taosdata/TDengine.git --depth 1 + ``` +2. Use IDE to open `docs/examples/java` directory +3. Configure environment variable `TDENGINE_JDBC_URL`, you can also configure it before launching the IDE, if so you can skip this step. +4. Run class `com.taos.example.highvolume.FastWriteExample` + +**Launch on server** + +If you want to launch the sample program on a remote server, please follow below steps: + +1. Package the sample programs. Execute below command under directory `TDengine/docs/examples/java` ： + ``` + mvn package + ``` +2. Create `examples/java` directory on the server + ``` + mkdir -p examples/java + ``` +3. Copy dependencies (below commands assume you are working on a local Windows host and try to launch on a remote Linux host) + - Copy dependent packages + ``` + scp -r .\target\lib @:~/examples/java + ``` + - Copy the jar of sample programs + ``` + scp -r .\target\javaexample-1.0.jar @:~/examples/java + ``` +4. Configure environment variable + Edit `~/.bash_profile` or `~/.bashrc` and add below: + + ``` + export TDENGINE_JDBC_URL="jdbc:TAOS://localhost:6030?user=root&password=taosdata" + ``` + + If your TDengine server is not deployed on localhost or doesn't use default port, you need to change the above URL to correct value in your environment. + +5. Launch the sample program + + ``` + java -classpath lib/*:javaexample-1.0.jar com.taos.example.highvolume.FastWriteExample + ``` + +6. The sample program doesn't exit unless you press CTRL + C to terminate it. + Below is the output of running on a server of 16 cores, 64GB memory and SSD hard disk. + + ``` + root@vm85$ java -classpath lib/*:javaexample-1.0.jar com.taos.example.highvolume.FastWriteExample 2 12 + 18:56:35.896 [main] INFO c.t.e.highvolume.FastWriteExample - readTaskCount=2, writeTaskCount=12 tableCount=1000 maxBatchSize=3000 + 18:56:36.011 [WriteThread-0] INFO c.taos.example.highvolume.WriteTask - started + 18:56:36.015 [WriteThread-0] INFO c.taos.example.highvolume.SQLWriter - maxSQLLength=1048576 + 18:56:36.021 [WriteThread-1] INFO c.taos.example.highvolume.WriteTask - started + 18:56:36.022 [WriteThread-1] INFO c.taos.example.highvolume.SQLWriter - maxSQLLength=1048576 + 18:56:36.031 [WriteThread-2] INFO c.taos.example.highvolume.WriteTask - started + 18:56:36.032 [WriteThread-2] INFO c.taos.example.highvolume.SQLWriter - maxSQLLength=1048576 + 18:56:36.041 [WriteThread-3] INFO c.taos.example.highvolume.WriteTask - started + 18:56:36.042 [WriteThread-3] INFO c.taos.example.highvolume.SQLWriter - maxSQLLength=1048576 + 18:56:36.093 [WriteThread-4] INFO c.taos.example.highvolume.WriteTask - started + 18:56:36.094 [WriteThread-4] INFO c.taos.example.highvolume.SQLWriter - maxSQLLength=1048576 + 18:56:36.099 [WriteThread-5] INFO c.taos.example.highvolume.WriteTask - started + 18:56:36.100 [WriteThread-5] INFO c.taos.example.highvolume.SQLWriter - maxSQLLength=1048576 + 18:56:36.100 [WriteThread-6] INFO c.taos.example.highvolume.WriteTask - started + 18:56:36.101 [WriteThread-6] INFO c.taos.example.highvolume.SQLWriter - maxSQLLength=1048576 + 18:56:36.103 [WriteThread-7] INFO c.taos.example.highvolume.WriteTask - started + 18:56:36.104 [WriteThread-7] INFO c.taos.example.highvolume.SQLWriter - maxSQLLength=1048576 + 18:56:36.105 [WriteThread-8] INFO c.taos.example.highvolume.WriteTask - started + 18:56:36.107 [WriteThread-8] INFO c.taos.example.highvolume.SQLWriter - maxSQLLength=1048576 + 18:56:36.108 [WriteThread-9] INFO c.taos.example.highvolume.WriteTask - started + 18:56:36.109 [WriteThread-9] INFO c.taos.example.highvolume.SQLWriter - maxSQLLength=1048576 + 18:56:36.156 [WriteThread-10] INFO c.taos.example.highvolume.WriteTask - started + 18:56:36.157 [WriteThread-11] INFO c.taos.example.highvolume.WriteTask - started + 18:56:36.158 [WriteThread-10] INFO c.taos.example.highvolume.SQLWriter - maxSQLLength=1048576 + 18:56:36.158 [ReadThread-0] INFO com.taos.example.highvolume.ReadTask - started + 18:56:36.158 [ReadThread-1] INFO com.taos.example.highvolume.ReadTask - started + 18:56:36.158 [WriteThread-11] INFO c.taos.example.highvolume.SQLWriter - maxSQLLength=1048576 + 18:56:46.369 [main] INFO c.t.e.highvolume.FastWriteExample - count=18554448 speed=1855444 + 18:56:56.946 [main] INFO c.t.e.highvolume.FastWriteExample - count=39059660 speed=2050521 + 18:57:07.322 [main] INFO c.t.e.highvolume.FastWriteExample - count=59403604 speed=2034394 + 18:57:18.032 [main] INFO c.t.e.highvolume.FastWriteExample - count=80262938 speed=2085933 + 18:57:28.432 [main] INFO c.t.e.highvolume.FastWriteExample - count=101139906 speed=2087696 + 18:57:38.921 [main] INFO c.t.e.highvolume.FastWriteExample - count=121807202 speed=2066729 + 18:57:49.375 [main] INFO c.t.e.highvolume.FastWriteExample - count=142952417 speed=2114521 + 18:58:00.689 [main] INFO c.t.e.highvolume.FastWriteExample - count=163650306 speed=2069788 + 18:58:11.646 [main] INFO c.t.e.highvolume.FastWriteExample - count=185019808 speed=2136950 + ``` + +
+ + + + +**Program Inventory** + +Sample programs in Python uses multi-process and cross-process message queues. + +| Function/CLass | Description | +| ---------------------------- | --------------------------------------------------------------------------- | +| main Function | Program entry point, create child processes and message queues | +| run_monitor_process Function | Create database, super table, calculate writing speed and output to console | +| run_read_task Function | Read data and distribute to message queues | +| MockDataSource Class | Simulate data source, return next 1,000 rows of each table | +| run_write_task Function | Read as much as possible data from message queue and write in batch | +| SQLWriter Class | Write in SQL and create table utomatically | +| StmtWriter Class | Write in parameter binding mode (not finished yet) | + +
+main function + +`main` function is responsible for creating message queues and fork child processes, there are 3 kinds of child processes: + +1. Monitoring process, initializes database and calculating writing speed +2. Reading process (n), reads data from data source +3. Writing process (m), wirtes data into TDengine + +`main` function provides 5 parameters: + +1. The number of reading tasks, default value is 1 +2. The number of writing tasks, default value is 1 +3. The number of tables, default value is 1,000 +4. The capacity of message queue, default value is 1,000,000 bytes +5. The batch size in single write, default value is 3000 + +```python +{{#include docs/examples/python/fast_write_example.py:main}} +``` + +
+ +
+run_monitor_process + +Monitoring process initilizes database and monitoring writing speed. + +```python +{{#include docs/examples/python/fast_write_example.py:monitor}} +``` + +
+ +
+ +run_read_task function + +Reading process reads data from other data system and distributes to the message queue allocated for it. + +```python +{{#include docs/examples/python/fast_write_example.py:read}} +``` + +
+ +
+ +MockDataSource + +Below is the simulated data source, we assume table name exists in each generated data. + +```python +{{#include docs/examples/python/mockdatasource.py}} +``` + +
+ +
+run_write_task function + +Writing process tries to read as much as possible data from message queue and writes in batch. + +```python +{{#include docs/examples/python/fast_write_example.py:write}} +``` + +
+ +
+ +SQLWriter class encapsulates the logic of composing SQL and writing data. Please be noted that the tables have not been created before writing, but are created automatically when catching the exception of table doesn't exist. For other exceptions caught, the SQL which caused the exception are logged for you to debug. This class also checks the SQL length, if the SQL length is closed to `maxSQLLength` the SQL will be executed immediately. To improve writing efficiency, it's better to increase `maxSQLLength` properly. + +SQLWriter + +```python +{{#include docs/examples/python/sql_writer.py}} +``` + +
+ +**Steps to Launch** + +
+ +Launch Sample Program in Python + +1. Prerequisities + + - TDengine client driver has been installed + - Python3 has been installed, the the version >= 3.8 + - TDengine Python connector `taospy` has been installed + +2. Install faster-fifo to replace python builtin multiprocessing.Queue + + ``` + pip3 install faster-fifo + ``` + +3. Click the "Copy" in the above sample programs to copy `fast_write_example.py` 、 `sql_writer.py` and `mockdatasource.py`. + +4. Execute the program + + ``` + python3 fast_write_example.py + ``` + + Below is the output of running on a server of 16 cores, 64GB memory and SSD hard disk. + + ``` + root@vm85$ python3 fast_write_example.py 8 8 + 2022-07-14 19:13:45,869 [root] - READ_TASK_COUNT=8, WRITE_TASK_COUNT=8, TABLE_COUNT=1000, QUEUE_SIZE=1000000, MAX_BATCH_SIZE=3000 + 2022-07-14 19:13:48,882 [root] - WriteTask-0 started with pid 718347 + 2022-07-14 19:13:48,883 [root] - WriteTask-1 started with pid 718348 + 2022-07-14 19:13:48,884 [root] - WriteTask-2 started with pid 718349 + 2022-07-14 19:13:48,884 [root] - WriteTask-3 started with pid 718350 + 2022-07-14 19:13:48,885 [root] - WriteTask-4 started with pid 718351 + 2022-07-14 19:13:48,885 [root] - WriteTask-5 started with pid 718352 + 2022-07-14 19:13:48,886 [root] - WriteTask-6 started with pid 718353 + 2022-07-14 19:13:48,886 [root] - WriteTask-7 started with pid 718354 + 2022-07-14 19:13:48,887 [root] - ReadTask-0 started with pid 718355 + 2022-07-14 19:13:48,888 [root] - ReadTask-1 started with pid 718356 + 2022-07-14 19:13:48,889 [root] - ReadTask-2 started with pid 718357 + 2022-07-14 19:13:48,889 [root] - ReadTask-3 started with pid 718358 + 2022-07-14 19:13:48,890 [root] - ReadTask-4 started with pid 718359 + 2022-07-14 19:13:48,891 [root] - ReadTask-5 started with pid 718361 + 2022-07-14 19:13:48,892 [root] - ReadTask-6 started with pid 718364 + 2022-07-14 19:13:48,893 [root] - ReadTask-7 started with pid 718365 + 2022-07-14 19:13:56,042 [DataBaseMonitor] - count=6676310 speed=667631.0 + 2022-07-14 19:14:06,196 [DataBaseMonitor] - count=20004310 speed=1332800.0 + 2022-07-14 19:14:16,366 [DataBaseMonitor] - count=32290310 speed=1228600.0 + 2022-07-14 19:14:26,527 [DataBaseMonitor] - count=44438310 speed=1214800.0 + 2022-07-14 19:14:36,673 [DataBaseMonitor] - count=56608310 speed=1217000.0 + 2022-07-14 19:14:46,834 [DataBaseMonitor] - count=68757310 speed=1214900.0 + 2022-07-14 19:14:57,280 [DataBaseMonitor] - count=80992310 speed=1223500.0 + 2022-07-14 19:15:07,689 [DataBaseMonitor] - count=93805310 speed=1281300.0 + 2022-07-14 19:15:18,020 [DataBaseMonitor] - count=106111310 speed=1230600.0 + 2022-07-14 19:15:28,356 [DataBaseMonitor] - count=118394310 speed=1228300.0 + 2022-07-14 19:15:38,690 [DataBaseMonitor] - count=130742310 speed=1234800.0 + 2022-07-14 19:15:49,000 [DataBaseMonitor] - count=143051310 speed=1230900.0 + 2022-07-14 19:15:59,323 [DataBaseMonitor] - count=155276310 speed=1222500.0 + 2022-07-14 19:16:09,649 [DataBaseMonitor] - count=167603310 speed=1232700.0 + 2022-07-14 19:16:19,995 [DataBaseMonitor] - count=179976310 speed=1237300.0 + ``` + +
+ +:::note +Don't establish connection to TDengine in the parent process if using Python connector in multi-process way, otherwise all the connections in child processes are blocked always. This is a known issue. + +::: + + + diff --git a/docs/en/07-develop/03-insert-data/highvolume.webp b/docs/en/07-develop/03-insert-data/highvolume.webp new file mode 100644 index 0000000000000000000000000000000000000000..46dfc74ae3b0043c591ff930c62251da49cae7ad Binary files /dev/null and b/docs/en/07-develop/03-insert-data/highvolume.webp differ diff --git a/docs/en/14-reference/03-connector/java.mdx b/docs/en/14-reference/03-connector/java.mdx index 310e0a15c61e0d1533332d744782f66085d4e5bb..22f99bb9ae8fa669155ba8ac7cec1ad2c609cb32 100644 --- a/docs/en/14-reference/03-connector/java.mdx +++ b/docs/en/14-reference/03-connector/java.mdx @@ -91,7 +91,7 @@ Add following dependency in the `pom.xml` file of your Maven project: You can build Java connector from source code after cloning the TDengine project: ``` -git clone https://github.com/taosdata/taos-connector-jdbc.git +git clone https://github.com/taosdata/taos-connector-jdbc.git --branch 2.0 cd taos-connector-jdbc mvn clean install -Dmaven.test.skip=true ``` @@ -140,34 +140,34 @@ When you use a JDBC native connection to connect to a TDengine cluster, you can 1. Do not specify hostname and port in Java applications. - ```java - public Connection getConn() throws Exception{ - Class.forName("com.taosdata.jdbc.TSDBDriver"); - String jdbcUrl = "jdbc:TAOS://:/test?user=root&password=taosdata"; - Properties connProps = new Properties(); - connProps.setProperty(TSDBDriver.PROPERTY_KEY_CHARSET, "UTF-8"); - connProps.setProperty(TSDBDriver.PROPERTY_KEY_LOCALE, "en_US.UTF-8"); - connProps.setProperty(TSDBDriver.PROPERTY_KEY_TIME_ZONE, "UTC-8"); - Connection conn = DriverManager.getConnection(jdbcUrl, connProps); - return conn; - } - ``` + ```java + public Connection getConn() throws Exception{ + Class.forName("com.taosdata.jdbc.TSDBDriver"); + String jdbcUrl = "jdbc:TAOS://:/test?user=root&password=taosdata"; + Properties connProps = new Properties(); + connProps.setProperty(TSDBDriver.PROPERTY_KEY_CHARSET, "UTF-8"); + connProps.setProperty(TSDBDriver.PROPERTY_KEY_LOCALE, "en_US.UTF-8"); + connProps.setProperty(TSDBDriver.PROPERTY_KEY_TIME_ZONE, "UTC-8"); + Connection conn = DriverManager.getConnection(jdbcUrl, connProps); + return conn; + } + ``` 2. specify the firstEp and the secondEp in the configuration file taos.cfg - ```shell - # first fully qualified domain name (FQDN) for TDengine system - firstEp cluster_node1:6030 + ```shell + # first fully qualified domain name (FQDN) for TDengine system + firstEp cluster_node1:6030 - # second fully qualified domain name (FQDN) for TDengine system, for cluster only - secondEp cluster_node2:6030 + # second fully qualified domain name (FQDN) for TDengine system, for cluster only + secondEp cluster_node2:6030 - # default system charset - # charset UTF-8 + # default system charset + # charset UTF-8 - # system locale - # locale en_US.UTF-8 - ``` + # system locale + # locale en_US.UTF-8 + ``` In the above example, JDBC uses the client's configuration file to establish a connection to a hostname `cluster_node1`, port 6030, and a database named `test`. When the firstEp node in the cluster fails, JDBC attempts to connect to the cluster using secondEp. diff --git a/docs/zh/02-intro.md b/docs/zh/02-intro.md index 673c2e96b65814fc1cd572d54f948793ed6fa521..191e1cbcc2921b95f7e312cae2a6d84deaff65fb 100644 --- a/docs/zh/02-intro.md +++ b/docs/zh/02-intro.md @@ -52,7 +52,7 @@ TDengine的主要功能如下： 采用 TDengine，可将典型的物联网、车联网、工业互联网大数据平台的总拥有成本大幅降低。表现在几个方面： 1. 由于其超强性能，它能将系统需要的计算资源和存储资源大幅降低 -2. 因为采用 SQL 接口，能与众多第三放软件无缝集成，学习迁移成本大幅下降 +2. 因为采用 SQL 接口，能与众多第三方软件无缝集成，学习迁移成本大幅下降 3. 因为其 All In One 的特性，系统复杂度降低，能降研发成本 4. 因为运维维护简单，运营维护成本能大幅降低 diff --git a/docs/zh/04-concept/index.md b/docs/zh/04-concept/index.md index 8e97d4a2f43537c1229c8e8ea092ddfc1257dde7..0a0e4a3a2f251a5316f95c5dbb071215d0af35db 100644 --- a/docs/zh/04-concept/index.md +++ b/docs/zh/04-concept/index.md @@ -148,7 +148,7 @@ TDengine 建议用数据采集点的名字（如上表中的 D1001）来做表 3. 子表一定属于一张超级表，但普通表不属于任何超级表 4. 普通表无法转为子表，子表也无法转为普通表。 -超级表与与基于超级表建立的子表之间的关系表现在： +超级表与基于超级表建立的子表之间的关系表现在： 1. 一张超级表包含有多张子表，这些子表具有相同的采集量 schema，但带有不同的标签值。 2. 不能通过子表调整数据或标签的模式，对于超级表的数据模式修改立即对所有的子表生效。 diff --git a/docs/zh/14-reference/03-connector/java.mdx b/docs/zh/14-reference/03-connector/java.mdx index 46b23c0a64da82a27769179179864c8553ff890f..f7bd540088f28528f36e63e13b2c4917f497c3bc 100644 --- a/docs/zh/14-reference/03-connector/java.mdx +++ b/docs/zh/14-reference/03-connector/java.mdx @@ -93,7 +93,7 @@ Maven 项目中，在 pom.xml 中添加以下依赖： 可以通过下载 TDengine 的源码，自己编译最新版本的 Java connector ```shell -git clone https://github.com/taosdata/taos-connector-jdbc.git +git clone https://github.com/taosdata/taos-connector-jdbc.git --branch 2.0 cd taos-connector-jdbc mvn clean install -Dmaven.test.skip=true ``` diff --git a/packaging/release.sh b/packaging/release.sh index 4823c9d10b4f15de76b79f65a0b542d8e99aa5b3..0ad8d9b1bfaa09a4be51c8448c2494feff2cdbf7 100755 --- a/packaging/release.sh +++ b/packaging/release.sh @@ -198,6 +198,7 @@ fi if [[ "$dbName" != "taos" ]]; then source ${enterprise_dir}/packaging/oem/sed_$dbName.sh replace_community_$dbName + replace_output_$dbName fi if [[ "$httpdBuild" == "true" ]]; then @@ -224,6 +225,7 @@ if [[ "$cpuType" == "x64" ]] || [[ "$cpuType" == "aarch64" ]] || [[ "$cpuType" = else if [[ "$dbName" != "taos" ]]; then replace_enterprise_$dbName + replace_output_$dbName fi cmake ../../ -DCPUTYPE=${cpuType} -DOSTYPE=${osType} -DSOMODE=${soMode} -DDBNAME=${dbName} -DVERTYPE=${verType} -DVERDATE="${build_time}" -DGITINFO=${gitinfo} -DGITINFOI=${gitinfoOfInternal} -DVERNUMBER=${verNumber} -DVERCOMPATIBLE=${verNumberComp} -DBUILD_HTTP=${BUILD_HTTP} -DBUILD_TOOLS=${BUILD_TOOLS} ${allocator_macro} fi diff --git a/src/client/src/tscSQLParser.c b/src/client/src/tscSQLParser.c index cc2a1fed78f1c318b555539caadc4f4d37a0e0d1..bb31b752a1bd5856f45dde901c6c589c92063279 100644 --- a/src/client/src/tscSQLParser.c +++ b/src/client/src/tscSQLParser.c @@ -3972,10 +3972,6 @@ int32_t doGetColumnIndexByName(SStrToken* pToken, SQueryInfo* pQueryInfo, SColum const char* msg0 = "ambiguous column name"; const char* msg1 = "invalid column name"; - if (pToken->n == 0) { - return TSDB_CODE_TSC_INVALID_OPERATION; - } - int16_t tsWinColumnIndex; if (isTablenameToken(pToken)) { pIndex->columnIndex = TSDB_TBNAME_COLUMN_INDEX; @@ -4061,6 +4057,12 @@ int32_t getTableIndexByName(SStrToken* pToken, SQueryInfo* pQueryInfo, SColumnIn } int32_t getColumnIndexByName(const SStrToken* pToken, SQueryInfo* pQueryInfo, SColumnIndex* pIndex, char* msg) { + const char* msg0 = "invalid column name"; + + if (pToken->n == 0) { + return invalidOperationMsg(msg, msg0); + } + if (pQueryInfo->pTableMetaInfo == NULL || pQueryInfo->numOfTables == 0) { return TSDB_CODE_TSC_INVALID_OPERATION; } @@ -7399,6 +7401,9 @@ int32_t setAlterTableInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) { } SColumnIndex idx = COLUMN_INDEX_INITIALIZER; + if (pItem->pVar.nType != TSDB_DATA_TYPE_BINARY) { + return invalidOperationMsg(pMsg, msg17); + } SStrToken name = {.z = pItem->pVar.pz, .n = pItem->pVar.nLen}; if (getColumnIndexByName(&name, pQueryInfo, &idx, tscGetErrorMsgPayload(pCmd)) != TSDB_CODE_SUCCESS) { @@ -7472,6 +7477,9 @@ int32_t setAlterTableInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) { int16_t numOfTags = tscGetNumOfTags(pTableMeta); SColumnIndex columnIndex = COLUMN_INDEX_INITIALIZER; + if (item->pVar.nType != TSDB_DATA_TYPE_BINARY) { + return invalidOperationMsg(pMsg, msg17); + } SStrToken name = {.z = item->pVar.pz, .n = item->pVar.nLen}; if (getColumnIndexByName(&name, pQueryInfo, &columnIndex, tscGetErrorMsgPayload(pCmd)) != TSDB_CODE_SUCCESS) { return TSDB_CODE_TSC_INVALID_OPERATION; @@ -7617,6 +7625,9 @@ int32_t setAlterTableInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) { tVariantListItem* pItem = taosArrayGet(pAlterSQL->varList, 0); SColumnIndex columnIndex = COLUMN_INDEX_INITIALIZER; + if (pItem->pVar.nType != TSDB_DATA_TYPE_BINARY) { + return invalidOperationMsg(pMsg, msg17); + } SStrToken name = {.z = pItem->pVar.pz, .n = pItem->pVar.nLen}; if (getColumnIndexByName(&name, pQueryInfo, &columnIndex, tscGetErrorMsgPayload(pCmd)) != TSDB_CODE_SUCCESS) { @@ -9665,7 +9676,9 @@ int32_t tscGetExprFilters(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SArray* pSelect SStrToken* pToken = &pParam->pNode->columnName; SColumnIndex idx = COLUMN_INDEX_INITIALIZER; - getColumnIndexByName(pToken, pQueryInfo, &idx, tscGetErrorMsgPayload(pCmd)); + if (getColumnIndexByName(pToken, pQueryInfo, &idx, tscGetErrorMsgPayload(pCmd)) != TSDB_CODE_SUCCESS) { + return TSDB_CODE_TSC_INVALID_OPERATION; + } STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, idx.tableIndex); schema = *tscGetTableColumnSchema(pTableMetaInfo->pTableMeta, idx.columnIndex); } else { diff --git a/src/kit/shell/CMakeLists.txt b/src/kit/shell/CMakeLists.txt index b311361c438d033ad3f7582d30df7d1c33357c1d..614fa328bae1f0ad32917da6b08dc372a8d696a2 100644 --- a/src/kit/shell/CMakeLists.txt +++ b/src/kit/shell/CMakeLists.txt @@ -46,6 +46,8 @@ ELSEIF (TD_DARWIN) LIST(APPEND SRC ./src/shellCommand.c) LIST(APPEND SRC ./src/shellImport.c) LIST(APPEND SRC ./src/shellCheck.c) + LIST(APPEND SRC ./src/shellAuto.c) + LIST(APPEND SRC ./src/tire.c) ADD_EXECUTABLE(shell ${SRC}) # linking with dylib TARGET_LINK_LIBRARIES(shell taos cJson) diff --git a/src/kit/shell/inc/shellAuto.h b/src/kit/shell/inc/shellAuto.h new file mode 100644 index 0000000000000000000000000000000000000000..0bd6bdf4038c112b453feea02950cc3aa5577a50 --- /dev/null +++ b/src/kit/shell/inc/shellAuto.h @@ -0,0 +1,37 @@ +/* + * Copyright (c) 2019 TAOS Data, Inc. + * + * This program is free software: you can use, redistribute, and/or modify + * it under the terms of the GNU Affero General Public License, version 3 + * or later ("AGPL"), as published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ + +#ifndef __SHELL_AUTO__ +#define __SHELL_AUTO__ + +#define TAB_KEY 0x09 + +// press tab key +void pressTabKey(TAOS * con, Command * cmd); + +// press othr key +void pressOtherKey(char c); + +// init shell auto funciton , shell start call once +bool shellAutoInit(); + +// exit shell auto funciton, shell exit call once +void shellAutoExit(); + +// callback autotab module +void callbackAutoTab(char* sqlstr, TAOS* pSql, bool usedb); + + +#endif diff --git a/src/kit/shell/inc/shellCommand.h b/src/kit/shell/inc/shellCommand.h index 6e4d3e382e3d7e8c50405c07da8ed73725230434..47ef6b30a9b37ef0d790bcfc427abdfc14907874 100644 --- a/src/kit/shell/inc/shellCommand.h +++ b/src/kit/shell/inc/shellCommand.h @@ -41,6 +41,7 @@ extern void deleteChar(Command *cmd); extern void moveCursorLeft(Command *cmd); extern void moveCursorRight(Command *cmd); extern void positionCursorHome(Command *cmd); +extern void positionCursorMiddle(Command *cmd); extern void positionCursorEnd(Command *cmd); extern void showOnScreen(Command *cmd); extern void updateBuffer(Command *cmd); @@ -51,5 +52,6 @@ int countPrefixOnes(unsigned char c); void clearScreen(int ecmd_pos, int cursor_pos); void printChar(char c, int times); void positionCursor(int step, int direction); +void getPrevCharSize(const char *str, int pos, int *size, int *width); #endif diff --git a/src/kit/shell/inc/tire.h b/src/kit/shell/inc/tire.h new file mode 100644 index 0000000000000000000000000000000000000000..88bae5480937cfdb1513415d13ba41d0a60e6b22 --- /dev/null +++ b/src/kit/shell/inc/tire.h @@ -0,0 +1,92 @@ +/* + * Copyright (c) 2019 TAOS Data, Inc. + * + * This program is free software: you can use, redistribute, and/or modify + * it under the terms of the GNU Affero General Public License, version 3 + * or later ("AGPL"), as published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ + +#ifndef __TRIE__ +#define __TRIE__ + +// +// The prefix search tree is a efficient storage words and search words tree, it support 95 visible ascii code character +// +#define FIRST_ASCII 40 // first visiable char is '0' +#define LAST_ASCII 122 // last visilbe char is 'z' + +// capacity save char is 95 +#define CHAR_CNT (LAST_ASCII - FIRST_ASCII + 1) +#define MAX_WORD_LEN 256 // max insert word length + +// define STire +#define TIRE_TREE 0 +#define TIRE_LIST 1 + +typedef struct STireNode { + struct STireNode** d; + bool end; // record end flag +}STireNode; + +typedef struct StrName { + char * name; + struct StrName * next; +}StrName; + + +typedef struct STire { + char type; // see define TIRE_ + STireNode root; + + StrName * head; + StrName * tail; + + int count; // all count + int ref; +}STire; + +typedef struct SMatchNode { + char* word; + struct SMatchNode* next; +}SMatchNode; + + +typedef struct SMatch { + SMatchNode* head; + SMatchNode* tail; // append node to tail + int count; + char pre[MAX_WORD_LEN]; +}SMatch; + + +// ----------- interface ------------- + +// create prefix search tree, return value call freeTire to free +STire* createTire(char type); + +// destroy prefix search tree +void freeTire(STire* tire); + +// add a new word +bool insertWord(STire* tire, char* word); + +// add a new word +bool deleteWord(STire* tire, char* word); + +// match prefix words, if match is not NULL , put all item to match and return match +SMatch* matchPrefix(STire* tire, char* prefix, SMatch* match); + +// get all items from tires tree +SMatch* enumAll(STire* tire); + +// free match result +void freeMatch(SMatch* match); + +#endif diff --git a/src/kit/shell/src/shellAuto.c b/src/kit/shell/src/shellAuto.c new file mode 100644 index 0000000000000000000000000000000000000000..8622b201a6fe6666476f0ac9916aebc169b78923 --- /dev/null +++ b/src/kit/shell/src/shellAuto.c @@ -0,0 +1,1761 @@ +/* + * Copyright (c) 2019 TAOS Data, Inc. + * + * This program is free software: you can use, redistribute, and/or modify + * it under the terms of the GNU Affero General Public License, version 3 + * or later ("AGPL"), as published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ + +#define __USE_XOPEN +#include "os.h" +#include "tglobal.h" +#include "shell.h" +#include "shellCommand.h" +#include "tkey.h" +#include "tulog.h" +#include "shellAuto.h" +#include "tire.h" +#include "tthread.h" + +// +// ------------- define area --------------- +// +#define UNION_ALL " union all " + + +// extern function +void insertChar(Command *cmd, char *c, int size); + + +typedef struct SAutoPtr { + STire* p; + int ref; +}SAutoPtr; + +typedef struct SWord{ + int type ; // word type , see WT_ define + char * word; + int32_t len; + struct SWord * next; + bool free; // if true need free +}SWord; + +typedef struct { + char * source; + int32_t source_len; // valid data length in source + int32_t count; + SWord* head; + // matched information + int32_t matchIndex; // matched word index in words + int32_t matchLen; // matched length at matched word +}SWords; + + +SWords shellCommands[] = { + {"alter database ", 0, 0, NULL}, + {"alter dnode balance ", 0, 0, NULL}, + {"alter dnode resetlog;", 0, 0, NULL}, + {"alter dnode debugFlag 141;", 0, 0, NULL}, + {"alter dnode monitor 1;", 0, 0, NULL}, + {"alter table ", 0, 0, NULL}, + {"alter table modify column", 0, 0, NULL}, + {"alter local resetlog;", 0, 0, NULL}, + {"alter local DebugFlag 143;", 0, 0, NULL}, + {"alter local cDebugFlag 143;", 0, 0, NULL}, + {"alter local uDebugFlag 143;", 0, 0, NULL}, + {"alter local rpcDebugFlag 143;", 0, 0, NULL}, + {"alter local tmrDebugFlag 143;", 0, 0, NULL}, + {"alter topic", 0, 0, NULL}, + {"alter user pass", 0, 0, NULL}, + {"alter user privilege read", 0, 0, NULL}, + {"alter user privilege write", 0, 0, NULL}, + {"create table using tags(", 0, 0, NULL}, + {"create database ", 0, 0, NULL}, + {"create table as ", 0, 0, NULL}, + {"create dnode ", 0, 0, NULL}, + {"create topic", 0, 0, NULL}, + {"create function ", 0, 0, NULL}, + {"create user pass", 0, 0, NULL}, + {"compact vnode in", 0, 0, NULL}, + {"describe ", 0, 0, NULL}, +#ifdef TD_ENTERPRISE + {"delete from where", 0, 0, NULL}, +#endif + {"drop database ", 0, 0, NULL}, + {"drop table ", 0, 0, NULL}, + {"drop dnode ", 0, 0, NULL}, + {"drop user ;", 0, 0, NULL}, + {"drop function", 0, 0, NULL}, + {"drop topic", 0, 0, NULL}, + {"kill connection", 0, 0, NULL}, + {"kill query", 0, 0, NULL}, + {"kill stream", 0, 0, NULL}, + {"select * from where ", 0, 0, NULL}, + {"select _block_dist() from \\G;", 0, 0, NULL}, + {"select client_version();", 0, 0, NULL}, + {"select current_user();", 0, 0, NULL}, + {"select database;", 0, 0, NULL}, + {"select server_version();", 0, 0, NULL}, + {"set max_binary_display_width ", 0, 0, NULL}, + {"show create database \\G;", 0, 0, NULL}, + {"show create stable \\G;", 0, 0, NULL}, + {"show create table \\G;", 0, 0, NULL}, + {"show connections;", 0, 0, NULL}, + {"show databases;", 0, 0, NULL}, + {"show dnodes;", 0, 0, NULL}, + {"show functions;", 0, 0, NULL}, + {"show modules;", 0, 0, NULL}, + {"show mnodes;", 0, 0, NULL}, + {"show queries;", 0, 0, NULL}, + {"show stables;", 0, 0, NULL}, + {"show stables like ", 0, 0, NULL}, + {"show streams;", 0, 0, NULL}, + {"show scores;", 0, 0, NULL}, + {"show tables;", 0, 0, NULL}, + {"show tables like", 0, 0, NULL}, + {"show users;", 0, 0, NULL}, + {"show variables;", 0, 0, NULL}, + {"show vgroups;", 0, 0, NULL}, + {"insert into values(", 0, 0, NULL}, + {"insert into using tags(", 0, 0, NULL}, + {"use ", 0, 0, NULL}, + {"quit", 0, 0, NULL} +}; + +char * keywords[] = { + "and ", + "asc ", + "desc ", + "from ", + "fill(", + "limit ", + "where ", + "interval(", + "order by ", + "order by ", + "offset ", + "or ", + "group by ", + "now()", + "session(", + "sliding ", + "slimit ", + "soffset ", + "state_window(", + "today() ", + "union all select ", +}; + +char * functions[] = { + "count(", + "sum(", + "avg(", + "last(", + "last_row(", + "top(", + "interp(", + "max(", + "min(", + "now()", + "today()", + "percentile(", + "tail(", + "pow(", + "abs(", + "atan(", + "acos(", + "asin(", + "apercentile(", + "bottom(", + "cast(", + "ceil(", + "char_length(", + "cos(", + "concat(", + "concat_ws(", + "csum(", + "diff(", + "derivative(", + "elapsed(", + "first(", + "floor(", + "hyperloglog(", + "histogram(", + "irate(", + "leastsquares(", + "length(", + "log(", + "lower(", + "ltrim(", + "mavg(", + "mode(", + "tan(", + "round(", + "rtrim(", + "sample(", + "sin(", + "spread(", + "substr(", + "statecount(", + "stateduration(", + "stddev(", + "sqrt(", + "timediff(", + "timezone(", + "timetruncate(", + "twa(", + "to_unixtimestamp(", + "unique(", + "upper(", +}; + +char * tb_actions[] = { + "add column", + "modify column", + "drop column", + "change tag", +}; + +char * db_options[] = { + "blocks", + "cachelast", + "comp", + "keep", + "replica", + "quorum", +}; + +char * data_types[] = { + "timestamp", + "int", + "float", + "double", + "binary(16)", + "nchar(16)", + "bigint", + "smallint", + "tinyint", + "bool", + "json" +}; + +char * key_tags[] = { + "tags(" +}; + + +// +// ------- gobal variant define --------- +// +int32_t firstMatchIndex = -1; // first match shellCommands index +int32_t lastMatchIndex = -1; // last match shellCommands index +int32_t curMatchIndex = -1; // current match shellCommands index +int32_t lastWordBytes = -1; // printShow last word length +bool waitAutoFill = false; + + +// +// ----------- global var array define ----------- +// +#define WT_VAR_DBNAME 0 +#define WT_VAR_STABLE 1 +#define WT_VAR_TABLE 2 +#define WT_VAR_DNODEID 3 +#define WT_VAR_USERNAME 4 +#define WT_VAR_ALLTABLE 5 +#define WT_VAR_FUNC 6 +#define WT_VAR_KEYWORD 7 +#define WT_VAR_TBACTION 8 +#define WT_VAR_DBOPTION 9 +#define WT_VAR_DATATYPE 10 +#define WT_VAR_KEYTAGS 11 +#define WT_VAR_ANYWORD 12 +#define WT_VAR_CNT 13 + +#define WT_FROM_DB_MAX 4 // max get content from db +#define WT_FROM_DB_CNT (WT_FROM_DB_MAX + 1) + +#define WT_TEXT 0xFF + +char dbName[256] = ""; // save use database name; +// tire array +STire* tires[WT_VAR_CNT]; +pthread_mutex_t tiresMutex; +//save thread handle obtain var name from db server +pthread_t* threads[WT_FROM_DB_CNT]; +// obtain var name with sql from server +char varTypes[WT_VAR_CNT][64] = { + "", + "", + "", + "", + "", + "", + "", + "", + "", + "", + "", + "", + "" +}; + +char varSqls[WT_FROM_DB_CNT][64] = { + "show databases;", + "show stables;", + "show tables;", + "show dnodes;", + "show users;" +}; + + +// var words current cursor, if user press any one key except tab, cursorVar can be reset to -1 +int cursorVar = -1; +bool varMode = false; // enter var names list mode + + +TAOS* varCon = NULL; +Command* varCmd = NULL; +SMatch* lastMatch = NULL; // save last match result +int cntDel = 0; // delete byte count after next press tab + + +// show auto tab introduction +void printfIntroduction() { + printf(" ********************* How to Use TAB in TAOS Shell ******************************\n"); + printf(" * Taos shell supports pressing TAB key to complete word. You can try it. *\n"); + printf(" * Press TAB key anywhere, You'll get surprise. *\n"); + printf(" * KEYBOARD SHORTCUT: *\n"); + printf(" * [ TAB ] ...... Complete the word or show help if no input *\n"); + printf(" * [ Ctrl + A ] ...... move cursor to [A]head of line *\n"); + printf(" * [ Ctrl + E ] ...... move cursor to [E]nd of line *\n"); + printf(" * [ Ctrl + W ] ...... move cursor to line of middle *\n"); + printf(" * [ Ctrl + L ] ...... clean screen *\n"); + printf(" * [ Ctrl + K ] ...... clean after cursor *\n"); + printf(" * [ Ctrl + U ] ...... clean before cursor *\n"); + printf(" * *\n"); + printf(" **********************************************************************************\n\n"); +} + +void showHelp() { + printf("\nThe following are supported commands for Taos shell:"); + printf("\n\ + ----- A ----- \n\ + alter database \n\ + alter dnode balance \n\ + alter dnode resetlog;\n\ + alter dnode DebugFlag 143;\n\ + alter dnode monitor 1;\n\ + alter table ADD COLUMN ; \n\ + alter table DROP COLUMN ; \n\ + alter table MODIFY COLUMN ;\n\ + alter local resetlog; \n\ + alter local DebugFlag 143; \n\ + alter topic \n\ + alter user pass\n\ + alter user privilege read ;\n\ + alter user privilege write ;\n\ + ----- C ----- \n\ + create table using tags ...\n\ + create database ;\n\ + create table as ...\n\ + create dnode \n\ + create topic \n\ + create function \n\ + create user pass ;\n\ + compact vnode in (vgid,vgid,vgid);\n\ + ----- D ----- \n\ + describe ;\n\ + delete from where ... \n\ + drop database ;\n\ + drop table ;\n\ + drop dnode ;\n\ + drop function ;\n\ + drop topic ;\n\ + drop user ;\n\ + ----- K ----- \n\ + kill connection ; \n\ + kill query ; \n\ + kill stream ; \n\ + ----- S ----- \n\ + select * from where ... \n\ + select _block_dist() from ;\n\ + select client_version();\n\ + select current_user();\n\ + select database;\n\ + select server_version();\n\ + set max_binary_display_width ; \n\ + show create database ;\n\ + show create stable ;\n\ + show create table ;\n\ + show connections;\n\ + show databases;\n\ + show dnodes;\n\ + show functions;\n\ + show modules;\n\ + show mnodes;\n\ + show queries;\n\ + show stables;\n\ + show stables like ''; note: regular expression only support '_' and '%%' match.\n\ + show streams;\n\ + show scores;\n\ + show tables;\n\ + show tables like ''; \n\ + show users;\n\ + show variables;\n\ + show vgroups;\n\ + ----- I ----- \n\ + insert into values(...) ;\n\ + ----- U ----- \n\ + use ;"); + + printf("\n\n"); + + //define in getDuration() function + printf("\ + Timestamp expression Format:\n\ + b - nanosecond \n\ + u - microsecond \n\ + a - millisecond \n\ + s - second \n\ + m - minute \n\ + h - hour \n\ + d - day \n\ + w - week \n\ + now - current time \n\ + Example : \n\ + select * from t1 where ts > now - 2w + 3d and ts <= now - 1w -2h ;\n"); + printf("\n"); +} + +// +// ------------------- parse words -------------------------- +// + +#define SHELL_COMMAND_COUNT() (sizeof(shellCommands) / sizeof(SWords)) + +// get at +SWord * atWord(SWords * command, int32_t index) { + SWord * word = command->head; + for (int32_t i = 0; i < index; i++) { + if (word == NULL) + return NULL; + word = word->next; + } + + return word; +} + +#define MATCH_WORD(x) atWord(x, x->matchIndex) + +int wordType(const char* p, int32_t len) { + for (int i = 0; i < WT_VAR_CNT; i++) { + if (strncmp(p, varTypes[i], len) == 0) + return i; + } + return WT_TEXT; +} + +// add word +SWord * addWord(const char* p, int32_t len, bool pattern) { + SWord* word = (SWord *) malloc(sizeof(SWord)); + memset(word, 0, sizeof(SWord)); + word->word = (char* )p; + word->len = len; + + // check format + if (pattern) { + word->type = wordType(p, len); + } else { + word->type = WT_TEXT; + } + + return word; +} + +// parse one command +void parseCommand(SWords * command, bool pattern) { + char * p = command->source; + int32_t start = 0; + int32_t size = command->source_len > 0 ? command->source_len : strlen(p); + + bool lastBlank = false; + for (int i = 0; i <= size; i++) { + if (p[i] == ' ' || i == size) { + // check continue blank like ' ' + if (p[i] == ' ') { + if (lastBlank) { + start ++; + continue; + } + if (i == 0) { // first blank + lastBlank = true; + start ++; + continue; + } + lastBlank = true; + } + + // found split or string end , append word + if (command->head == NULL) { + command->head = addWord(p + start, i - start, pattern); + command->count = 1; + } else { + SWord * word = command->head; + while (word->next) { + word = word->next; + } + word->next = addWord(p + start, i - start, pattern); + command->count ++; + } + start = i + 1; + } else { + lastBlank = false; + } + } +} + +// free Command +void freeCommand(SWords * command) { + SWord * word = command->head; + if (word == NULL) { + return ; + } + + // loop + while (word->next) { + SWord * tmp = word; + word = word->next; + // if malloc need free + if(tmp->free && tmp->word) + free(tmp->word); + free(tmp); + } + + // if malloc need free + if(word->free && word->word) + free(word->word); + free(word); +} + +void GenerateVarType(int type, char** p, int count) { + STire* tire = createTire(TIRE_LIST); + for (int i = 0; i < count; i++) { + insertWord(tire, p[i]); + } + + pthread_mutex_lock(&tiresMutex); + tires[type] = tire; + pthread_mutex_unlock(&tiresMutex); +} + +// +// -------------------- shell auto ---------------- +// + + +// init shell auto funciton , shell start call once +bool shellAutoInit() { + // command + int32_t count = SHELL_COMMAND_COUNT(); + for (int32_t i = 0; i < count; i ++) { + parseCommand(shellCommands + i, true); + } + + // tires + memset(tires, 0, sizeof(STire*) * WT_VAR_CNT); + pthread_mutex_init(&tiresMutex, NULL); + + // threads + memset(threads, 0, sizeof(pthread_t*) * WT_FROM_DB_CNT); + + // generate varType + GenerateVarType(WT_VAR_FUNC, functions, sizeof(functions) /sizeof(char *)); + GenerateVarType(WT_VAR_KEYWORD, keywords, sizeof(keywords) /sizeof(char *)); + GenerateVarType(WT_VAR_DBOPTION, db_options, sizeof(db_options) /sizeof(char *)); + GenerateVarType(WT_VAR_TBACTION, tb_actions, sizeof(tb_actions) /sizeof(char *)); + GenerateVarType(WT_VAR_DATATYPE, data_types, sizeof(data_types) /sizeof(char *)); + GenerateVarType(WT_VAR_KEYTAGS, key_tags, sizeof(key_tags) /sizeof(char *)); + + printfIntroduction(); + + return true; +} + +// exit shell auto funciton, shell exit call once +void shellAutoExit() { + // free command + int32_t count = SHELL_COMMAND_COUNT(); + for (int32_t i = 0; i < count; i ++) { + freeCommand(shellCommands + i); + } + + // free tires + pthread_mutex_lock(&tiresMutex); + for (int32_t i = 0; i < WT_VAR_CNT; i++) { + if (tires[i]) { + freeTire(tires[i]); + tires[i] = NULL; + } + } + pthread_mutex_unlock(&tiresMutex); + // destory + pthread_mutex_destroy(&tiresMutex); + + // free threads + for (int32_t i = 0; i < WT_VAR_CNT; i++) { + if (threads[i]) { + taosDestroyThread(threads[i]); + threads[i] = NULL; + } + } + + // free lastMatch + if (lastMatch) { + freeMatch(lastMatch); + lastMatch = NULL; + } +} + +// +// ------------------- auto ptr for tires -------------------------- +// +bool setNewAuotPtr(int type, STire* pNew) { + if (pNew == NULL) + return false; + + pthread_mutex_lock(&tiresMutex); + STire* pOld = tires[type]; + if (pOld != NULL) { + // previous have value, release self ref count + if (--pOld->ref == 0) { + freeTire(pOld); + } + } + + // set new + tires[type] = pNew; + tires[type]->ref = 1; + pthread_mutex_unlock(&tiresMutex); + + return true; +} + +// get ptr +STire* getAutoPtr(int type) { + if (tires[type] == NULL) { + return NULL; + } + + pthread_mutex_lock(&tiresMutex); + tires[type]->ref++; + pthread_mutex_unlock(&tiresMutex); + + return tires[type]; +} + +// put back tire to tires[type], if tire not equal tires[type].p, need free tire +void putBackAutoPtr(int type, STire* tire) { + if (tire == NULL) { + return ; + } + + pthread_mutex_lock(&tiresMutex); + if (tires[type] != tire) { + //update by out, can't put back , so free + if (--tire->ref == 1) { + // support multi thread getAuotPtr + freeTire(tire); + } + + } else { + tires[type]->ref--; + assert(tires[type]->ref > 0); + } + pthread_mutex_unlock(&tiresMutex); + + return ; +} + + + +// +// ------------------- var Word -------------------------- +// + +#define MAX_CACHED_CNT 100000 // max cached rows 10w +// write sql result to var name, return write rows cnt +int writeVarNames(int type, TAOS_RES* tres) { + // fetch row + TAOS_ROW row = taos_fetch_row(tres); + if (row == NULL) { + return 0; + } + + TAOS_FIELD *fields = taos_fetch_fields(tres); + // create new tires + char tireType = type == WT_VAR_TABLE ? TIRE_TREE : TIRE_LIST; + STire* tire = createTire(tireType); + + // enum rows + char name[1024]; + int numOfRows = 0; + do { + int32_t* lengths = taos_fetch_lengths(tres); + int32_t bytes = lengths[0]; + if(fields[0].type == TSDB_DATA_TYPE_SMALLINT) { + sprintf(name,"%d", *(int16_t*)row[0]); + } else { + memcpy(name, row[0], bytes); + } + + name[bytes] = 0; //set string end + // insert to tire + insertWord(tire, name); + + if (++numOfRows > MAX_CACHED_CNT ) { + break; + } + + row = taos_fetch_row(tres); + } while (row != NULL); + + // replace old tire + setNewAuotPtr(type, tire); + + return numOfRows; +} + +bool firstMatchCommand(TAOS * con, Command * cmd); +// +// thread obtain var thread from db server +// +void* varObtainThread(void* param) { + int type = *(int* )param; + free(param); + + if (varCon == NULL || type > WT_FROM_DB_MAX) { + return NULL; + } + + TAOS_RES* pSql = taos_query_h(varCon, varSqls[type], NULL); + if (taos_errno(pSql)) { + taos_free_result(pSql); + return NULL; + } + + // write var names from pSql + int cnt = writeVarNames(type, pSql); + + // free sql + taos_free_result(pSql); + + // check need call auto tab + if (cnt > 0 && waitAutoFill) { + // press tab key by program + firstMatchCommand(varCon, varCmd); + } + + return NULL; +} + +// only match next one word from all match words, return valuue must free by caller +char* matchNextPrefix(STire* tire, char* pre) { + SMatch* match = NULL; + + // re-use last result + if (lastMatch) { + if (strcmp(pre, lastMatch->pre) == 0) { + // same pre + match = lastMatch; + } + } + + if (match == NULL) { + // not same with last result + if (pre[0] == 0) { + // EMPTY PRE + match = enumAll(tire); + } else { + // NOT EMPTY + match = matchPrefix(tire, pre, NULL); + } + + // save to lastMatch + if (match) { + if (lastMatch) + freeMatch(lastMatch); + lastMatch = match; + } + } + + // check valid + if (match == NULL || match->head == NULL) { + // no one matched + return false; + } + + if (cursorVar == -1) { + // first + cursorVar = 0; + return strdup(match->head->word); + } + + // according to cursorVar , calculate next one + int i = 0; + SMatchNode* item = match->head; + while (item) { + if (i == cursorVar + 1) { + // found next position ok + if (item->next == NULL) { + // match last item, reset cursorVar to head + cursorVar = -1; + } else { + cursorVar = i; + } + + return strdup(item->word); + } + + // check end item + if (item->next == NULL) { + // if cursorVar > var list count, return last and reset cursorVar + cursorVar = -1; + + return strdup(item->word); + } + + // move next + item = item->next; + i++; + } + + return NULL; +} + +// search pre word from tire tree, return value must free by caller +char* tireSearchWord(int type, char* pre) { + if (type == WT_TEXT) { + return NULL; + } + + if(type > WT_FROM_DB_MAX) { + // NOT FROM DB , tires[type] alwary not null + STire* tire = tires[type]; + if (tire == NULL) + return NULL; + return matchNextPrefix(tire, pre); + } + + // TYPE CONTEXT GET FROM DB + pthread_mutex_lock(&tiresMutex); + + // check need obtain from server + if (tires[type] == NULL) { + waitAutoFill = true; + // need async obtain var names from db sever + if (threads[type] != NULL) { + if (taosThreadRunning(threads[type])) { + // thread running , need not obtain again, return + pthread_mutex_unlock(&tiresMutex); + return NULL; + } + // destroy previous thread handle for new create thread handle + taosDestroyThread(threads[type]); + threads[type] = NULL; + } + + // create new + void * param = malloc(sizeof(int)); + *((int* )param) = type; + threads[type] = taosCreateThread(varObtainThread, param); + pthread_mutex_unlock(&tiresMutex); + return NULL; + } + pthread_mutex_unlock(&tiresMutex); + + // can obtain var names from local + STire* tire = getAutoPtr(type); + if (tire == NULL) { + return NULL; + } + + char* str = matchNextPrefix(tire, pre); + // used finish, put back pointer to autoptr array + putBackAutoPtr(type, tire); + + return str; +} + +// match var word, word1 is pattern , word2 is input from shell +bool matchVarWord(SWord* word1, SWord* word2) { + // search input word from tire tree + char pre[512]; + memcpy(pre, word2->word, word2->len); + pre[word2->len] = 0; + + char* str = NULL; + if (word1->type == WT_VAR_ALLTABLE) { + // ALL_TABLE + str = tireSearchWord(WT_VAR_STABLE, pre); + if (str == NULL) { + str = tireSearchWord(WT_VAR_TABLE, pre); + if(str == NULL) + return false; + } + } else { + // OTHER + str = tireSearchWord(word1->type, pre); + if (str == NULL) { + // not found or word1->type variable list not obtain from server, return not match + return false; + } + } + + // free previous malloc + if(word1->free && word1->word) { + free(word1->word); + } + + // save + word1->word = str; + word1->len = strlen(str); + word1->free = true; // need free + + return true; +} + +// +// ------------------- match words -------------------------- +// + + +// compare command cmd1 come from shellCommands , cmd2 come from user input +int32_t compareCommand(SWords * cmd1, SWords * cmd2) { + SWord * word1 = cmd1->head; + SWord * word2 = cmd2->head; + + if (word1 == NULL || word2 == NULL) { + return -1; + } + + for (int32_t i = 0; i < cmd1->count; i++) { + if (word1->type == WT_TEXT) { + // WT_TEXT match + if (word1->len == word2->len) { + if (strncasecmp(word1->word, word2->word, word1->len) != 0) + return -1; + } else if (word1->len < word2->len) { + return -1; + } else { + // word1->len > word2->len + if (strncasecmp(word1->word, word2->word, word2->len) == 0) { + cmd1->matchIndex = i; + cmd1->matchLen = word2->len; + return i; + } else { + return -1; + } + } + } else { + // WT_VAR auto match any one word + if (word2->next == NULL) { // input words last one + if (matchVarWord(word1, word2)) { + cmd1->matchIndex = i; + cmd1->matchLen = word2->len; + varMode = true; + return i; + } + return -1; + } + } + + // move next + word1 = word1->next; + word2 = word2->next; + if (word1 == NULL || word2 == NULL) { + return -1; + } + } + + return -1; +} + +// match command +SWords * matchCommand(SWords * input, bool continueSearch) { + int32_t count = SHELL_COMMAND_COUNT(); + for (int32_t i = 0; i < count; i ++) { + SWords * shellCommand = shellCommands + i; + if (continueSearch && lastMatchIndex != -1 && i <= lastMatchIndex) { + // new match must greate than lastMatchIndex + if (varMode && i == lastMatchIndex) { + // do nothing, var match on lastMatchIndex + } else { + continue; + } + } + + // command is large + if (input->count > shellCommand->count ) { + continue; + } + + // compare + int32_t index = compareCommand(shellCommand, input); + if (index != -1) { + if (firstMatchIndex == -1) + firstMatchIndex = i; + curMatchIndex = i; + return &shellCommands[i]; + } + } + + // not match + return NULL; +} + +// +// ------------------- print screen -------------------------- +// + +// delete char count +void deleteCount(Command * cmd, int count) { + int size = 0; + int width = 0; + clearScreen(cmd->endOffset + prompt_size, cmd->screenOffset + prompt_size); + + // loop delete + while (--count >= 0 && cmd->cursorOffset > 0) { + getPrevCharSize(cmd->command, cmd->cursorOffset, &size, &width); + memmove(cmd->command + cmd->cursorOffset - size, cmd->command + cmd->cursorOffset, + cmd->commandSize - cmd->cursorOffset); + cmd->commandSize -= size; + cmd->cursorOffset -= size; + cmd->screenOffset -= width; + cmd->endOffset -= width; + } +} + +// show screen +void printScreen(TAOS * con, Command * cmd, SWords * match) { + // modify Command + if (firstMatchIndex == -1 || curMatchIndex == -1) { + // no match + return ; + } + + // first tab press + const char * str = NULL; + int strLen = 0; + + if (firstMatchIndex == curMatchIndex && lastWordBytes == -1) { + // first press tab + SWord * word = MATCH_WORD(match); + str = word->word + match->matchLen; + strLen = word->len - match->matchLen; + lastMatchIndex = firstMatchIndex; + lastWordBytes = word->len; + } else { + if (lastWordBytes == -1) + return ; + deleteCount(cmd, lastWordBytes); + + SWord * word = MATCH_WORD(match); + str = word->word; + strLen = word->len; + // set current to last + lastMatchIndex = curMatchIndex; + lastWordBytes = word->len; + } + + // insert new + insertChar(cmd, (char *)str, strLen); +} + + +// main key press tab , matched return true else false +bool firstMatchCommand(TAOS * con, Command * cmd) { + // parse command + SWords* input = (SWords *)malloc(sizeof(SWords)); + memset(input, 0, sizeof(SWords)); + input->source = cmd->command; + input->source_len = cmd->commandSize; + parseCommand(input, false); + + // if have many , default match first, if press tab again , switch to next + curMatchIndex = -1; + lastMatchIndex = -1; + SWords * match = matchCommand(input, true); + if (match == NULL) { + // not match , nothing to do + freeCommand(input); + free(input); + return false; + } + + // print to screen + printScreen(con, cmd, match); + freeCommand(input); + free(input); + return true; +} + +// create input source +void createInputFromFirst(SWords* input, SWords * firstMatch) { + // + // if next pressTabKey , input context come from firstMatch, set matched length with source_len + // + input->source = (char*)malloc(1024); + memset((void* )input->source, 0, 1024); + + SWord * word = firstMatch->head; + + // source_len = full match word->len + half match with firstMatch->matchLen + for (int i = 0; i < firstMatch->matchIndex && word; i++) { + // combine source from each word + strncpy(input->source + input->source_len, word->word, word->len); + strcat(input->source, " "); // append blank splite + input->source_len += word->len + 1; // 1 is blank length + // move next + word = word->next; + } + // appand half matched word for last + if (word) { + strncpy(input->source + input->source_len, word->word, firstMatch->matchLen); + input->source_len += firstMatch->matchLen; + } +} + +// user press Tabkey again is named next , matched return true else false +bool nextMatchCommand(TAOS * con, Command * cmd, SWords * firstMatch) { + if (firstMatch == NULL || firstMatch->head == NULL) { + return false; + } + SWords* input = (SWords *)malloc(sizeof(SWords)); + memset(input, 0, sizeof(SWords)); + + // create input from firstMatch + createInputFromFirst(input, firstMatch); + + // parse input + parseCommand(input, false); + + // if have many , default match first, if press tab again , switch to next + SWords * match = matchCommand(input, true); + if (match == NULL) { + // if not match , reset all index + firstMatchIndex = -1; + curMatchIndex = -1; + match = matchCommand(input, false); + if (match == NULL) { + freeCommand(input); + if (input->source) + free(input->source); + free(input); + return false; + } + } + + // print to screen + printScreen(con, cmd, match); + + // free + if (input->source) { + free(input->source); + input->source = NULL; + } + freeCommand(input); + free(input); + + return true; +} + +// fill with type +bool fillWithType(TAOS * con, Command * cmd, char* pre, int type) { + // get type + STire* tire = tires[type]; + char* str = matchNextPrefix(tire, pre); + if (str == NULL) { + return false; + } + + // need insert part string + char * part = str + strlen(pre); + + // show + int count = strlen(part); + insertChar(cmd, part, count); + cntDel = count; // next press tab delete current append count + + free(str); + return true; +} + +// fill with type +bool fillTableName(TAOS * con, Command * cmd, char* pre) { + // search stable and table + char * str = tireSearchWord(WT_VAR_STABLE, pre); + if (str == NULL) { + str = tireSearchWord(WT_VAR_TABLE, pre); + if(str == NULL) + return false; + } + + // need insert part string + char * part = str + strlen(pre); + + // delete autofill count last append + if(cntDel > 0) { + deleteCount(cmd, cntDel); + cntDel = 0; + } + + // show + int count = strlen(part); + insertChar(cmd, part, count); + cntDel = count; // next press tab delete current append count + + free(str); + return true; +} + +// +// find last word from sql select clause +// example : +// 1 select cou -> press tab select count( +// 2 select count(*),su -> select count(*), sum( +// 3 select count(*), su -> select count(*), sum( +// +char * lastWord(char * p) { + // get near from end revert find ' ' and ',' + char * p1 = strrchr(p, ' '); + char * p2 = strrchr(p, ','); + + if (p1 && p2) { + return MAX(p1, p2) + 1; + } else if (p1) { + return p1 + 1; + } else if(p2) { + return p2 + 1; + } else { + return p; + } +} + +bool fieldsInputEnd(char* sql) { + // not in '()' + char* p1 = strrchr(sql, '('); + char* p2 = strrchr(sql, ')'); + if (p1 && p2 == NULL) { + // like select count( ' ' + return false; + } else if (p1 && p2 && p1 > p2) { + // like select sum(age), count( ' ' + return false; + } + + // not in ',' + char * p3 = strrchr(sql, ','); + char * p = p3; + // like select ts, age,' ' + if (p) { + ++p; + bool allBlank = true; // after last ',' all char is blank + int cnt = 0; // blank count , like ' ' as one blank + char * plast = NULL; // last blank position + while(*p) { + if (*p == ' ') { + plast = p; + cnt ++; + } else { + allBlank = false; + } + ++p; + } + + // any one word is not blank + if(allBlank) { + return false; + } + + // like 'select count(*),sum(age) fr' need return true + if (plast && plast > p3 && p2 > p1 && plast > p2 && p1 > p3) { + return true; + } + + // if last char not ' ', then not end field, like 'select count(*), su' can fill sum( + if(sql[strlen(sql)-1] != ' ' && cnt <= 1) { + return false; + } + } + + char * p4 = strrchr(sql, ' '); + if(p4 == NULL) { + // only one word + return false; + } + + return true; +} + +// need insert from +bool needInsertFrom(char * sql, int len) { + // last is blank + if(sql[len-1] != ' ') { + // insert from keyword + return false; + } + + // select fields input is end + if (!fieldsInputEnd(sql)) { + return false; + } + + // can insert from keyword + return true; +} + +bool matchSelectQuery(TAOS * con, Command * cmd) { + // if continue press Tab , delete bytes by previous autofill + if (cntDel > 0) { + deleteCount(cmd, cntDel); + cntDel = 0; + } + + // match select ... + int len = cmd->commandSize; + char * p = cmd->command; + + // remove prefix blank + while (p[0] == ' ' && len > 0) { + p++; + len--; + } + + // special range + if(len < 7 || len > 512) { + return false; + } + + // select and from + if(strncasecmp(p, "select ", 7) != 0) { + // not select query clause + return false; + } + p += 7; + len -= 7; + + char* ps = p = strndup(p, len); + + // union all + char * p1; + do { + p1 = strstr(p, UNION_ALL); + if(p1) { + p = p1 + strlen(UNION_ALL); + } + } while (p1); + + char * from = strstr(p, " from "); + //last word , maybe empty string or some letters of a string + char * last = lastWord(p); + bool ret = false; + if (from == NULL) { + bool fieldEnd = fieldsInputEnd(p); + // cheeck fields input end then insert from keyword + if (fieldEnd && p[len-1] == ' ') { + insertChar(cmd, "from", 4); + free(ps); + return true; + } + + // fill funciton + if(fieldEnd) { + // fields is end , need match keyword + ret = fillWithType(con, cmd, last, WT_VAR_KEYWORD); + } else { + ret = fillWithType(con, cmd, last, WT_VAR_FUNC); + } + + free(ps); + return ret; + } + + // have from + char * blank = strstr(from + 6, " "); + if (blank == NULL) { + // no table name, need fill + ret = fillTableName(con, cmd, last); + } else { + ret = fillWithType(con, cmd, last, WT_VAR_KEYWORD); + } + + free(ps); + return ret; +} + +// if is input create fields or tags area, return true +bool isCreateFieldsArea(char * p) { + char * left = strrchr(p, '('); + if (left == NULL) { + // like 'create table st' + return false; + } + + char * right = strrchr(p, ')'); + if(right == NULL) { + // like 'create table st( ' + return true; + } + + if (left > right) { + // like 'create table st( ts timestamp, age int) tags(area ' + return true; + } + + return false; +} + +bool matchCreateTable(TAOS * con, Command * cmd) { + // if continue press Tab , delete bytes by previous autofill + if (cntDel > 0) { + deleteCount(cmd, cntDel); + cntDel = 0; + } + + // match select ... + int len = cmd->commandSize; + char * p = cmd->command; + + // remove prefix blank + while (p[0] == ' ' && len > 0) { + p++; + len--; + } + + // special range + if(len < 7 || len > 1024) { + return false; + } + + // select and from + if(strncasecmp(p, "create table ", 13) != 0) { + // not select query clause + return false; + } + p += 13; + len -= 13; + + char* ps = strndup(p, len); + bool ret = false; + char * last = lastWord(ps); + + // check in create fields or tags input area + if (isCreateFieldsArea(ps)) { + ret = fillWithType(con, cmd, last, WT_VAR_DATATYPE); + } + + // tags + if (!ret) { + // find only one ')' , can insert tags + char * p1 = strchr(ps, ')'); + if (p1) { + if(strchr(p1 + 1, ')') == NULL && strstr(p1 + 1, "tags") == NULL) { + // can insert tags keyword + ret = fillWithType(con, cmd, last, WT_VAR_KEYTAGS); + } + } + } + + free(ps); + return ret; +} + +bool matchOther(TAOS * con, Command * cmd) { + int len = cmd->commandSize; + char* p = cmd->command; + + if (p[len - 1] == '\\') { + // append '\G' + char a[] = "G;"; + insertChar(cmd, a, 2); + return true; + } + + return false; +} + + +// main key press tab +void pressTabKey(TAOS * con, Command * cmd) { + // check + if (cmd->commandSize == 0) { + // empty + showHelp(); + showOnScreen(cmd); + return ; + } + + // save connection to global + varCon = con; + varCmd = cmd; + bool matched = false; + + // manual match like create table st( ... + matched = matchCreateTable(con, cmd); + if (matched) + return ; + + // shellCommands match + if (firstMatchIndex == -1) { + matched = firstMatchCommand(con, cmd); + } else { + matched = nextMatchCommand(con, cmd, &shellCommands[firstMatchIndex]); + } + if (matched) + return ; + + // NOT MATCHED ANYONE + // match other like '\G' ... + matched = matchOther(con, cmd); + if (matched) + return ; + + // manual match like select * from ... + matched = matchSelectQuery(con, cmd); + if (matched) + return ; + + return ; +} + +// press othr key +void pressOtherKey(char c) { + // reset global variant + firstMatchIndex = -1; + lastMatchIndex = -1; + curMatchIndex = -1; + lastWordBytes = -1; + + // var names + cursorVar = -1; + varMode = false; + waitAutoFill = false; + cntDel = 0; + + if (lastMatch) { + freeMatch(lastMatch); + lastMatch = NULL; + } +} + +// put name into name, return name length +int getWordName(char* p, char * name, int nameLen) { + //remove prefix blank + while (*p == ' ') { + p++; + } + + // get databases name; + int i = 0; + while(p[i] != 0 && i < nameLen - 1) { + name[i] = p[i]; + i++; + if(p[i] == ' ' || p[i] == ';'|| p[i] == '(') { + // name end + break; + } + } + name[i] = 0; + + return i; +} + +// deal use db, if have 'use' return true +bool dealUseDB(char * sql) { + // check use keyword + if(strncasecmp(sql, "use ", 4) != 0) { + return false; + } + + char db[256]; + char *p = sql + 4; + if (getWordName(p, db, sizeof(db)) == 0) { + // no name , return + return true; + } + + // dbName is previous use open db name + if (strcasecmp(db, dbName) == 0) { + // same , no need switch + return true; + } + + // switch new db + pthread_mutex_lock(&tiresMutex); + // STABLE set null + STire* tire = tires[WT_VAR_STABLE]; + tires[WT_VAR_STABLE] = NULL; + if(tire) { + freeTire(tire); + } + // TABLE set null + tire = tires[WT_VAR_TABLE]; + tires[WT_VAR_TABLE] = NULL; + if(tire) { + freeTire(tire); + } + // save + strcpy(dbName, db); + pthread_mutex_unlock(&tiresMutex); + + return true; +} + +// deal create, if have 'create' return true +bool dealCreateCommand(char * sql) { + // check keyword + if(strncasecmp(sql, "create ", 7) != 0) { + return false; + } + + char name[1024]; + char *p = sql + 7; + if (getWordName(p, name, sizeof(name)) == 0) { + // no name , return + return true; + } + + int type = -1; + // dbName is previous use open db name + if (strcasecmp(name, "database") == 0) { + type = WT_VAR_DBNAME; + } else if (strcasecmp(name, "table") == 0) { + if(strstr(sql, " tags") != NULL && strstr(sql, " using ") == NULL) + type = WT_VAR_STABLE; + else + type = WT_VAR_TABLE; + } else if (strcasecmp(name, "user") == 0) { + type = WT_VAR_USERNAME; + } else { + // no match , return + return true; + } + + // move next + p += strlen(name); + + // get next word , that is table name + if (getWordName(p, name, sizeof(name)) == 0) { + // no name , return + return true; + } + + // switch new db + pthread_mutex_lock(&tiresMutex); + // STABLE set null + STire* tire = tires[type]; + if(tire) { + insertWord(tire, name); + } + pthread_mutex_unlock(&tiresMutex); + + return true; +} + +// deal create, if have 'drop' return true +bool dealDropCommand(char * sql) { + // check keyword + if(strncasecmp(sql, "drop ", 5) != 0) { + return false; + } + + char name[1024]; + char *p = sql + 5; + if (getWordName(p, name, sizeof(name)) == 0) { + // no name , return + return true; + } + + int type = -1; + // dbName is previous use open db name + if (strcasecmp(name, "database") == 0) { + type = WT_VAR_DBNAME; + } else if (strcasecmp(name, "table") == 0) { + type = WT_VAR_ALLTABLE; + } else if (strcasecmp(name, "dnode") == 0) { + type = WT_VAR_DNODEID; + } else if (strcasecmp(name, "user") == 0) { + type = WT_VAR_USERNAME; + } else { + // no match , return + return true; + } + + // move next + p += strlen(name); + + // get next word , that is table name + if (getWordName(p, name, sizeof(name)) == 0) { + // no name , return + return true; + } + + // switch new db + pthread_mutex_lock(&tiresMutex); + // STABLE set null + if(type == WT_VAR_ALLTABLE) { + bool del = false; + // del in stable + STire* tire = tires[WT_VAR_STABLE]; + if(tire) + del = deleteWord(tire, name); + // del in table + if(!del) { + tire = tires[WT_VAR_TABLE]; + if(tire) + del = deleteWord(tire, name); + } + } else { + // OTHER TYPE + STire* tire = tires[type]; + if(tire) + deleteWord(tire, name); + } + pthread_mutex_unlock(&tiresMutex); + + return true; +} + +// callback autotab module after shell sql execute +void callbackAutoTab(char* sqlstr, TAOS* pSql, bool usedb) { + char * sql = sqlstr; + // remove prefix blank + while (*sql == ' ') { + sql++; + } + + if(dealUseDB(sql)) { + // change to new db + return ; + } + + // create command add name to autotab + if(dealCreateCommand(sql)) { + return ; + } + + // drop command remove name from autotab + if(dealDropCommand(sql)) { + return ; + } + + return ; +} diff --git a/src/kit/shell/src/shellCommand.c b/src/kit/shell/src/shellCommand.c index d78e152dbdbc5c0144c65d50a32daadbce1cf534..2fe09691e3285c2e3031672404b0aa6ed7bac244 100644 --- a/src/kit/shell/src/shellCommand.c +++ b/src/kit/shell/src/shellCommand.c @@ -79,8 +79,11 @@ void insertChar(Command *cmd, char *c, int size) { /* update the values */ cmd->commandSize += size; cmd->cursorOffset += size; - cmd->screenOffset += wcwidth(wc); - cmd->endOffset += wcwidth(wc); + for (int i = 0; i < size; i++) { + mbtowc(&wc, c + i, size); + cmd->screenOffset += wcwidth(wc); + cmd->endOffset += wcwidth(wc); + } showOnScreen(cmd); } @@ -179,6 +182,16 @@ void positionCursorHome(Command *cmd) { } } +void positionCursorMiddle(Command *cmd) { + if (cmd->endOffset > 0) { + clearScreen(cmd->endOffset + prompt_size, cmd->screenOffset + prompt_size); + cmd->cursorOffset = cmd->commandSize/2; + cmd->screenOffset = cmd->endOffset/2; + showOnScreen(cmd); + } +} + + void positionCursorEnd(Command *cmd) { assert(cmd->cursorOffset <= cmd->commandSize && cmd->endOffset >= cmd->screenOffset); diff --git a/src/kit/shell/src/shellDarwin.c b/src/kit/shell/src/shellDarwin.c index 7803113a0f98f0152c0ab6da948252064c33c70d..0108d92e8f6a73317bc510857e17d8c81a62d512 100644 --- a/src/kit/shell/src/shellDarwin.c +++ b/src/kit/shell/src/shellDarwin.c @@ -22,6 +22,7 @@ #include "tkey.h" #include "tscLog.h" +#include "shellAuto.h" #define OPT_ABORT 1 /* �Cabort */ @@ -255,7 +256,12 @@ int32_t shellReadCommand(TAOS *con, char *command) { utf8_array[k] = c; } insertChar(&cmd, utf8_array, count); + pressOtherKey(c); + } else if (c == TAB_KEY) { + // press TAB key + pressTabKey(con, &cmd); } else if (c < '\033') { + pressOtherKey(c); // Ctrl keys. TODO: Implement ctrl combinations switch (c) { case 1: // ctrl A @@ -301,6 +307,9 @@ int32_t shellReadCommand(TAOS *con, char *command) { case 21: // Ctrl + U clearLineBefore(&cmd); break; + case 23: // Ctrl + W; + positionCursorMiddle(&cmd); + break; } } else if (c == '\033') { c = getchar(); @@ -377,9 +386,11 @@ int32_t shellReadCommand(TAOS *con, char *command) { break; } } else if (c == 0x7f) { + pressOtherKey(c); // press delete key backspaceChar(&cmd); } else { + pressOtherKey(c); insertChar(&cmd, &c, 1); } } diff --git a/src/kit/shell/src/shellEngine.c b/src/kit/shell/src/shellEngine.c index e1e443413a4b64473b5dcbd71eaa452de2994d57..b0c3f4934d2d7180413b2e5a3463b8eb9da8428a 100644 --- a/src/kit/shell/src/shellEngine.c +++ b/src/kit/shell/src/shellEngine.c @@ -27,6 +27,7 @@ #include "tglobal.h" #include "tsclient.h" #include "cJSON.h" +#include "shellAuto.h" #include @@ -327,6 +328,12 @@ void shellRunCommandOnServer(TAOS *con, char command[]) { fprintf(stdout, "Database changed.\n\n"); fflush(stdout); +#ifndef WINDOWS + // call back auto tab module + callbackAutoTab(command, pSql, true); +#endif + + atomic_store_64(&result, 0); freeResultWithRid(oresult); return; @@ -365,6 +372,11 @@ void shellRunCommandOnServer(TAOS *con, char command[]) { int num_rows_affacted = taos_affected_rows(pSql); et = taosGetTimestampUs(); printf("Query OK, %d of %d row(s) in database (%.6fs)\n", num_rows_affacted, num_rows_affacted, (et - st) / 1E6); + +#ifndef WINDOWS + // call auto tab + callbackAutoTab(command, pSql, false); +#endif } printf("\n"); diff --git a/src/kit/shell/src/shellLinux.c b/src/kit/shell/src/shellLinux.c index 863da2e1a727f073fe5eabf2b9d8d17f4e05c4b4..fd24a61c7d55c8a91ac9631a7263ca44b81d1606 100644 --- a/src/kit/shell/src/shellLinux.c +++ b/src/kit/shell/src/shellLinux.c @@ -20,6 +20,7 @@ #include "shellCommand.h" #include "tkey.h" #include "tulog.h" +#include "shellAuto.h" #define OPT_ABORT 1 /* �Cabort */ @@ -283,7 +284,12 @@ int32_t shellReadCommand(TAOS *con, char *command) { utf8_array[k] = c; } insertChar(&cmd, utf8_array, count); + pressOtherKey(c); + } else if (c == TAB_KEY) { + // press TAB key + pressTabKey(con, &cmd); } else if (c < '\033') { + pressOtherKey(c); // Ctrl keys. TODO: Implement ctrl combinations switch (c) { case 1: // ctrl A @@ -329,8 +335,12 @@ int32_t shellReadCommand(TAOS *con, char *command) { case 21: // Ctrl + U; clearLineBefore(&cmd); break; + case 23: // Ctrl + W; + positionCursorMiddle(&cmd); + break; } } else if (c == '\033') { + pressOtherKey(c); c = (char)getchar(); switch (c) { case '[': @@ -405,9 +415,11 @@ int32_t shellReadCommand(TAOS *con, char *command) { break; } } else if (c == 0x7f) { + pressOtherKey(c); // press delete key backspaceChar(&cmd); } else { + pressOtherKey(c); insertChar(&cmd, &c, 1); } } diff --git a/src/kit/shell/src/shellMain.c b/src/kit/shell/src/shellMain.c index 149afc503e0f21b0bd347d9452a1811567bc5221..2dcf1052168e0de384ed666fa2f7c0044f5f618b 100644 --- a/src/kit/shell/src/shellMain.c +++ b/src/kit/shell/src/shellMain.c @@ -17,6 +17,8 @@ #include "shell.h" #include "tconfig.h" #include "tnettest.h" +#include "shellCommand.h" +#include "shellAuto.h" pthread_t pid; static tsem_t cancelSem; @@ -162,10 +164,16 @@ int main(int argc, char* argv[]) { /* Get grant information */ shellGetGrantInfo(args.con); +#ifndef WINDOWS + shellAutoInit(); +#endif /* Loop to query the input. */ while (1) { pthread_create(&pid, NULL, shellLoopQuery, args.con); pthread_join(pid, NULL); } +#ifndef WINDOWS + shellAutoExit(); +#endif } diff --git a/src/kit/shell/src/tire.c b/src/kit/shell/src/tire.c new file mode 100644 index 0000000000000000000000000000000000000000..b4dc7976bd53f11cccbac2f5db600edeeee861d5 --- /dev/null +++ b/src/kit/shell/src/tire.c @@ -0,0 +1,435 @@ +/* + * Copyright (c) 2019 TAOS Data, Inc. + * + * This program is free software: you can use, redistribute, and/or modify + * it under the terms of the GNU Affero General Public License, version 3 + * or later ("AGPL"), as published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ + +#define __USE_XOPEN + +#include "os.h" +#include "tire.h" + +// ----------- interface ------------- + +// create prefix search tree +STire* createTire(char type) { + STire* tire = malloc(sizeof(STire)); + memset(tire, 0, sizeof(STire)); + tire->ref = 1; // init is 1 + tire->type = type; + tire->root.d = (STireNode **)calloc(CHAR_CNT, sizeof(STireNode *)); + return tire; +} + +// free tire node +void freeTireNode(STireNode* node) { + if (node == NULL) + return ; + + // nest free sub node on array d + if(node->d) { + for (int i = 0; i < CHAR_CNT; i++) { + freeTireNode(node->d[i]); + } + tfree(node->d); + } + + // free self + tfree(node); +} + +// destroy prefix search tree +void freeTire(STire* tire) { + // free nodes + for (int i = 0; i < CHAR_CNT; i++) { + freeTireNode(tire->root.d[i]); + } + tfree(tire->root.d); + + // free from list + StrName * item = tire->head; + while (item) { + StrName * next = item->next; + // free string + tfree(item->name); + // free node + tfree(item); + + // move next + item = next; + } + tire->head = tire->tail = NULL; + + // free tire + tfree(tire); +} + +// insert a new word to list +bool insertToList(STire* tire, char* word) { + StrName * p = (StrName *)malloc(sizeof(StrName)); + p->name = strdup(word); + p->next = NULL; + + if(tire->head == NULL) { + tire->head = p; + tire->tail = p; + }else { + tire->tail->next = p; + tire->tail = p; + } + + return true; +} + +// insert a new word to tree +bool insertToTree(STire* tire, char* word, int len) { + int m = 0; + STireNode ** nodes = tire->root.d; + for (int i = 0; i < len; i++) { + m = word[i] - FIRST_ASCII; + if (m < 0 || m > CHAR_CNT) { + return false; + } + + if (nodes[m] == NULL) { + // no pointer + STireNode* p = (STireNode* )tmalloc(sizeof(STireNode)); + memset(p, 0, sizeof(STireNode)); + nodes[m] = p; + if (i == len - 1) { + // is end + p->end = true; + break; + } + } + + if (nodes[m]->d == NULL) { + // malloc d + nodes[m]->d = (STireNode **)calloc(CHAR_CNT, sizeof(STireNode *)); + } + + // move to next node + nodes = nodes[m]->d; + } + + // add count + tire->count += 1; + return true; +} + +// insert a new word +bool insertWord(STire* tire, char* word) { + int len = strlen(word); + if (len >= MAX_WORD_LEN) { + return false; + } + + switch (tire->type) { + case TIRE_TREE: + return insertToTree(tire, word, len); + case TIRE_LIST: + return insertToList(tire, word); + default: + break; + } + return false; +} + +// delete one word from list +bool deleteFromList(STire* tire, char* word) { + StrName * item = tire->head; + while (item) { + if (strcmp(item->name, word) == 0) { + // found, reset empty to delete + item->name[0] = 0; + } + + // move next + item = item->next; + } + return true; +} + +// delete one word from tree +bool deleteFromTree(STire* tire, char* word, int len) { + int m = 0; + bool del = false; + + STireNode** nodes = tire->root.d; + for (int i = 0; i < len; i++) { + m = word[i] - FIRST_ASCII; + if (m < 0 || m >= CHAR_CNT) { + return false; + } + + if (nodes[m] == NULL) { + // no found + return false; + } else { + // not null + if(i == len - 1) { + // this is last, only set end false , not free node + nodes[m]->end = false; + del = true; + break; + } + } + + if(nodes[m]->d == NULL) + break; + // move to next node + nodes = nodes[m]->d; + } + + // reduce count + if (del) { + tire->count -= 1; + } + + return del; +} + +// insert a new word +bool deleteWord(STire* tire, char* word) { + int len = strlen(word); + if (len >= MAX_WORD_LEN) { + return false; + } + + switch (tire->type) { + case TIRE_TREE: + return deleteFromTree(tire, word, len); + case TIRE_LIST: + return deleteFromList(tire, word); + default: + break; + } + return false; +} + +void addWordToMatch(SMatch* match, char* word){ + // malloc new + SMatchNode* node = (SMatchNode* )tmalloc(sizeof(SMatchNode)); + memset(node, 0, sizeof(SMatchNode)); + node->word = strdup(word); + + // append to match + if (match->head == NULL) { + match->head = match->tail = node; + } else { + match->tail->next = node; + match->tail = node; + } + match->count += 1; +} + +// enum all words from node +void enumAllWords(STireNode** nodes, char* prefix, SMatch* match) { + STireNode * c; + char word[MAX_WORD_LEN]; + int len = strlen(prefix); + for (int i = 0; i < CHAR_CNT; i++) { + c = nodes[i]; + + if (c == NULL) { + // chain end node + continue; + } else { + // combine word string + memset(word, 0, sizeof(word)); + strcpy(word, prefix); + word[len] = FIRST_ASCII + i; // append current char + + // chain middle node + if (c->end) { + // have end flag + addWordToMatch(match, word); + } + // nested call next layer + if (c->d) + enumAllWords(c->d, word, match); + } + } +} + +// match prefix from list +void matchPrefixFromList(STire* tire, char* prefix, SMatch* match) { + StrName * item = tire->head; + int len = strlen(prefix); + while (item) { + if ( strncmp(item->name, prefix, len) == 0) { + // prefix matched + addWordToMatch(match, item->name); + } + + // move next + item = item->next; + } +} + +// match prefix words, if match is not NULL , put all item to match and return match +void matchPrefixFromTree(STire* tire, char* prefix, SMatch* match) { + SMatch* root = match; + int m = 0; + STireNode* c = 0; + int len = strlen(prefix); + if (len >= MAX_WORD_LEN) { + return; + } + + STireNode** nodes = tire->root.d; + for (int i = 0; i < len; i++) { + m = prefix[i] - FIRST_ASCII; + if (m < 0 || m > CHAR_CNT) { + return; + } + + // match + c = nodes[m]; + if (c == NULL) { + // arrive end + break; + } + + // previous items already matched + if (i == len - 1) { + // malloc match if not pass by param match + if (root == NULL) { + root = (SMatch* )tmalloc(sizeof(SMatch)); + memset(root, 0, sizeof(SMatch)); + strcpy(root->pre, prefix); + } + + // prefix is match to end char + if (c->d) + enumAllWords(c->d, prefix, root); + } else { + // move to next node continue match + if(c->d == NULL) + break; + nodes = c->d; + } + } + + // return + return ; +} + +SMatch* matchPrefix(STire* tire, char* prefix, SMatch* match) { + if(match == NULL) { + match = (SMatch* )tmalloc(sizeof(SMatch)); + memset(match, 0, sizeof(SMatch)); + } + + switch (tire->type) { + case TIRE_TREE: + matchPrefixFromTree(tire, prefix, match); + case TIRE_LIST: + matchPrefixFromList(tire, prefix, match); + default: + break; + } + + // return if need + if (match->count == 0) { + freeMatch(match); + match = NULL; + } + + return match; +} + + +// get all items from tires tree +void enumFromList(STire* tire, SMatch* match) { + StrName * item = tire->head; + while (item) { + if (item->name[0] != 0) { + // not delete + addWordToMatch(match, item->name); + } + + // move next + item = item->next; + } +} + +// get all items from tires tree +void enumFromTree(STire* tire, SMatch* match) { + char pre[2] ={0, 0}; + STireNode* c; + + // enum first layer + for (int i = 0; i < CHAR_CNT; i++) { + pre[0] = FIRST_ASCII + i; + + // each node + c = tire->root.d[i]; + if (c == NULL) { + // this branch no data + continue; + } + + // this branch have data + if(c->end) + addWordToMatch(match, pre); + else + matchPrefix(tire, pre, match); + } +} + +// get all items from tires tree +SMatch* enumAll(STire* tire) { + SMatch* match = (SMatch* )tmalloc(sizeof(SMatch)); + memset(match, 0, sizeof(SMatch)); + + switch (tire->type) { + case TIRE_TREE: + enumFromTree(tire, match); + case TIRE_LIST: + enumFromList(tire, match); + default: + break; + } + + // return if need + if (match->count == 0) { + freeMatch(match); + match = NULL; + } + + return match; +} + + +// free match result +void freeMatchNode(SMatchNode* node) { + // first free next + if (node->next) + freeMatchNode(node->next); + + // second free self + if (node->word) + free(node->word); + free(node); +} + +// free match result +void freeMatch(SMatch* match) { + // first free next + if (match->head) { + freeMatchNode(match->head); + } + + // second free self + free(match); +} diff --git a/src/tsdb/src/tsdbMain.c b/src/tsdb/src/tsdbMain.c index 5392fc075b7389f077ff39416e4c96c881338384..63ea4ab6df0e84e81a9308de509ccd24c933c54e 100644 --- a/src/tsdb/src/tsdbMain.c +++ b/src/tsdb/src/tsdbMain.c @@ -128,7 +128,7 @@ int tsdbCloseRepo(STsdbRepo *repo, int toCommit) { tsdbStopStream(pRepo); if(pRepo->pthread){ - taosDestoryThread(pRepo->pthread); + taosDestroyThread(pRepo->pthread); pRepo->pthread = NULL; } diff --git a/src/tsdb/src/tsdbRead.c b/src/tsdb/src/tsdbRead.c index c2fd51414bb0705e24783ad947666009ba521e66..3d72a7bde3bb87eca567819503ceb9746968768b 100644 --- a/src/tsdb/src/tsdbRead.c +++ b/src/tsdb/src/tsdbRead.c @@ -986,7 +986,9 @@ static SMemRow getSMemRowInTableMem(STableCheckInfo* pCheckInfo, int32_t order, return rmem; } else { pCheckInfo->chosen = CHECKINFO_CHOSEN_BOTH; - *extraRow = rimem; + if (extraRow) { + *extraRow = rimem; + } return rmem; } } else { diff --git a/src/util/inc/tthread.h b/src/util/inc/tthread.h index 7443ad706dcbef529d857fe823cddd0cc1efbdd3..9ef1c230359c154d54f7c577a3387cea0d57c551 100644 --- a/src/util/inc/tthread.h +++ b/src/util/inc/tthread.h @@ -26,7 +26,7 @@ extern "C" { // create new thread pthread_t* taosCreateThread( void *(*__start_routine) (void *), void* param); // destory thread -bool taosDestoryThread(pthread_t* pthread); +bool taosDestroyThread(pthread_t* pthread); // thread running return true bool taosThreadRunning(pthread_t* pthread); diff --git a/src/util/src/tthread.c b/src/util/src/tthread.c index 043b2de2f241297d209041294428dde2c55e974e..f77dea592e8454dcc15e05f5c03c9db56e0ccc6b 100644 --- a/src/util/src/tthread.c +++ b/src/util/src/tthread.c @@ -38,7 +38,7 @@ pthread_t* taosCreateThread( void *(*__start_routine) (void *), void* param) { } // destory thread -bool taosDestoryThread(pthread_t* pthread) { +bool taosDestroyThread(pthread_t* pthread) { if(pthread == NULL) return false; if(taosThreadRunning(pthread)) { pthread_cancel(*pthread); diff --git a/src/vnode/src/vnodeMain.c b/src/vnode/src/vnodeMain.c index 23181981e0dd630b4eac71538a3072bab347380a..f215453f740b979e5b71a4d59a2698b6dd569ff7 100644 --- a/src/vnode/src/vnodeMain.c +++ b/src/vnode/src/vnodeMain.c @@ -461,7 +461,7 @@ void vnodeStopWaitingThread(SVnodeObj* pVnode) { if(loop == 0) { vInfo("vgId:%d :SDEL force kill thread to quit. pthread=%p pWrite=%p", pVnode->vgId, pWaitThread->pthread, pWaitThread->param); // thread not stop , so need kill - taosDestoryThread(pWaitThread->pthread); + taosDestroyThread(pWaitThread->pthread); // write msg need remove from queue SVWriteMsg* pWrite = (SVWriteMsg* )pWaitThread->param; if (pWrite) diff --git a/tests/parallel_test/cases.task b/tests/parallel_test/cases.task index 36c57cdf9597b03bdd875e8c56260395333ce5b8..c2c462324e0cdc02b248ac704064d7c2ec76dfeb 100644 --- a/tests/parallel_test/cases.task +++ b/tests/parallel_test/cases.task @@ -1,4 +1,5 @@ # 20,,pytest,python3 insert/retentionpolicy.py change date time +500,,docs-examples-test,./test_node.sh 299,,pytest,python3 test.py -f update/merge_commit_data-0.py 290,,pytest,python3 test.py -f update/merge_commit_data.py 241,,pytest,python3 test.py -f update/merge_commit_data2.py @@ -593,7 +594,6 @@ 8,,develop-test,python3 ./test.py -f 5-taos-tools/taosdump/taosdumpTestTypeInt.py 8,,develop-test,python3 ./test.py -f 5-taos-tools/taosdump/taosdumpTestTypeDouble.py 8,,pytest,python3 test.py -f update/update2.py -7,,docs-examples-test,./test_node.sh 7,,system-test,python3 ./test.py -f 5-taos-tools/taosbenchmark/taosdemoTestInsertWithJsonSml-otherPara.py 7,,pytest,python3 test.py -f tools/taosdumpTest2.py 7,,pytest,python3 test.py -f tools/taosdemoTestdatatype.py diff --git a/tests/pytest/table/tagNameCaseSensitive.py b/tests/pytest/table/tagNameCaseSensitive.py index ebd7f55a2d569223d81f0fc6c92eb4e87424d8d0..c9ee64fa242484e64c9fa6cfdb6ed468436f2199 100644 --- a/tests/pytest/table/tagNameCaseSensitive.py +++ b/tests/pytest/table/tagNameCaseSensitive.py @@ -65,7 +65,7 @@ class TDTestCase: tdSql.query("select * from `STB6`") tdSql.checkRows(6) - tdSql.execute("delete from `STB6` where ` ` = 1 and ts = '2022-06-24 11:17:31.000'") + tdSql.execute("delete from `STB6` where ` ` = 1 and ts = 1656040651000") tdSql.checkAffectedRows(1) tdSql.query("select * from `STB6`") tdSql.checkRows(5) @@ -74,6 +74,10 @@ class TDTestCase: tdSql.query("select * from `STB6`") tdSql.checkRows(2) + tdSql.execute("alter table `STB6` add tag `1` int") + tdSql.execute("create table t1 using `STB6`(`1`) tags(1)") + tdSql.error("alter table t1 set tag 1=2222") + tdSql.error("alter table `STB6` add tag `` nchar(20)") def stop(self): @@ -82,4 +86,4 @@ class TDTestCase: tdCases.addWindows(__file__, TDTestCase()) -tdCases.addLinux(__file__, TDTestCase()) \ No newline at end of file +tdCases.addLinux(__file__, TDTestCase())