Skip to content

TDengine
TDengine

taosdata / TDengine
大约 2 年 前同步成功

通知 1193
Star 22018
Fork 4786
TDengine
TDengine
提交 f017c4dd 编写于 作者: sangshuduo's avatar sangshuduo
浏览文件
操作

Merge branch '2.6' into feat/sangshuduo/TD-14141-update-taostools-for2.6

上级 0eddae72 0b4e617d
隐藏空白更改
内联 并排
Showing with 2891 addition and 41 deletion
docs/en/07-develop/03-insert-data/05-high-volume.md 0 → 100644
浏览文件 @ f017c4dd
---
sidebar_label: High Performance Writing
title: High Performance Writing
---
import Tabs from "@theme/Tabs";
import TabItem from "@theme/TabItem";
This chapter introduces how to write data into TDengine with high throughput.
## How to achieve high performance data writing
To achieve high performance writing, there are a few aspects to consider. In the following sections we will describe these important factors in achieving high performance writing.
### Application Program
From the perspective of application program, you need to consider:
1. The data size of each single write, also known as batch size. Generally speaking, higher batch size generates better writing performance. However, once the batch size is over a specific value, you will not get any additional benefit anymore. When using SQL to write into TDengine, it's better to put as much as possible data in single SQL. The maximum SQL length supported by TDengine is 1,048,576 bytes, i.e. 1 MB. It can be configured by parameter `maxSQLLength` on client side, and the default value is 65,480.
2. The number of concurrent connections. Normally more connections can get better result. However, once the number of connections exceeds the processing ability of the server side, the performance may downgrade.
3. The distribution of data to be written across tables or sub-tables. Writing to single table in one batch is more efficient than writing to multiple tables in one batch.
4. Data Writing Protocol.
- Prameter binding mode is more efficient than SQL because it doesn't have the cost of parsing SQL.
- Writing to known existing tables is more efficient than wirting to uncertain tables in automatic creating mode because the later needs to check whether the table exists or not before actually writing data into it
- Writing in SQL is more efficient than writing in schemaless mode because schemaless writing creats table automatically and may alter table schema
Application programs need to take care of the above factors and try to take advantage of them. The application progam should write to single table in each write batch. The batch size needs to be tuned to a proper value on a specific system. The number of concurrent connections needs to be tuned to a proper value too to achieve the best writing throughput.
### Data Source
Application programs need to read data from data source then write into TDengine. If you meet one or more of below situations, you need to setup message queues between the threads for reading from data source and the threads for writing into TDengine.
1. There are multiple data sources, the data generation speed of each data source is much slower than the speed of single writing thread. In this case, the purpose of message queues is to consolidate the data from multiple data sources together to increase the batch size of single write.
2. The speed of data generation from single data source is much higher than the speed of single writing thread. The purpose of message queue in this case is to provide buffer so that data is not lost and multiple writing threads can get data from the buffer.
3. The data for single table are from multiple data source. In this case the purpose of message queues is to combine the data for single table together to improve the write efficiency.
If the data source is Kafka, then the appication program is a consumer of Kafka, you can benefit from some kafka features to achieve high performance writing:
1. Put the data for a table in single partition of single topic so that it's easier to put the data for each table together and write in batch
2. Subscribe multiple topics to accumulate data together.
3. Add more consumers to gain more concurrency and throughput.
4. Incrase the size of single fetch to increase the size of write batch.
### Tune TDengine
TDengine is a distributed and high performance time series database, there are also some ways to tune TDengine to get better writing performance.
1. Set proper number of `vgroups` according to available CPU cores. Normally, we recommend 2 \* number_of_cores as a starting point. If the verification result shows this is not enough to utilize CPU resources, you can use a higher value.
2. Set proper `minTablesPerVnode`, `tableIncStepPerVnode`, and `maxVgroupsPerDb` according to the number of tables so that tables are distributed even across vgroups. The purpose is to balance the workload among all vnodes so that system resources can be utilized better to get higher performance.
For more performance tuning tips, please refer to [Performance Optimization](../../operation/optimize) and [Configuration Parameters](../../reference/config).
## Sample Programs
This section will introduce the sample programs to demonstrate how to write into TDengine with high performance.
### Scenario
Below are the scenario for the sample programs of high performance wrting.
- Application program reads data from data source, the sample program simulates a data source by generating data
- The speed of single writing thread is much slower than the speed of generating data, so the program starts multiple writing threads while each thread establish a connection to TDengine and each thread has a message queue of fixed size.
- Application program maps the received data to different writing threads based on table name to make sure all the data for each table is always processed by a specific writing thread.
- Each writing thread writes the received data into TDengine once the message queue becomes empty or the read data meets a threshold.
![Thread Model of High Performance Writing into TDengine](highvolume.webp)
### Sample Programs
The sample programs listed in this section are based on the scenario described previously. If your scenarios is different, please try to adjust the code based on the principles described in this chapter.
The sample programs assume the source data is for all the different sub tables in same super table (meters). The super table has been created before the sample program starts to writing data. Sub tables are created automatically according to received data. If there are multiple super tables in your case, please try to adjust the part of creating table automatically.
<Tabs defaultValue="java" groupId="lang">
<TabItem label="Java" value="java">
**Program Inventory**
| Class | Description |
| ---------------- | ----------------------------------------------------------------------------------------------------- |
| FastWriteExample | Main Program |
| ReadTask | Read data from simulated data source and put into a queue according to the hash value of table name |
| WriteTask | Read data from Queue, compose a wirte batch and write into TDengine |
| MockDataSource | Generate data for some sub tables of super table meters |
| SQLWriter | WriteTask uses this class to compose SQL, create table automatically, check SQL length and write data |
| StmtWriter | Write in Parameter binding mode (Not finished yet) |
| DataBaseMonitor | Calculate the writing speed and output on console every 10 seconds |
Below is the list of complete code of the classes in above table and more detailed description.
<details>
<summary>FastWriteExample</summary>
The main Program is responsible for:
1. Create message queues
2. Start writing threads
3. Start reading threads
4. Otuput writing speed every 10 seconds
The main program provides 4 parameters for tuning：
1. The number of reading threads, default value is 1
2. The number of writing threads, default alue is 2
3. The total number of tables in the generated data, default value is 1000. These tables are distributed evenly across all writing threads. If the number of tables is very big, it will cost much time to firstly create these tables.
4. The batch size of single write, default value is 3,000
The capacity of message queue also impacts performance and can be tuned by modifying program. Normally it's always better to have a larger message queue. A larger message queue means lower possibility of being blocked when enqueueing and higher throughput. But a larger message queue consumes more memory space. The default value used in the sample programs is already big enoug.
```java
{{#include docs/examples/java/src/main/java/com/taos/example/highvolume/FastWriteExample.java}}
```
</details>
<details>
<summary>ReadTask</summary>
ReadTask reads data from data source. Each ReadTask is associated with a simulated data source, each data source generates data for a group of specific tables, and the data of any table is only generated from a single specific data source.
ReadTask puts data in message queue in blocking mode. That means, the putting operation is blocked if the message queue is full.
```java
{{#include docs/examples/java/src/main/java/com/taos/example/highvolume/ReadTask.java}}
```
</details>
<details>
<summary>WriteTask</summary>
```java
{{#include docs/examples/java/src/main/java/com/taos/example/highvolume/WriteTask.java}}
```
</details>
<details>
<summary>MockDataSource</summary>
```java
{{#include docs/examples/java/src/main/java/com/taos/example/highvolume/MockDataSource.java}}
```
</details>
<details>
<summary>SQLWriter</summary>
SQLWriter class encapsulates the logic of composing SQL and writing data. Please be noted that the tables have not been created before writing, but are created automatically when catching the exception of table doesn't exist. For other exceptions caught, the SQL which caused the exception are logged for you to debug.
```java
{{#include docs/examples/java/src/main/java/com/taos/example/highvolume/SQLWriter.java}}
```
</details>
<details>
<summary>DataBaseMonitor</summary>
```java
{{#include docs/examples/java/src/main/java/com/taos/example/highvolume/DataBaseMonitor.java}}
```
</details>
**Steps to Launch**
<details>
<summary>Launch Java Sample Program</summary>
You need to set environment variable `TDENGINE_JDBC_URL` before launching the program. If TDengine Server is setup on localhost, then the default value for user name, password and port can be used, like below:
```
TDENGINE_JDBC_URL="jdbc:TAOS://localhost:6030?user=root&password=taosdata"
```
**Launch in IDE**
1. Clone TDengine repolitory
```
git clone git@github.com:taosdata/TDengine.git --depth 1
```
2. Use IDE to open `docs/examples/java` directory
3. Configure environment variable `TDENGINE_JDBC_URL`, you can also configure it before launching the IDE, if so you can skip this step.
4. Run class `com.taos.example.highvolume.FastWriteExample`
**Launch on server**
If you want to launch the sample program on a remote server, please follow below steps:
1. Package the sample programs. Execute below command under directory `TDengine/docs/examples/java` ：
```
mvn package
```
2. Create `examples/java` directory on the server
```
mkdir -p examples/java
```
3. Copy dependencies (below commands assume you are working on a local Windows host and try to launch on a remote Linux host)
- Copy dependent packages
```
scp -r .\target\lib <user>@<host>:~/examples/java
```
- Copy the jar of sample programs
```
scp -r .\target\javaexample-1.0.jar <user>@<host>:~/examples/java
```
4. Configure environment variable
Edit `~/.bash_profile` or `~/.bashrc` and add below:
```
export TDENGINE_JDBC_URL="jdbc:TAOS://localhost:6030?user=root&password=taosdata"
```
If your TDengine server is not deployed on localhost or doesn't use default port, you need to change the above URL to correct value in your environment.
5. Launch the sample program
```
java -classpath lib/*:javaexample-1.0.jar com.taos.example.highvolume.FastWriteExample <read_thread_count> <white_thread_count> <total_table_count> <max_batch_size>
```
6. The sample program doesn't exit unless you press <kbd>CTRL</kbd> + <kbd>C</kbd> to terminate it.
Below is the output of running on a server of 16 cores, 64GB memory and SSD hard disk.
```
root@vm85$ java -classpath lib/*:javaexample-1.0.jar com.taos.example.highvolume.FastWriteExample 2 12
18:56:35.896 [main] INFO c.t.e.highvolume.FastWriteExample - readTaskCount=2, writeTaskCount=12 tableCount=1000 maxBatchSize=3000
18:56:36.011 [WriteThread-0] INFO c.taos.example.highvolume.WriteTask - started
18:56:36.015 [WriteThread-0] INFO c.taos.example.highvolume.SQLWriter - maxSQLLength=1048576
18:56:36.021 [WriteThread-1] INFO c.taos.example.highvolume.WriteTask - started
18:56:36.022 [WriteThread-1] INFO c.taos.example.highvolume.SQLWriter - maxSQLLength=1048576
18:56:36.031 [WriteThread-2] INFO c.taos.example.highvolume.WriteTask - started
18:56:36.032 [WriteThread-2] INFO c.taos.example.highvolume.SQLWriter - maxSQLLength=1048576
18:56:36.041 [WriteThread-3] INFO c.taos.example.highvolume.WriteTask - started
18:56:36.042 [WriteThread-3] INFO c.taos.example.highvolume.SQLWriter - maxSQLLength=1048576
18:56:36.093 [WriteThread-4] INFO c.taos.example.highvolume.WriteTask - started
18:56:36.094 [WriteThread-4] INFO c.taos.example.highvolume.SQLWriter - maxSQLLength=1048576
18:56:36.099 [WriteThread-5] INFO c.taos.example.highvolume.WriteTask - started
18:56:36.100 [WriteThread-5] INFO c.taos.example.highvolume.SQLWriter - maxSQLLength=1048576
18:56:36.100 [WriteThread-6] INFO c.taos.example.highvolume.WriteTask - started
18:56:36.101 [WriteThread-6] INFO c.taos.example.highvolume.SQLWriter - maxSQLLength=1048576
18:56:36.103 [WriteThread-7] INFO c.taos.example.highvolume.WriteTask - started
18:56:36.104 [WriteThread-7] INFO c.taos.example.highvolume.SQLWriter - maxSQLLength=1048576
18:56:36.105 [WriteThread-8] INFO c.taos.example.highvolume.WriteTask - started
18:56:36.107 [WriteThread-8] INFO c.taos.example.highvolume.SQLWriter - maxSQLLength=1048576
18:56:36.108 [WriteThread-9] INFO c.taos.example.highvolume.WriteTask - started
18:56:36.109 [WriteThread-9] INFO c.taos.example.highvolume.SQLWriter - maxSQLLength=1048576
18:56:36.156 [WriteThread-10] INFO c.taos.example.highvolume.WriteTask - started
18:56:36.157 [WriteThread-11] INFO c.taos.example.highvolume.WriteTask - started
18:56:36.158 [WriteThread-10] INFO c.taos.example.highvolume.SQLWriter - maxSQLLength=1048576
18:56:36.158 [ReadThread-0] INFO com.taos.example.highvolume.ReadTask - started
18:56:36.158 [ReadThread-1] INFO com.taos.example.highvolume.ReadTask - started
18:56:36.158 [WriteThread-11] INFO c.taos.example.highvolume.SQLWriter - maxSQLLength=1048576
18:56:46.369 [main] INFO c.t.e.highvolume.FastWriteExample - count=18554448 speed=1855444
18:56:56.946 [main] INFO c.t.e.highvolume.FastWriteExample - count=39059660 speed=2050521
18:57:07.322 [main] INFO c.t.e.highvolume.FastWriteExample - count=59403604 speed=2034394
18:57:18.032 [main] INFO c.t.e.highvolume.FastWriteExample - count=80262938 speed=2085933
18:57:28.432 [main] INFO c.t.e.highvolume.FastWriteExample - count=101139906 speed=2087696
18:57:38.921 [main] INFO c.t.e.highvolume.FastWriteExample - count=121807202 speed=2066729
18:57:49.375 [main] INFO c.t.e.highvolume.FastWriteExample - count=142952417 speed=2114521
18:58:00.689 [main] INFO c.t.e.highvolume.FastWriteExample - count=163650306 speed=2069788
18:58:11.646 [main] INFO c.t.e.highvolume.FastWriteExample - count=185019808 speed=2136950
```
</details>
</TabItem>
<TabItem label="Python" value="python">
**Program Inventory**
Sample programs in Python uses multi-process and cross-process message queues.
| Function/CLass | Description |
| ---------------------------- | --------------------------------------------------------------------------- |
| main Function | Program entry point, create child processes and message queues |
| run_monitor_process Function | Create database, super table, calculate writing speed and output to console |
| run_read_task Function | Read data and distribute to message queues |
| MockDataSource Class | Simulate data source, return next 1,000 rows of each table |
| run_write_task Function | Read as much as possible data from message queue and write in batch |
| SQLWriter Class | Write in SQL and create table utomatically |
| StmtWriter Class | Write in parameter binding mode (not finished yet) |
<details>
<summary>main function</summary>
`main` function is responsible for creating message queues and fork child processes, there are 3 kinds of child processes:
1. Monitoring process, initializes database and calculating writing speed
2. Reading process (n), reads data from data source
3. Writing process (m), wirtes data into TDengine
`main` function provides 5 parameters:
1. The number of reading tasks, default value is 1
2. The number of writing tasks, default value is 1
3. The number of tables, default value is 1,000
4. The capacity of message queue, default value is 1,000,000 bytes
5. The batch size in single write, default value is 3000
```python
{{#include docs/examples/python/fast_write_example.py:main}}
```
</details>
<details>
<summary>run_monitor_process</summary>
Monitoring process initilizes database and monitoring writing speed.
```python
{{#include docs/examples/python/fast_write_example.py:monitor}}
```
</details>
<details>
<summary>run_read_task function</summary>
Reading process reads data from other data system and distributes to the message queue allocated for it.
```python
{{#include docs/examples/python/fast_write_example.py:read}}
```
</details>
<details>
<summary>MockDataSource</summary>
Below is the simulated data source, we assume table name exists in each generated data.
```python
{{#include docs/examples/python/mockdatasource.py}}
```
</details>
<details>
<summary>run_write_task function</summary>
Writing process tries to read as much as possible data from message queue and writes in batch.
```python
{{#include docs/examples/python/fast_write_example.py:write}}
```
</details>
<details>
SQLWriter class encapsulates the logic of composing SQL and writing data. Please be noted that the tables have not been created before writing, but are created automatically when catching the exception of table doesn't exist. For other exceptions caught, the SQL which caused the exception are logged for you to debug. This class also checks the SQL length, if the SQL length is closed to `maxSQLLength` the SQL will be executed immediately. To improve writing efficiency, it's better to increase `maxSQLLength` properly.
<summary>SQLWriter</summary>
```python
{{#include docs/examples/python/sql_writer.py}}
```
</details>
**Steps to Launch**
<details>
<summary>Launch Sample Program in Python</summary>
1. Prerequisities
- TDengine client driver has been installed
- Python3 has been installed, the the version >= 3.8
- TDengine Python connector `taospy` has been installed
2. Install faster-fifo to replace python builtin multiprocessing.Queue
```
pip3 install faster-fifo
```
3. Click the "Copy" in the above sample programs to copy `fast_write_example.py` 、 `sql_writer.py` and `mockdatasource.py`.
4. Execute the program
```
python3 fast_write_example.py <READ_TASK_COUNT> <WRITE_TASK_COUNT> <TABLE_COUNT> <QUEUE_SIZE> <MAX_BATCH_SIZE>
```
Below is the output of running on a server of 16 cores, 64GB memory and SSD hard disk.
```
root@vm85$ python3 fast_write_example.py 8 8
2022-07-14 19:13:45,869 [root] - READ_TASK_COUNT=8, WRITE_TASK_COUNT=8, TABLE_COUNT=1000, QUEUE_SIZE=1000000, MAX_BATCH_SIZE=3000
2022-07-14 19:13:48,882 [root] - WriteTask-0 started with pid 718347
2022-07-14 19:13:48,883 [root] - WriteTask-1 started with pid 718348
2022-07-14 19:13:48,884 [root] - WriteTask-2 started with pid 718349
2022-07-14 19:13:48,884 [root] - WriteTask-3 started with pid 718350
2022-07-14 19:13:48,885 [root] - WriteTask-4 started with pid 718351
2022-07-14 19:13:48,885 [root] - WriteTask-5 started with pid 718352
2022-07-14 19:13:48,886 [root] - WriteTask-6 started with pid 718353
2022-07-14 19:13:48,886 [root] - WriteTask-7 started with pid 718354
2022-07-14 19:13:48,887 [root] - ReadTask-0 started with pid 718355
2022-07-14 19:13:48,888 [root] - ReadTask-1 started with pid 718356
2022-07-14 19:13:48,889 [root] - ReadTask-2 started with pid 718357
2022-07-14 19:13:48,889 [root] - ReadTask-3 started with pid 718358
2022-07-14 19:13:48,890 [root] - ReadTask-4 started with pid 718359
2022-07-14 19:13:48,891 [root] - ReadTask-5 started with pid 718361
2022-07-14 19:13:48,892 [root] - ReadTask-6 started with pid 718364
2022-07-14 19:13:48,893 [root] - ReadTask-7 started with pid 718365
2022-07-14 19:13:56,042 [DataBaseMonitor] - count=6676310 speed=667631.0
2022-07-14 19:14:06,196 [DataBaseMonitor] - count=20004310 speed=1332800.0
2022-07-14 19:14:16,366 [DataBaseMonitor] - count=32290310 speed=1228600.0
2022-07-14 19:14:26,527 [DataBaseMonitor] - count=44438310 speed=1214800.0
2022-07-14 19:14:36,673 [DataBaseMonitor] - count=56608310 speed=1217000.0
2022-07-14 19:14:46,834 [DataBaseMonitor] - count=68757310 speed=1214900.0
2022-07-14 19:14:57,280 [DataBaseMonitor] - count=80992310 speed=1223500.0
2022-07-14 19:15:07,689 [DataBaseMonitor] - count=93805310 speed=1281300.0
2022-07-14 19:15:18,020 [DataBaseMonitor] - count=106111310 speed=1230600.0
2022-07-14 19:15:28,356 [DataBaseMonitor] - count=118394310 speed=1228300.0
2022-07-14 19:15:38,690 [DataBaseMonitor] - count=130742310 speed=1234800.0
2022-07-14 19:15:49,000 [DataBaseMonitor] - count=143051310 speed=1230900.0
2022-07-14 19:15:59,323 [DataBaseMonitor] - count=155276310 speed=1222500.0
2022-07-14 19:16:09,649 [DataBaseMonitor] - count=167603310 speed=1232700.0
2022-07-14 19:16:19,995 [DataBaseMonitor] - count=179976310 speed=1237300.0
```
</details>
:::note
Don't establish connection to TDengine in the parent process if using Python connector in multi-process way, otherwise all the connections in child processes are blocked always. This is a known issue.
:::
</TabItem>
</Tabs>
docs/en/14-reference/03-connector/java.mdx
浏览文件 @ f017c4dd
...@@ -91,7 +91,7 @@ Add following dependency in the `pom.xml` file of your Maven project: ...@@ -91,7 +91,7 @@ Add following dependency in the `pom.xml` file of your Maven project:
You can build Java connector from source code after cloning the TDengine project: You can build Java connector from source code after cloning the TDengine project:
``` ```
git clone https://github.com/taosdata/taos-connector-jdbc.git git clone https://github.com/taosdata/taos-connector-jdbc.git --branch 2.0
cd taos-connector-jdbc cd taos-connector-jdbc
mvn clean install -Dmaven.test.skip=true mvn clean install -Dmaven.test.skip=true
``` ```
...@@ -140,34 +140,34 @@ When you use a JDBC native connection to connect to a TDengine cluster, you can ...@@ -140,34 +140,34 @@ When you use a JDBC native connection to connect to a TDengine cluster, you can
1. Do not specify hostname and port in Java applications. 1. Do not specify hostname and port in Java applications.
```java ```java
public Connection getConn() throws Exception{ public Connection getConn() throws Exception{
Class.forName("com.taosdata.jdbc.TSDBDriver"); Class.forName("com.taosdata.jdbc.TSDBDriver");
String jdbcUrl = "jdbc:TAOS://:/test?user=root&password=taosdata"; String jdbcUrl = "jdbc:TAOS://:/test?user=root&password=taosdata";
Properties connProps = new Properties(); Properties connProps = new Properties();
connProps.setProperty(TSDBDriver.PROPERTY_KEY_CHARSET, "UTF-8"); connProps.setProperty(TSDBDriver.PROPERTY_KEY_CHARSET, "UTF-8");
connProps.setProperty(TSDBDriver.PROPERTY_KEY_LOCALE, "en_US.UTF-8"); connProps.setProperty(TSDBDriver.PROPERTY_KEY_LOCALE, "en_US.UTF-8");
connProps.setProperty(TSDBDriver.PROPERTY_KEY_TIME_ZONE, "UTC-8"); connProps.setProperty(TSDBDriver.PROPERTY_KEY_TIME_ZONE, "UTC-8");
Connection conn = DriverManager.getConnection(jdbcUrl, connProps); Connection conn = DriverManager.getConnection(jdbcUrl, connProps);
return conn; return conn;
} }
``` ```
2. specify the firstEp and the secondEp in the configuration file taos.cfg 2. specify the firstEp and the secondEp in the configuration file taos.cfg
```shell ```shell
# first fully qualified domain name (FQDN) for TDengine system # first fully qualified domain name (FQDN) for TDengine system
firstEp cluster_node1:6030 firstEp cluster_node1:6030
# second fully qualified domain name (FQDN) for TDengine system, for cluster only # second fully qualified domain name (FQDN) for TDengine system, for cluster only
secondEp cluster_node2:6030 secondEp cluster_node2:6030
# default system charset # default system charset
# charset UTF-8 # charset UTF-8
# system locale # system locale
# locale en_US.UTF-8 # locale en_US.UTF-8
``` ```
In the above example, JDBC uses the client's configuration file to establish a connection to a hostname `cluster_node1`, port 6030, and a database named `test`. When the firstEp node in the cluster fails, JDBC attempts to connect to the cluster using secondEp. In the above example, JDBC uses the client's configuration file to establish a connection to a hostname `cluster_node1`, port 6030, and a database named `test`. When the firstEp node in the cluster fails, JDBC attempts to connect to the cluster using secondEp.
......
docs/zh/02-intro.md
浏览文件 @ f017c4dd
...@@ -52,7 +52,7 @@ TDengine的主要功能如下： ...@@ -52,7 +52,7 @@ TDengine的主要功能如下：
采用 TDengine，可将典型的物联网、车联网、工业互联网大数据平台的总拥有成本大幅降低。表现在几个方面： 采用 TDengine，可将典型的物联网、车联网、工业互联网大数据平台的总拥有成本大幅降低。表现在几个方面：
1. 由于其超强性能，它能将系统需要的计算资源和存储资源大幅降低 1. 由于其超强性能，它能将系统需要的计算资源和存储资源大幅降低
2. 因为采用 SQL 接口，能与众多第三放软件无缝集成，学习迁移成本大幅下降 2. 因为采用 SQL 接口，能与众多第三方软件无缝集成，学习迁移成本大幅下降
3. 因为其 All In One 的特性，系统复杂度降低，能降研发成本 3. 因为其 All In One 的特性，系统复杂度降低，能降研发成本
4. 因为运维维护简单，运营维护成本能大幅降低 4. 因为运维维护简单，运营维护成本能大幅降低
......
docs/zh/04-concept/index.md
浏览文件 @ f017c4dd
...@@ -148,7 +148,7 @@ TDengine 建议用数据采集点的名字（如上表中的 D1001）来做表 ...@@ -148,7 +148,7 @@ TDengine 建议用数据采集点的名字（如上表中的 D1001）来做表
3. 子表一定属于一张超级表，但普通表不属于任何超级表 3. 子表一定属于一张超级表，但普通表不属于任何超级表
4. 普通表无法转为子表，子表也无法转为普通表。 4. 普通表无法转为子表，子表也无法转为普通表。
超级表与与基于超级表建立的子表之间的关系表现在： 超级表与基于超级表建立的子表之间的关系表现在：
1. 一张超级表包含有多张子表，这些子表具有相同的采集量 schema，但带有不同的标签值。 1. 一张超级表包含有多张子表，这些子表具有相同的采集量 schema，但带有不同的标签值。
2. 不能通过子表调整数据或标签的模式，对于超级表的数据模式修改立即对所有的子表生效。 2. 不能通过子表调整数据或标签的模式，对于超级表的数据模式修改立即对所有的子表生效。
......
docs/zh/14-reference/03-connector/java.mdx
浏览文件 @ f017c4dd
...@@ -93,7 +93,7 @@ Maven 项目中，在 pom.xml 中添加以下依赖： ...@@ -93,7 +93,7 @@ Maven 项目中，在 pom.xml 中添加以下依赖：
可以通过下载 TDengine 的源码，自己编译最新版本的 Java connector 可以通过下载 TDengine 的源码，自己编译最新版本的 Java connector
```shell ```shell
git clone https://github.com/taosdata/taos-connector-jdbc.git git clone https://github.com/taosdata/taos-connector-jdbc.git --branch 2.0
cd taos-connector-jdbc cd taos-connector-jdbc
mvn clean install -Dmaven.test.skip=true mvn clean install -Dmaven.test.skip=true
``` ```
......
packaging/release.sh
浏览文件 @ f017c4dd
...@@ -198,6 +198,7 @@ fi ...@@ -198,6 +198,7 @@ fi
if [[ "$dbName" != "taos" ]]; then if [[ "$dbName" != "taos" ]]; then
source ${enterprise_dir}/packaging/oem/sed_$dbName.sh source ${enterprise_dir}/packaging/oem/sed_$dbName.sh
replace_community_$dbName replace_community_$dbName
replace_output_$dbName
fi fi
if [[ "$httpdBuild" == "true" ]]; then if [[ "$httpdBuild" == "true" ]]; then
...@@ -224,6 +225,7 @@ if [[ "$cpuType" == "x64" ]] || [[ "$cpuType" == "aarch64" ]] || [[ "$cpuType" = ...@@ -224,6 +225,7 @@ if [[ "$cpuType" == "x64" ]] || [[ "$cpuType" == "aarch64" ]] || [[ "$cpuType" =
else else
if [[ "$dbName" != "taos" ]]; then if [[ "$dbName" != "taos" ]]; then
replace_enterprise_$dbName replace_enterprise_$dbName
replace_output_$dbName
fi fi
cmake ../../ -DCPUTYPE=${cpuType} -DOSTYPE=${osType} -DSOMODE=${soMode} -DDBNAME=${dbName} -DVERTYPE=${verType} -DVERDATE="${build_time}" -DGITINFO=${gitinfo} -DGITINFOI=${gitinfoOfInternal} -DVERNUMBER=${verNumber} -DVERCOMPATIBLE=${verNumberComp} -DBUILD_HTTP=${BUILD_HTTP} -DBUILD_TOOLS=${BUILD_TOOLS} ${allocator_macro} cmake ../../ -DCPUTYPE=${cpuType} -DOSTYPE=${osType} -DSOMODE=${soMode} -DDBNAME=${dbName} -DVERTYPE=${verType} -DVERDATE="${build_time}" -DGITINFO=${gitinfo} -DGITINFOI=${gitinfoOfInternal} -DVERNUMBER=${verNumber} -DVERCOMPATIBLE=${verNumberComp} -DBUILD_HTTP=${BUILD_HTTP} -DBUILD_TOOLS=${BUILD_TOOLS} ${allocator_macro}
fi fi
......
src/client/src/tscSQLParser.c
浏览文件 @ f017c4dd
...@@ -3972,10 +3972,6 @@ int32_t doGetColumnIndexByName(SStrToken* pToken, SQueryInfo* pQueryInfo, SColum ...@@ -3972,10 +3972,6 @@ int32_t doGetColumnIndexByName(SStrToken* pToken, SQueryInfo* pQueryInfo, SColum
const char* msg0 = "ambiguous column name"; const char* msg0 = "ambiguous column name";
const char* msg1 = "invalid column name"; const char* msg1 = "invalid column name";
if (pToken->n == 0) {
return TSDB_CODE_TSC_INVALID_OPERATION;
}
int16_t tsWinColumnIndex; int16_t tsWinColumnIndex;
if (isTablenameToken(pToken)) { if (isTablenameToken(pToken)) {
pIndex->columnIndex = TSDB_TBNAME_COLUMN_INDEX; pIndex->columnIndex = TSDB_TBNAME_COLUMN_INDEX;
...@@ -4061,6 +4057,12 @@ int32_t getTableIndexByName(SStrToken* pToken, SQueryInfo* pQueryInfo, SColumnIn ...@@ -4061,6 +4057,12 @@ int32_t getTableIndexByName(SStrToken* pToken, SQueryInfo* pQueryInfo, SColumnIn
} }
int32_t getColumnIndexByName(const SStrToken* pToken, SQueryInfo* pQueryInfo, SColumnIndex* pIndex, char* msg) { int32_t getColumnIndexByName(const SStrToken* pToken, SQueryInfo* pQueryInfo, SColumnIndex* pIndex, char* msg) {
const char* msg0 = "invalid column name";
if (pToken->n == 0) {
return invalidOperationMsg(msg, msg0);
}
if (pQueryInfo->pTableMetaInfo == NULL || pQueryInfo->numOfTables == 0) { if (pQueryInfo->pTableMetaInfo == NULL || pQueryInfo->numOfTables == 0) {
return TSDB_CODE_TSC_INVALID_OPERATION; return TSDB_CODE_TSC_INVALID_OPERATION;
} }
...@@ -7399,6 +7401,9 @@ int32_t setAlterTableInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) { ...@@ -7399,6 +7401,9 @@ int32_t setAlterTableInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) {
} }
SColumnIndex idx = COLUMN_INDEX_INITIALIZER; SColumnIndex idx = COLUMN_INDEX_INITIALIZER;
if (pItem->pVar.nType != TSDB_DATA_TYPE_BINARY) {
return invalidOperationMsg(pMsg, msg17);
}
SStrToken name = {.z = pItem->pVar.pz, .n = pItem->pVar.nLen}; SStrToken name = {.z = pItem->pVar.pz, .n = pItem->pVar.nLen};
if (getColumnIndexByName(&name, pQueryInfo, &idx, tscGetErrorMsgPayload(pCmd)) != TSDB_CODE_SUCCESS) { if (getColumnIndexByName(&name, pQueryInfo, &idx, tscGetErrorMsgPayload(pCmd)) != TSDB_CODE_SUCCESS) {
...@@ -7472,6 +7477,9 @@ int32_t setAlterTableInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) { ...@@ -7472,6 +7477,9 @@ int32_t setAlterTableInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) {
int16_t numOfTags = tscGetNumOfTags(pTableMeta); int16_t numOfTags = tscGetNumOfTags(pTableMeta);
SColumnIndex columnIndex = COLUMN_INDEX_INITIALIZER; SColumnIndex columnIndex = COLUMN_INDEX_INITIALIZER;
if (item->pVar.nType != TSDB_DATA_TYPE_BINARY) {
return invalidOperationMsg(pMsg, msg17);
}
SStrToken name = {.z = item->pVar.pz, .n = item->pVar.nLen}; SStrToken name = {.z = item->pVar.pz, .n = item->pVar.nLen};
if (getColumnIndexByName(&name, pQueryInfo, &columnIndex, tscGetErrorMsgPayload(pCmd)) != TSDB_CODE_SUCCESS) { if (getColumnIndexByName(&name, pQueryInfo, &columnIndex, tscGetErrorMsgPayload(pCmd)) != TSDB_CODE_SUCCESS) {
return TSDB_CODE_TSC_INVALID_OPERATION; return TSDB_CODE_TSC_INVALID_OPERATION;
...@@ -7617,6 +7625,9 @@ int32_t setAlterTableInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) { ...@@ -7617,6 +7625,9 @@ int32_t setAlterTableInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) {
tVariantListItem* pItem = taosArrayGet(pAlterSQL->varList, 0); tVariantListItem* pItem = taosArrayGet(pAlterSQL->varList, 0);
SColumnIndex columnIndex = COLUMN_INDEX_INITIALIZER; SColumnIndex columnIndex = COLUMN_INDEX_INITIALIZER;
if (pItem->pVar.nType != TSDB_DATA_TYPE_BINARY) {
return invalidOperationMsg(pMsg, msg17);
}
SStrToken name = {.z = pItem->pVar.pz, .n = pItem->pVar.nLen}; SStrToken name = {.z = pItem->pVar.pz, .n = pItem->pVar.nLen};
if (getColumnIndexByName(&name, pQueryInfo, &columnIndex, tscGetErrorMsgPayload(pCmd)) != TSDB_CODE_SUCCESS) { if (getColumnIndexByName(&name, pQueryInfo, &columnIndex, tscGetErrorMsgPayload(pCmd)) != TSDB_CODE_SUCCESS) {
...@@ -9665,7 +9676,9 @@ int32_t tscGetExprFilters(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SArray* pSelect ...@@ -9665,7 +9676,9 @@ int32_t tscGetExprFilters(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SArray* pSelect
SStrToken* pToken = &pParam->pNode->columnName; SStrToken* pToken = &pParam->pNode->columnName;
SColumnIndex idx = COLUMN_INDEX_INITIALIZER; SColumnIndex idx = COLUMN_INDEX_INITIALIZER;
getColumnIndexByName(pToken, pQueryInfo, &idx, tscGetErrorMsgPayload(pCmd)); if (getColumnIndexByName(pToken, pQueryInfo, &idx, tscGetErrorMsgPayload(pCmd)) != TSDB_CODE_SUCCESS) {
return TSDB_CODE_TSC_INVALID_OPERATION;
}
STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, idx.tableIndex); STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, idx.tableIndex);
schema = *tscGetTableColumnSchema(pTableMetaInfo->pTableMeta, idx.columnIndex); schema = *tscGetTableColumnSchema(pTableMetaInfo->pTableMeta, idx.columnIndex);
} else { } else {
......
src/kit/shell/CMakeLists.txt
浏览文件 @ f017c4dd
...@@ -46,6 +46,8 @@ ELSEIF (TD_DARWIN) ...@@ -46,6 +46,8 @@ ELSEIF (TD_DARWIN)
LIST(APPEND SRC ./src/shellCommand.c) LIST(APPEND SRC ./src/shellCommand.c)
LIST(APPEND SRC ./src/shellImport.c) LIST(APPEND SRC ./src/shellImport.c)
LIST(APPEND SRC ./src/shellCheck.c) LIST(APPEND SRC ./src/shellCheck.c)
LIST(APPEND SRC ./src/shellAuto.c)
LIST(APPEND SRC ./src/tire.c)
ADD_EXECUTABLE(shell ${SRC}) ADD_EXECUTABLE(shell ${SRC})
# linking with dylib # linking with dylib
TARGET_LINK_LIBRARIES(shell taos cJson) TARGET_LINK_LIBRARIES(shell taos cJson)
......
src/kit/shell/inc/shellAuto.h 0 → 100644
浏览文件 @ f017c4dd
/*
* Copyright (c) 2019 TAOS Data, Inc. <jhtao@taosdata.com>
*
* This program is free software: you can use, redistribute, and/or modify
* it under the terms of the GNU Affero General Public License, version 3
* or later ("AGPL"), as published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#ifndef __SHELL_AUTO__
#define __SHELL_AUTO__
#define TAB_KEY 0x09
// press tab key
void pressTabKey(TAOS * con, Command * cmd);
// press othr key
void pressOtherKey(char c);
// init shell auto funciton , shell start call once
bool shellAutoInit();
// exit shell auto funciton, shell exit call once
void shellAutoExit();
// callback autotab module
void callbackAutoTab(char* sqlstr, TAOS* pSql, bool usedb);
#endif
src/kit/shell/inc/shellCommand.h
浏览文件 @ f017c4dd
...@@ -41,6 +41,7 @@ extern void deleteChar(Command *cmd); ...@@ -41,6 +41,7 @@ extern void deleteChar(Command *cmd);
extern void moveCursorLeft(Command *cmd); extern void moveCursorLeft(Command *cmd);
extern void moveCursorRight(Command *cmd); extern void moveCursorRight(Command *cmd);
extern void positionCursorHome(Command *cmd); extern void positionCursorHome(Command *cmd);
extern void positionCursorMiddle(Command *cmd);
extern void positionCursorEnd(Command *cmd); extern void positionCursorEnd(Command *cmd);
extern void showOnScreen(Command *cmd); extern void showOnScreen(Command *cmd);
extern void updateBuffer(Command *cmd); extern void updateBuffer(Command *cmd);
...@@ -51,5 +52,6 @@ int countPrefixOnes(unsigned char c); ...@@ -51,5 +52,6 @@ int countPrefixOnes(unsigned char c);
void clearScreen(int ecmd_pos, int cursor_pos); void clearScreen(int ecmd_pos, int cursor_pos);
void printChar(char c, int times); void printChar(char c, int times);
void positionCursor(int step, int direction); void positionCursor(int step, int direction);
void getPrevCharSize(const char *str, int pos, int *size, int *width);
#endif #endif
src/kit/shell/inc/tire.h 0 → 100644
浏览文件 @ f017c4dd
/*
* Copyright (c) 2019 TAOS Data, Inc. <jhtao@taosdata.com>
*
* This program is free software: you can use, redistribute, and/or modify
* it under the terms of the GNU Affero General Public License, version 3
* or later ("AGPL"), as published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#ifndef __TRIE__
#define __TRIE__
//
// The prefix search tree is a efficient storage words and search words tree, it support 95 visible ascii code character
//
#define FIRST_ASCII 40 // first visiable char is '0'
#define LAST_ASCII 122 // last visilbe char is 'z'
// capacity save char is 95
#define CHAR_CNT (LAST_ASCII - FIRST_ASCII + 1)
#define MAX_WORD_LEN 256 // max insert word length
// define STire
#define TIRE_TREE 0
#define TIRE_LIST 1
typedef struct STireNode {
struct STireNode** d;
bool end; // record end flag
}STireNode;
typedef struct StrName {
char * name;
struct StrName * next;
}StrName;
typedef struct STire {
char type; // see define TIRE_
STireNode root;
StrName * head;
StrName * tail;
int count; // all count
int ref;
}STire;
typedef struct SMatchNode {
char* word;
struct SMatchNode* next;
}SMatchNode;
typedef struct SMatch {
SMatchNode* head;
SMatchNode* tail; // append node to tail
int count;
char pre[MAX_WORD_LEN];
}SMatch;
// ----------- interface -------------
// create prefix search tree, return value call freeTire to free
STire* createTire(char type);
// destroy prefix search tree
void freeTire(STire* tire);
// add a new word
bool insertWord(STire* tire, char* word);
// add a new word
bool deleteWord(STire* tire, char* word);
// match prefix words, if match is not NULL , put all item to match and return match
SMatch* matchPrefix(STire* tire, char* prefix, SMatch* match);
// get all items from tires tree
SMatch* enumAll(STire* tire);
// free match result
void freeMatch(SMatch* match);
#endif
src/kit/shell/src/shellAuto.c 0 → 100644
浏览文件 @ f017c4dd
/*
* Copyright (c) 2019 TAOS Data, Inc. <jhtao@taosdata.com>
*
* This program is free software: you can use, redistribute, and/or modify
* it under the terms of the GNU Affero General Public License, version 3
* or later ("AGPL"), as published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#define __USE_XOPEN
#include "os.h"
#include "tglobal.h"
#include "shell.h"
#include "shellCommand.h"
#include "tkey.h"
#include "tulog.h"
#include "shellAuto.h"
#include "tire.h"
#include "tthread.h"
//
// ------------- define area ---------------
//
#define UNION_ALL " union all "
// extern function
void insertChar(Command *cmd, char *c, int size);
typedef struct SAutoPtr {
STire* p;
int ref;
}SAutoPtr;
typedef struct SWord{
int type ; // word type , see WT_ define
char * word;
int32_t len;
struct SWord * next;
bool free; // if true need free
}SWord;
typedef struct {
char * source;
int32_t source_len; // valid data length in source
int32_t count;
SWord* head;
// matched information
int32_t matchIndex; // matched word index in words
int32_t matchLen; // matched length at matched word
}SWords;
SWords shellCommands[] = {
{"alter database <db_name> <db_options> <anyword> <db_options> <anyword> <db_options> <anyword> <db_options> <anyword> <db_options> <anyword> <db_options> <anyword>", 0, 0, NULL},
{"alter dnode <dnode_id> balance ", 0, 0, NULL},
{"alter dnode <dnode_id> resetlog;", 0, 0, NULL},
{"alter dnode <dnode_id> debugFlag 141;", 0, 0, NULL},
{"alter dnode <dnode_id> monitor 1;", 0, 0, NULL},
{"alter table <tb_name> <tb_actions>", 0, 0, NULL},
{"alter table modify column", 0, 0, NULL},
{"alter local resetlog;", 0, 0, NULL},
{"alter local DebugFlag 143;", 0, 0, NULL},
{"alter local cDebugFlag 143;", 0, 0, NULL},
{"alter local uDebugFlag 143;", 0, 0, NULL},
{"alter local rpcDebugFlag 143;", 0, 0, NULL},
{"alter local tmrDebugFlag 143;", 0, 0, NULL},
{"alter topic", 0, 0, NULL},
{"alter user <user_name> pass", 0, 0, NULL},
{"alter user <user_name> privilege read", 0, 0, NULL},
{"alter user <user_name> privilege write", 0, 0, NULL},
{"create table <anyword> using <stb_name> tags(", 0, 0, NULL},
{"create database ", 0, 0, NULL},
{"create table <anyword> as ", 0, 0, NULL},
{"create dnode ", 0, 0, NULL},
{"create topic", 0, 0, NULL},
{"create function ", 0, 0, NULL},
{"create user <anyword> pass", 0, 0, NULL},
{"compact vnode in", 0, 0, NULL},
{"describe <all_table>", 0, 0, NULL},
#ifdef TD_ENTERPRISE
{"delete from <all_table> where", 0, 0, NULL},
#endif
{"drop database <db_name>", 0, 0, NULL},
{"drop table <all_table> ", 0, 0, NULL},
{"drop dnode <dnode_id>", 0, 0, NULL},
{"drop user <user_name> ;", 0, 0, NULL},
{"drop function", 0, 0, NULL},
{"drop topic", 0, 0, NULL},
{"kill connection", 0, 0, NULL},
{"kill query", 0, 0, NULL},
{"kill stream", 0, 0, NULL},
{"select * from <all_table> where ", 0, 0, NULL},
{"select _block_dist() from <all_table> \\G;", 0, 0, NULL},
{"select client_version();", 0, 0, NULL},
{"select current_user();", 0, 0, NULL},
{"select database;", 0, 0, NULL},
{"select server_version();", 0, 0, NULL},
{"set max_binary_display_width ", 0, 0, NULL},
{"show create database <db_name> \\G;", 0, 0, NULL},
{"show create stable <stb_name> \\G;", 0, 0, NULL},
{"show create table <tb_name> \\G;", 0, 0, NULL},
{"show connections;", 0, 0, NULL},
{"show databases;", 0, 0, NULL},
{"show dnodes;", 0, 0, NULL},
{"show functions;", 0, 0, NULL},
{"show modules;", 0, 0, NULL},
{"show mnodes;", 0, 0, NULL},
{"show queries;", 0, 0, NULL},
{"show stables;", 0, 0, NULL},
{"show stables like ", 0, 0, NULL},
{"show streams;", 0, 0, NULL},
{"show scores;", 0, 0, NULL},
{"show tables;", 0, 0, NULL},
{"show tables like", 0, 0, NULL},
{"show users;", 0, 0, NULL},
{"show variables;", 0, 0, NULL},
{"show vgroups;", 0, 0, NULL},
{"insert into <tb_name> values(", 0, 0, NULL},
{"insert into <tb_name> using <stb_name> tags(", 0, 0, NULL},
{"use <db_name>", 0, 0, NULL},
{"quit", 0, 0, NULL}
};
char * keywords[] = {
"and ",
"asc ",
"desc ",
"from ",
"fill(",
"limit ",
"where ",
"interval(",
"order by ",
"order by ",
"offset ",
"or ",
"group by ",
"now()",
"session(",
"sliding ",
"slimit ",
"soffset ",
"state_window(",
"today() ",
"union all select ",
};
char * functions[] = {
"count(",
"sum(",
"avg(",
"last(",
"last_row(",
"top(",
"interp(",
"max(",
"min(",
"now()",
"today()",
"percentile(",
"tail(",
"pow(",
"abs(",
"atan(",
"acos(",
"asin(",
"apercentile(",
"bottom(",
"cast(",
"ceil(",
"char_length(",
"cos(",
"concat(",
"concat_ws(",
"csum(",
"diff(",
"derivative(",
"elapsed(",
"first(",
"floor(",
"hyperloglog(",
"histogram(",
"irate(",
"leastsquares(",
"length(",
"log(",
"lower(",
"ltrim(",
"mavg(",
"mode(",
"tan(",
"round(",
"rtrim(",
"sample(",
"sin(",
"spread(",
"substr(",
"statecount(",
"stateduration(",
"stddev(",
"sqrt(",
"timediff(",
"timezone(",
"timetruncate(",
"twa(",
"to_unixtimestamp(",
"unique(",
"upper(",
};
char * tb_actions[] = {
"add column",
"modify column",
"drop column",
"change tag",
};
char * db_options[] = {
"blocks",
"cachelast",
"comp",
"keep",
"replica",
"quorum",
};
char * data_types[] = {
"timestamp",
"int",
"float",
"double",
"binary(16)",
"nchar(16)",
"bigint",
"smallint",
"tinyint",
"bool",
"json"
};
char * key_tags[] = {
"tags("
};
//
// ------- gobal variant define ---------
//
int32_t firstMatchIndex = -1; // first match shellCommands index
int32_t lastMatchIndex = -1; // last match shellCommands index
int32_t curMatchIndex = -1; // current match shellCommands index
int32_t lastWordBytes = -1; // printShow last word length
bool waitAutoFill = false;
//
// ----------- global var array define -----------
//
#define WT_VAR_DBNAME 0
#define WT_VAR_STABLE 1
#define WT_VAR_TABLE 2
#define WT_VAR_DNODEID 3
#define WT_VAR_USERNAME 4
#define WT_VAR_ALLTABLE 5
#define WT_VAR_FUNC 6
#define WT_VAR_KEYWORD 7
#define WT_VAR_TBACTION 8
#define WT_VAR_DBOPTION 9
#define WT_VAR_DATATYPE 10
#define WT_VAR_KEYTAGS 11
#define WT_VAR_ANYWORD 12
#define WT_VAR_CNT 13
#define WT_FROM_DB_MAX 4 // max get content from db
#define WT_FROM_DB_CNT (WT_FROM_DB_MAX + 1)
#define WT_TEXT 0xFF
char dbName[256] = ""; // save use database name;
// tire array
STire* tires[WT_VAR_CNT];
pthread_mutex_t tiresMutex;
//save thread handle obtain var name from db server
pthread_t* threads[WT_FROM_DB_CNT];
// obtain var name with sql from server
char varTypes[WT_VAR_CNT][64] = {
"<db_name>",
"<stb_name>",
"<tb_name>",
"<dnode_id>",
"<user_name>",
"<all_table>",
"<function>",
"<keyword>",
"<tb_actions>",
"<db_options>",
"<data_types>",
"<key_tags>",
"<anyword>"
};
char varSqls[WT_FROM_DB_CNT][64] = {
"show databases;",
"show stables;",
"show tables;",
"show dnodes;",
"show users;"
};
// var words current cursor, if user press any one key except tab, cursorVar can be reset to -1
int cursorVar = -1;
bool varMode = false; // enter var names list mode
TAOS* varCon = NULL;
Command* varCmd = NULL;
SMatch* lastMatch = NULL; // save last match result
int cntDel = 0; // delete byte count after next press tab
// show auto tab introduction
void printfIntroduction() {
printf(" ********************* How to Use TAB in TAOS Shell ******************************\n");
printf(" * Taos shell supports pressing TAB key to complete word. You can try it. *\n");
printf(" * Press TAB key anywhere, You'll get surprise. *\n");
printf(" * KEYBOARD SHORTCUT: *\n");
printf(" * [ TAB ] ...... Complete the word or show help if no input *\n");
printf(" * [ Ctrl + A ] ...... move cursor to [A]head of line *\n");
printf(" * [ Ctrl + E ] ...... move cursor to [E]nd of line *\n");
printf(" * [ Ctrl + W ] ...... move cursor to line of middle *\n");
printf(" * [ Ctrl + L ] ...... clean screen *\n");
printf(" * [ Ctrl + K ] ...... clean after cursor *\n");
printf(" * [ Ctrl + U ] ...... clean before cursor *\n");
printf(" * *\n");
printf(" **********************************************************************************\n\n");
}
void showHelp() {
printf("\nThe following are supported commands for Taos shell:");
printf("\n\
----- A ----- \n\
alter database <db_name> <db_options> \n\
alter dnode <dnode_id> balance \n\
alter dnode <dnode_id> resetlog;\n\
alter dnode <dnode_id> DebugFlag 143;\n\
alter dnode <dnode_id> monitor 1;\n\
alter table <tb_name> ADD COLUMN <field_name> <data_type>; \n\
alter table <tb_name> DROP COLUMN <field_name>; \n\
alter table <tb_name> MODIFY COLUMN <field_name> <data_type(length)>;\n\
alter local resetlog; \n\
alter local DebugFlag 143; \n\
alter topic <topic_name>\n\
alter user <user_name> pass\n\
alter user <user_name> privilege read ;\n\
alter user <user_name> privilege write ;\n\
----- C ----- \n\
create table <tb_name> using <stb_name> tags ...\n\
create database <db_name>;\n\
create table <anyword> as ...\n\
create dnode <dnode_id>\n\
create topic <top_name>\n\
create function <function_name>\n\
create user <user_name> pass <password>;\n\
compact vnode in (vgid,vgid,vgid);\n\
----- D ----- \n\
describe <all_table> ;\n\
delete from <all_table> where ... \n\
drop database <db_name>;\n\
drop table <all_table>;\n\
drop dnode <dnode_id>;\n\
drop function <function_id>;\n\
drop topic <topic_id>;\n\
drop user <user_name>;\n\
----- K ----- \n\
kill connection <connection_id>; \n\
kill query <query_id>; \n\
kill stream <stream_id>; \n\
----- S ----- \n\
select * from <all_table> where ... \n\
select _block_dist() from <all_table>;\n\
select client_version();\n\
select current_user();\n\
select database;\n\
select server_version();\n\
set max_binary_display_width <width>; \n\
show create database <db_name>;\n\
show create stable <stb_name>;\n\
show create table <tb_name>;\n\
show connections;\n\
show databases;\n\
show dnodes;\n\
show functions;\n\
show modules;\n\
show mnodes;\n\
show queries;\n\
show stables;\n\
show stables like '<regular expression>'; note: regular expression only support '_' and '%%' match.\n\
show streams;\n\
show scores;\n\
show tables;\n\
show tables like '<regular expression>'; \n\
show users;\n\
show variables;\n\
show vgroups;\n\
----- I ----- \n\
insert into <tb_name> values(...) ;\n\
----- U ----- \n\
use <db_name>;");
printf("\n\n");
//define in getDuration() function
printf("\
Timestamp expression Format:\n\
b - nanosecond \n\
u - microsecond \n\
a - millisecond \n\
s - second \n\
m - minute \n\
h - hour \n\
d - day \n\
w - week \n\
now - current time \n\
Example : \n\
select * from t1 where ts > now - 2w + 3d and ts <= now - 1w -2h ;\n");
printf("\n");
}
//
// ------------------- parse words --------------------------
//
#define SHELL_COMMAND_COUNT() (sizeof(shellCommands) / sizeof(SWords))
// get at
SWord * atWord(SWords * command, int32_t index) {
SWord * word = command->head;
for (int32_t i = 0; i < index; i++) {
if (word == NULL)
return NULL;
word = word->next;
}
return word;
}
#define MATCH_WORD(x) atWord(x, x->matchIndex)
int wordType(const char* p, int32_t len) {
for (int i = 0; i < WT_VAR_CNT; i++) {
if (strncmp(p, varTypes[i], len) == 0)
return i;
}
return WT_TEXT;
}
// add word
SWord * addWord(const char* p, int32_t len, bool pattern) {
SWord* word = (SWord *) malloc(sizeof(SWord));
memset(word, 0, sizeof(SWord));
word->word = (char* )p;
word->len = len;
// check format
if (pattern) {
word->type = wordType(p, len);
} else {
word->type = WT_TEXT;
}
return word;
}
// parse one command
void parseCommand(SWords * command, bool pattern) {
char * p = command->source;
int32_t start = 0;
int32_t size = command->source_len > 0 ? command->source_len : strlen(p);
bool lastBlank = false;
for (int i = 0; i <= size; i++) {
if (p[i] == ' ' || i == size) {
// check continue blank like ' '
if (p[i] == ' ') {
if (lastBlank) {
start ++;
continue;
}
if (i == 0) { // first blank
lastBlank = true;
start ++;
continue;
}
lastBlank = true;
}
// found split or string end , append word
if (command->head == NULL) {
command->head = addWord(p + start, i - start, pattern);
command->count = 1;
} else {
SWord * word = command->head;
while (word->next) {
word = word->next;
}
word->next = addWord(p + start, i - start, pattern);
command->count ++;
}
start = i + 1;
} else {
lastBlank = false;
}
}
}
// free Command
void freeCommand(SWords * command) {
SWord * word = command->head;
if (word == NULL) {
return ;
}
// loop
while (word->next) {
SWord * tmp = word;
word = word->next;
// if malloc need free
if(tmp->free && tmp->word)
free(tmp->word);
free(tmp);
}
// if malloc need free
if(word->free && word->word)
free(word->word);
free(word);
}
void GenerateVarType(int type, char** p, int count) {
STire* tire = createTire(TIRE_LIST);
for (int i = 0; i < count; i++) {
insertWord(tire, p[i]);
}
pthread_mutex_lock(&tiresMutex);
tires[type] = tire;
pthread_mutex_unlock(&tiresMutex);
}
//
// -------------------- shell auto ----------------
//
// init shell auto funciton , shell start call once
bool shellAutoInit() {
// command
int32_t count = SHELL_COMMAND_COUNT();
for (int32_t i = 0; i < count; i ++) {
parseCommand(shellCommands + i, true);
}
// tires
memset(tires, 0, sizeof(STire*) * WT_VAR_CNT);
pthread_mutex_init(&tiresMutex, NULL);
// threads
memset(threads, 0, sizeof(pthread_t*) * WT_FROM_DB_CNT);
// generate varType
GenerateVarType(WT_VAR_FUNC, functions, sizeof(functions) /sizeof(char *));
GenerateVarType(WT_VAR_KEYWORD, keywords, sizeof(keywords) /sizeof(char *));
GenerateVarType(WT_VAR_DBOPTION, db_options, sizeof(db_options) /sizeof(char *));
GenerateVarType(WT_VAR_TBACTION, tb_actions, sizeof(tb_actions) /sizeof(char *));
GenerateVarType(WT_VAR_DATATYPE, data_types, sizeof(data_types) /sizeof(char *));
GenerateVarType(WT_VAR_KEYTAGS, key_tags, sizeof(key_tags) /sizeof(char *));
printfIntroduction();
return true;
}
// exit shell auto funciton, shell exit call once
void shellAutoExit() {
// free command
int32_t count = SHELL_COMMAND_COUNT();
for (int32_t i = 0; i < count; i ++) {
freeCommand(shellCommands + i);
}
// free tires
pthread_mutex_lock(&tiresMutex);
for (int32_t i = 0; i < WT_VAR_CNT; i++) {
if (tires[i]) {
freeTire(tires[i]);
tires[i] = NULL;
}
}
pthread_mutex_unlock(&tiresMutex);
// destory
pthread_mutex_destroy(&tiresMutex);
// free threads
for (int32_t i = 0; i < WT_VAR_CNT; i++) {
if (threads[i]) {
taosDestroyThread(threads[i]);
threads[i] = NULL;
}
}
// free lastMatch
if (lastMatch) {
freeMatch(lastMatch);
lastMatch = NULL;
}
}
//
// ------------------- auto ptr for tires --------------------------
//
bool setNewAuotPtr(int type, STire* pNew) {
if (pNew == NULL)
return false;
pthread_mutex_lock(&tiresMutex);
STire* pOld = tires[type];
if (pOld != NULL) {
// previous have value, release self ref count
if (--pOld->ref == 0) {
freeTire(pOld);
}
}
// set new
tires[type] = pNew;
tires[type]->ref = 1;
pthread_mutex_unlock(&tiresMutex);
return true;
}
// get ptr
STire* getAutoPtr(int type) {
if (tires[type] == NULL) {
return NULL;
}
pthread_mutex_lock(&tiresMutex);
tires[type]->ref++;
pthread_mutex_unlock(&tiresMutex);
return tires[type];
}
// put back tire to tires[type], if tire not equal tires[type].p, need free tire
void putBackAutoPtr(int type, STire* tire) {
if (tire == NULL) {
return ;
}
pthread_mutex_lock(&tiresMutex);
if (tires[type] != tire) {
//update by out, can't put back , so free
if (--tire->ref == 1) {
// support multi thread getAuotPtr
freeTire(tire);
}
} else {
tires[type]->ref--;
assert(tires[type]->ref > 0);
}
pthread_mutex_unlock(&tiresMutex);
return ;
}
//
// ------------------- var Word --------------------------
//
#define MAX_CACHED_CNT 100000 // max cached rows 10w
// write sql result to var name, return write rows cnt
int writeVarNames(int type, TAOS_RES* tres) {
// fetch row
TAOS_ROW row = taos_fetch_row(tres);
if (row == NULL) {
return 0;
}
TAOS_FIELD *fields = taos_fetch_fields(tres);
// create new tires
char tireType = type == WT_VAR_TABLE ? TIRE_TREE : TIRE_LIST;
STire* tire = createTire(tireType);
// enum rows
char name[1024];
int numOfRows = 0;
do {
int32_t* lengths = taos_fetch_lengths(tres);
int32_t bytes = lengths[0];
if(fields[0].type == TSDB_DATA_TYPE_SMALLINT) {
sprintf(name,"%d", *(int16_t*)row[0]);
} else {
memcpy(name, row[0], bytes);
}
name[bytes] = 0; //set string end
// insert to tire
insertWord(tire, name);
if (++numOfRows > MAX_CACHED_CNT ) {
break;
}
row = taos_fetch_row(tres);
} while (row != NULL);
// replace old tire
setNewAuotPtr(type, tire);
return numOfRows;
}
bool firstMatchCommand(TAOS * con, Command * cmd);
//
// thread obtain var thread from db server
//
void* varObtainThread(void* param) {
int type = *(int* )param;
free(param);
if (varCon == NULL || type > WT_FROM_DB_MAX) {
return NULL;
}
TAOS_RES* pSql = taos_query_h(varCon, varSqls[type], NULL);
if (taos_errno(pSql)) {
taos_free_result(pSql);
return NULL;
}
// write var names from pSql
int cnt = writeVarNames(type, pSql);
// free sql
taos_free_result(pSql);
// check need call auto tab
if (cnt > 0 && waitAutoFill) {
// press tab key by program
firstMatchCommand(varCon, varCmd);
}
return NULL;
}
// only match next one word from all match words, return valuue must free by caller
char* matchNextPrefix(STire* tire, char* pre) {
SMatch* match = NULL;
// re-use last result
if (lastMatch) {
if (strcmp(pre, lastMatch->pre) == 0) {
// same pre
match = lastMatch;
}
}
if (match == NULL) {
// not same with last result
if (pre[0] == 0) {
// EMPTY PRE
match = enumAll(tire);
} else {
// NOT EMPTY
match = matchPrefix(tire, pre, NULL);
}
// save to lastMatch
if (match) {
if (lastMatch)
freeMatch(lastMatch);
lastMatch = match;
}
}
// check valid
if (match == NULL || match->head == NULL) {
// no one matched
return false;
}
if (cursorVar == -1) {
// first
cursorVar = 0;
return strdup(match->head->word);
}
// according to cursorVar , calculate next one
int i = 0;
SMatchNode* item = match->head;
while (item) {
if (i == cursorVar + 1) {
// found next position ok
if (item->next == NULL) {
// match last item, reset cursorVar to head
cursorVar = -1;
} else {
cursorVar = i;
}
return strdup(item->word);
}
// check end item
if (item->next == NULL) {
// if cursorVar > var list count, return last and reset cursorVar
cursorVar = -1;
return strdup(item->word);
}
// move next
item = item->next;
i++;
}
return NULL;
}
// search pre word from tire tree, return value must free by caller
char* tireSearchWord(int type, char* pre) {
if (type == WT_TEXT) {
return NULL;
}
if(type > WT_FROM_DB_MAX) {
// NOT FROM DB , tires[type] alwary not null
STire* tire = tires[type];
if (tire == NULL)
return NULL;
return matchNextPrefix(tire, pre);
}
// TYPE CONTEXT GET FROM DB
pthread_mutex_lock(&tiresMutex);
// check need obtain from server
if (tires[type] == NULL) {
waitAutoFill = true;
// need async obtain var names from db sever
if (threads[type] != NULL) {
if (taosThreadRunning(threads[type])) {
// thread running , need not obtain again, return
pthread_mutex_unlock(&tiresMutex);
return NULL;
}
// destroy previous thread handle for new create thread handle
taosDestroyThread(threads[type]);
threads[type] = NULL;
}
// create new
void * param = malloc(sizeof(int));
*((int* )param) = type;
threads[type] = taosCreateThread(varObtainThread, param);
pthread_mutex_unlock(&tiresMutex);
return NULL;
}
pthread_mutex_unlock(&tiresMutex);
// can obtain var names from local
STire* tire = getAutoPtr(type);
if (tire == NULL) {
return NULL;
}
char* str = matchNextPrefix(tire, pre);
// used finish, put back pointer to autoptr array
putBackAutoPtr(type, tire);
return str;
}
// match var word, word1 is pattern , word2 is input from shell
bool matchVarWord(SWord* word1, SWord* word2) {
// search input word from tire tree
char pre[512];
memcpy(pre, word2->word, word2->len);
pre[word2->len] = 0;
char* str = NULL;
if (word1->type == WT_VAR_ALLTABLE) {
// ALL_TABLE
str = tireSearchWord(WT_VAR_STABLE, pre);
if (str == NULL) {
str = tireSearchWord(WT_VAR_TABLE, pre);
if(str == NULL)
return false;
}
} else {
// OTHER
str = tireSearchWord(word1->type, pre);
if (str == NULL) {
// not found or word1->type variable list not obtain from server, return not match
return false;
}
}
// free previous malloc
if(word1->free && word1->word) {
free(word1->word);
}
// save
word1->word = str;
word1->len = strlen(str);
word1->free = true; // need free
return true;
}
//
// ------------------- match words --------------------------
//
// compare command cmd1 come from shellCommands , cmd2 come from user input
int32_t compareCommand(SWords * cmd1, SWords * cmd2) {
SWord * word1 = cmd1->head;
SWord * word2 = cmd2->head;
if (word1 == NULL || word2 == NULL) {
return -1;
}
for (int32_t i = 0; i < cmd1->count; i++) {
if (word1->type == WT_TEXT) {
// WT_TEXT match
if (word1->len == word2->len) {
if (strncasecmp(word1->word, word2->word, word1->len) != 0)
return -1;
} else if (word1->len < word2->len) {
return -1;
} else {
// word1->len > word2->len
if (strncasecmp(word1->word, word2->word, word2->len) == 0) {
cmd1->matchIndex = i;
cmd1->matchLen = word2->len;
return i;
} else {
return -1;
}
}
} else {
// WT_VAR auto match any one word
if (word2->next == NULL) { // input words last one
if (matchVarWord(word1, word2)) {
cmd1->matchIndex = i;
cmd1->matchLen = word2->len;
varMode = true;
return i;
}
return -1;
}
}
// move next
word1 = word1->next;
word2 = word2->next;
if (word1 == NULL || word2 == NULL) {
return -1;
}
}
return -1;
}
// match command
SWords * matchCommand(SWords * input, bool continueSearch) {
int32_t count = SHELL_COMMAND_COUNT();
for (int32_t i = 0; i < count; i ++) {
SWords * shellCommand = shellCommands + i;
if (continueSearch && lastMatchIndex != -1 && i <= lastMatchIndex) {
// new match must greate than lastMatchIndex
if (varMode && i == lastMatchIndex) {
// do nothing, var match on lastMatchIndex
} else {
continue;
}
}
// command is large
if (input->count > shellCommand->count ) {
continue;
}
// compare
int32_t index = compareCommand(shellCommand, input);
if (index != -1) {
if (firstMatchIndex == -1)
firstMatchIndex = i;
curMatchIndex = i;
return &shellCommands[i];
}
}
// not match
return NULL;
}
//
// ------------------- print screen --------------------------
//
// delete char count
void deleteCount(Command * cmd, int count) {
int size = 0;
int width = 0;
clearScreen(cmd->endOffset + prompt_size, cmd->screenOffset + prompt_size);
// loop delete
while (--count >= 0 && cmd->cursorOffset > 0) {
getPrevCharSize(cmd->command, cmd->cursorOffset, &size, &width);
memmove(cmd->command + cmd->cursorOffset - size, cmd->command + cmd->cursorOffset,
cmd->commandSize - cmd->cursorOffset);
cmd->commandSize -= size;
cmd->cursorOffset -= size;
cmd->screenOffset -= width;
cmd->endOffset -= width;
}
}
// show screen
void printScreen(TAOS * con, Command * cmd, SWords * match) {
// modify Command
if (firstMatchIndex == -1 || curMatchIndex == -1) {
// no match
return ;
}
// first tab press
const char * str = NULL;
int strLen = 0;
if (firstMatchIndex == curMatchIndex && lastWordBytes == -1) {
// first press tab
SWord * word = MATCH_WORD(match);
str = word->word + match->matchLen;
strLen = word->len - match->matchLen;
lastMatchIndex = firstMatchIndex;
lastWordBytes = word->len;
} else {
if (lastWordBytes == -1)
return ;
deleteCount(cmd, lastWordBytes);
SWord * word = MATCH_WORD(match);
str = word->word;
strLen = word->len;
// set current to last
lastMatchIndex = curMatchIndex;
lastWordBytes = word->len;
}
// insert new
insertChar(cmd, (char *)str, strLen);
}
// main key press tab , matched return true else false
bool firstMatchCommand(TAOS * con, Command * cmd) {
// parse command
SWords* input = (SWords *)malloc(sizeof(SWords));
memset(input, 0, sizeof(SWords));
input->source = cmd->command;
input->source_len = cmd->commandSize;
parseCommand(input, false);
// if have many , default match first, if press tab again , switch to next
curMatchIndex = -1;
lastMatchIndex = -1;
SWords * match = matchCommand(input, true);
if (match == NULL) {
// not match , nothing to do
freeCommand(input);
free(input);
return false;
}
// print to screen
printScreen(con, cmd, match);
freeCommand(input);
free(input);
return true;
}
// create input source
void createInputFromFirst(SWords* input, SWords * firstMatch) {
//
// if next pressTabKey , input context come from firstMatch, set matched length with source_len
//
input->source = (char*)malloc(1024);
memset((void* )input->source, 0, 1024);
SWord * word = firstMatch->head;
// source_len = full match word->len + half match with firstMatch->matchLen
for (int i = 0; i < firstMatch->matchIndex && word; i++) {
// combine source from each word
strncpy(input->source + input->source_len, word->word, word->len);
strcat(input->source, " "); // append blank splite
input->source_len += word->len + 1; // 1 is blank length
// move next
word = word->next;
}
// appand half matched word for last
if (word) {
strncpy(input->source + input->source_len, word->word, firstMatch->matchLen);
input->source_len += firstMatch->matchLen;
}
}
// user press Tabkey again is named next , matched return true else false
bool nextMatchCommand(TAOS * con, Command * cmd, SWords * firstMatch) {
if (firstMatch == NULL || firstMatch->head == NULL) {
return false;
}
SWords* input = (SWords *)malloc(sizeof(SWords));
memset(input, 0, sizeof(SWords));
// create input from firstMatch
createInputFromFirst(input, firstMatch);
// parse input
parseCommand(input, false);
// if have many , default match first, if press tab again , switch to next
SWords * match = matchCommand(input, true);
if (match == NULL) {
// if not match , reset all index
firstMatchIndex = -1;
curMatchIndex = -1;
match = matchCommand(input, false);
if (match == NULL) {
freeCommand(input);
if (input->source)
free(input->source);
free(input);
return false;
}
}
// print to screen
printScreen(con, cmd, match);
// free
if (input->source) {
free(input->source);
input->source = NULL;
}
freeCommand(input);
free(input);
return true;
}
// fill with type
bool fillWithType(TAOS * con, Command * cmd, char* pre, int type) {
// get type
STire* tire = tires[type];
char* str = matchNextPrefix(tire, pre);
if (str == NULL) {
return false;
}
// need insert part string
char * part = str + strlen(pre);
// show
int count = strlen(part);
insertChar(cmd, part, count);
cntDel = count; // next press tab delete current append count
free(str);
return true;
}
// fill with type
bool fillTableName(TAOS * con, Command * cmd, char* pre) {
// search stable and table
char * str = tireSearchWord(WT_VAR_STABLE, pre);
if (str == NULL) {
str = tireSearchWord(WT_VAR_TABLE, pre);
if(str == NULL)
return false;
}
// need insert part string
char * part = str + strlen(pre);
// delete autofill count last append
if(cntDel > 0) {
deleteCount(cmd, cntDel);
cntDel = 0;
}
// show
int count = strlen(part);
insertChar(cmd, part, count);
cntDel = count; // next press tab delete current append count
free(str);
return true;
}
//
// find last word from sql select clause
// example :
// 1 select cou -> press tab select count(
// 2 select count(*),su -> select count(*), sum(
// 3 select count(*), su -> select count(*), sum(
//
char * lastWord(char * p) {
// get near from end revert find ' ' and ','
char * p1 = strrchr(p, ' ');
char * p2 = strrchr(p, ',');
if (p1 && p2) {
return MAX(p1, p2) + 1;
} else if (p1) {
return p1 + 1;
} else if(p2) {
return p2 + 1;
} else {
return p;
}
}
bool fieldsInputEnd(char* sql) {
// not in '()'
char* p1 = strrchr(sql, '(');
char* p2 = strrchr(sql, ')');
if (p1 && p2 == NULL) {
// like select count( ' '
return false;
} else if (p1 && p2 && p1 > p2) {
// like select sum(age), count( ' '
return false;
}
// not in ','
char * p3 = strrchr(sql, ',');
char * p = p3;
// like select ts, age,' '
if (p) {
++p;
bool allBlank = true; // after last ',' all char is blank
int cnt = 0; // blank count , like ' ' as one blank
char * plast = NULL; // last blank position
while(*p) {
if (*p == ' ') {
plast = p;
cnt ++;
} else {
allBlank = false;
}
++p;
}
// any one word is not blank
if(allBlank) {
return false;
}
// like 'select count(*),sum(age) fr' need return true
if (plast && plast > p3 && p2 > p1 && plast > p2 && p1 > p3) {
return true;
}
// if last char not ' ', then not end field, like 'select count(*), su' can fill sum(
if(sql[strlen(sql)-1] != ' ' && cnt <= 1) {
return false;
}
}
char * p4 = strrchr(sql, ' ');
if(p4 == NULL) {
// only one word
return false;
}
return true;
}
// need insert from
bool needInsertFrom(char * sql, int len) {
// last is blank
if(sql[len-1] != ' ') {
// insert from keyword
return false;
}
// select fields input is end
if (!fieldsInputEnd(sql)) {
return false;
}
// can insert from keyword
return true;
}
bool matchSelectQuery(TAOS * con, Command * cmd) {
// if continue press Tab , delete bytes by previous autofill
if (cntDel > 0) {
deleteCount(cmd, cntDel);
cntDel = 0;
}
// match select ...
int len = cmd->commandSize;
char * p = cmd->command;
// remove prefix blank
while (p[0] == ' ' && len > 0) {
p++;
len--;
}
// special range
if(len < 7 || len > 512) {
return false;
}
// select and from
if(strncasecmp(p, "select ", 7) != 0) {
// not select query clause
return false;
}
p += 7;
len -= 7;
char* ps = p = strndup(p, len);
// union all
char * p1;
do {
p1 = strstr(p, UNION_ALL);
if(p1) {
p = p1 + strlen(UNION_ALL);
}
} while (p1);
char * from = strstr(p, " from ");
//last word , maybe empty string or some letters of a string
char * last = lastWord(p);
bool ret = false;
if (from == NULL) {
bool fieldEnd = fieldsInputEnd(p);
// cheeck fields input end then insert from keyword
if (fieldEnd && p[len-1] == ' ') {
insertChar(cmd, "from", 4);
free(ps);
return true;
}
// fill funciton
if(fieldEnd) {
// fields is end , need match keyword
ret = fillWithType(con, cmd, last, WT_VAR_KEYWORD);
} else {
ret = fillWithType(con, cmd, last, WT_VAR_FUNC);
}
free(ps);
return ret;
}
// have from
char * blank = strstr(from + 6, " ");
if (blank == NULL) {
// no table name, need fill
ret = fillTableName(con, cmd, last);
} else {
ret = fillWithType(con, cmd, last, WT_VAR_KEYWORD);
}
free(ps);
return ret;
}
// if is input create fields or tags area, return true
bool isCreateFieldsArea(char * p) {
char * left = strrchr(p, '(');
if (left == NULL) {
// like 'create table st'
return false;
}
char * right = strrchr(p, ')');
if(right == NULL) {
// like 'create table st( '
return true;
}
if (left > right) {
// like 'create table st( ts timestamp, age int) tags(area '
return true;
}
return false;
}
bool matchCreateTable(TAOS * con, Command * cmd) {
// if continue press Tab , delete bytes by previous autofill
if (cntDel > 0) {
deleteCount(cmd, cntDel);
cntDel = 0;
}
// match select ...
int len = cmd->commandSize;
char * p = cmd->command;
// remove prefix blank
while (p[0] == ' ' && len > 0) {
p++;
len--;
}
// special range
if(len < 7 || len > 1024) {
return false;
}
// select and from
if(strncasecmp(p, "create table ", 13) != 0) {
// not select query clause
return false;
}
p += 13;
len -= 13;
char* ps = strndup(p, len);
bool ret = false;
char * last = lastWord(ps);
// check in create fields or tags input area
if (isCreateFieldsArea(ps)) {
ret = fillWithType(con, cmd, last, WT_VAR_DATATYPE);
}
// tags
if (!ret) {
// find only one ')' , can insert tags
char * p1 = strchr(ps, ')');
if (p1) {
if(strchr(p1 + 1, ')') == NULL && strstr(p1 + 1, "tags") == NULL) {
// can insert tags keyword
ret = fillWithType(con, cmd, last, WT_VAR_KEYTAGS);
}
}
}
free(ps);
return ret;
}
bool matchOther(TAOS * con, Command * cmd) {
int len = cmd->commandSize;
char* p = cmd->command;
if (p[len - 1] == '\\') {
// append '\G'
char a[] = "G;";
insertChar(cmd, a, 2);
return true;
}
return false;
}
// main key press tab
void pressTabKey(TAOS * con, Command * cmd) {
// check
if (cmd->commandSize == 0) {
// empty
showHelp();
showOnScreen(cmd);
return ;
}
// save connection to global
varCon = con;
varCmd = cmd;
bool matched = false;
// manual match like create table st( ...
matched = matchCreateTable(con, cmd);
if (matched)
return ;
// shellCommands match
if (firstMatchIndex == -1) {
matched = firstMatchCommand(con, cmd);
} else {
matched = nextMatchCommand(con, cmd, &shellCommands[firstMatchIndex]);
}
if (matched)
return ;
// NOT MATCHED ANYONE
// match other like '\G' ...
matched = matchOther(con, cmd);
if (matched)
return ;
// manual match like select * from ...
matched = matchSelectQuery(con, cmd);
if (matched)
return ;
return ;
}
// press othr key
void pressOtherKey(char c) {
// reset global variant
firstMatchIndex = -1;
lastMatchIndex = -1;
curMatchIndex = -1;
lastWordBytes = -1;
// var names
cursorVar = -1;
varMode = false;
waitAutoFill = false;
cntDel = 0;
if (lastMatch) {
freeMatch(lastMatch);
lastMatch = NULL;
}
}
// put name into name, return name length
int getWordName(char* p, char * name, int nameLen) {
//remove prefix blank
while (*p == ' ') {
p++;
}
// get databases name;
int i = 0;
while(p[i] != 0 && i < nameLen - 1) {
name[i] = p[i];
i++;
if(p[i] == ' ' || p[i] == ';'|| p[i] == '(') {
// name end
break;
}
}
name[i] = 0;
return i;
}
// deal use db, if have 'use' return true
bool dealUseDB(char * sql) {
// check use keyword
if(strncasecmp(sql, "use ", 4) != 0) {
return false;
}
char db[256];
char *p = sql + 4;
if (getWordName(p, db, sizeof(db)) == 0) {
// no name , return
return true;
}
// dbName is previous use open db name
if (strcasecmp(db, dbName) == 0) {
// same , no need switch
return true;
}
// switch new db
pthread_mutex_lock(&tiresMutex);
// STABLE set null
STire* tire = tires[WT_VAR_STABLE];
tires[WT_VAR_STABLE] = NULL;
if(tire) {
freeTire(tire);
}
// TABLE set null
tire = tires[WT_VAR_TABLE];
tires[WT_VAR_TABLE] = NULL;
if(tire) {
freeTire(tire);
}
// save
strcpy(dbName, db);
pthread_mutex_unlock(&tiresMutex);
return true;
}
// deal create, if have 'create' return true
bool dealCreateCommand(char * sql) {
// check keyword
if(strncasecmp(sql, "create ", 7) != 0) {
return false;
}
char name[1024];
char *p = sql + 7;
if (getWordName(p, name, sizeof(name)) == 0) {
// no name , return
return true;
}
int type = -1;
// dbName is previous use open db name
if (strcasecmp(name, "database") == 0) {
type = WT_VAR_DBNAME;
} else if (strcasecmp(name, "table") == 0) {
if(strstr(sql, " tags") != NULL && strstr(sql, " using ") == NULL)
type = WT_VAR_STABLE;
else
type = WT_VAR_TABLE;
} else if (strcasecmp(name, "user") == 0) {
type = WT_VAR_USERNAME;
} else {
// no match , return
return true;
}
// move next
p += strlen(name);
// get next word , that is table name
if (getWordName(p, name, sizeof(name)) == 0) {
// no name , return
return true;
}
// switch new db
pthread_mutex_lock(&tiresMutex);
// STABLE set null
STire* tire = tires[type];
if(tire) {
insertWord(tire, name);
}
pthread_mutex_unlock(&tiresMutex);
return true;
}
// deal create, if have 'drop' return true
bool dealDropCommand(char * sql) {
// check keyword
if(strncasecmp(sql, "drop ", 5) != 0) {
return false;
}
char name[1024];
char *p = sql + 5;
if (getWordName(p, name, sizeof(name)) == 0) {
// no name , return
return true;
}
int type = -1;
// dbName is previous use open db name
if (strcasecmp(name, "database") == 0) {
type = WT_VAR_DBNAME;
} else if (strcasecmp(name, "table") == 0) {
type = WT_VAR_ALLTABLE;
} else if (strcasecmp(name, "dnode") == 0) {
type = WT_VAR_DNODEID;
} else if (strcasecmp(name, "user") == 0) {
type = WT_VAR_USERNAME;
} else {
// no match , return
return true;
}
// move next
p += strlen(name);
// get next word , that is table name
if (getWordName(p, name, sizeof(name)) == 0) {
// no name , return
return true;
}
// switch new db
pthread_mutex_lock(&tiresMutex);
// STABLE set null
if(type == WT_VAR_ALLTABLE) {
bool del = false;
// del in stable
STire* tire = tires[WT_VAR_STABLE];
if(tire)
del = deleteWord(tire, name);
// del in table
if(!del) {
tire = tires[WT_VAR_TABLE];
if(tire)
del = deleteWord(tire, name);
}
} else {
// OTHER TYPE
STire* tire = tires[type];
if(tire)
deleteWord(tire, name);
}
pthread_mutex_unlock(&tiresMutex);
return true;
}
// callback autotab module after shell sql execute
void callbackAutoTab(char* sqlstr, TAOS* pSql, bool usedb) {
char * sql = sqlstr;
// remove prefix blank
while (*sql == ' ') {
sql++;
}
if(dealUseDB(sql)) {
// change to new db
return ;
}
// create command add name to autotab
if(dealCreateCommand(sql)) {
return ;
}
// drop command remove name from autotab
if(dealDropCommand(sql)) {
return ;
}
return ;
}
src/kit/shell/src/shellCommand.c
浏览文件 @ f017c4dd
...@@ -79,8 +79,11 @@ void insertChar(Command *cmd, char *c, int size) { ...@@ -79,8 +79,11 @@ void insertChar(Command *cmd, char *c, int size) {
/* update the values */ /* update the values */
cmd->commandSize += size; cmd->commandSize += size;
cmd->cursorOffset += size; cmd->cursorOffset += size;
cmd->screenOffset += wcwidth(wc); for (int i = 0; i < size; i++) {
cmd->endOffset += wcwidth(wc); mbtowc(&wc, c + i, size);
cmd->screenOffset += wcwidth(wc);
cmd->endOffset += wcwidth(wc);
}
showOnScreen(cmd); showOnScreen(cmd);
} }
...@@ -179,6 +182,16 @@ void positionCursorHome(Command *cmd) { ...@@ -179,6 +182,16 @@ void positionCursorHome(Command *cmd) {
} }
} }
void positionCursorMiddle(Command *cmd) {
if (cmd->endOffset > 0) {
clearScreen(cmd->endOffset + prompt_size, cmd->screenOffset + prompt_size);
cmd->cursorOffset = cmd->commandSize/2;
cmd->screenOffset = cmd->endOffset/2;
showOnScreen(cmd);
}
}
void positionCursorEnd(Command *cmd) { void positionCursorEnd(Command *cmd) {
assert(cmd->cursorOffset <= cmd->commandSize && cmd->endOffset >= cmd->screenOffset); assert(cmd->cursorOffset <= cmd->commandSize && cmd->endOffset >= cmd->screenOffset);
......
src/kit/shell/src/shellDarwin.c
浏览文件 @ f017c4dd
...@@ -22,6 +22,7 @@ ...@@ -22,6 +22,7 @@
#include "tkey.h" #include "tkey.h"
#include "tscLog.h" #include "tscLog.h"
#include "shellAuto.h"
#define OPT_ABORT 1 /* �Cabort */ #define OPT_ABORT 1 /* �Cabort */
...@@ -255,7 +256,12 @@ int32_t shellReadCommand(TAOS *con, char *command) { ...@@ -255,7 +256,12 @@ int32_t shellReadCommand(TAOS *con, char *command) {
utf8_array[k] = c; utf8_array[k] = c;
} }
insertChar(&cmd, utf8_array, count); insertChar(&cmd, utf8_array, count);
pressOtherKey(c);
} else if (c == TAB_KEY) {
// press TAB key
pressTabKey(con, &cmd);
} else if (c < '\033') { } else if (c < '\033') {
pressOtherKey(c);
// Ctrl keys. TODO: Implement ctrl combinations // Ctrl keys. TODO: Implement ctrl combinations
switch (c) { switch (c) {
case 1: // ctrl A case 1: // ctrl A
...@@ -301,6 +307,9 @@ int32_t shellReadCommand(TAOS *con, char *command) { ...@@ -301,6 +307,9 @@ int32_t shellReadCommand(TAOS *con, char *command) {
case 21: // Ctrl + U case 21: // Ctrl + U
clearLineBefore(&cmd); clearLineBefore(&cmd);
break; break;
case 23: // Ctrl + W;
positionCursorMiddle(&cmd);
break;
} }
} else if (c == '\033') { } else if (c == '\033') {
c = getchar(); c = getchar();
...@@ -377,9 +386,11 @@ int32_t shellReadCommand(TAOS *con, char *command) { ...@@ -377,9 +386,11 @@ int32_t shellReadCommand(TAOS *con, char *command) {
break; break;
} }
} else if (c == 0x7f) { } else if (c == 0x7f) {
pressOtherKey(c);
// press delete key // press delete key
backspaceChar(&cmd); backspaceChar(&cmd);
} else { } else {
pressOtherKey(c);
insertChar(&cmd, &c, 1); insertChar(&cmd, &c, 1);
} }
} }
......
src/kit/shell/src/shellEngine.c
浏览文件 @ f017c4dd
...@@ -27,6 +27,7 @@ ...@@ -27,6 +27,7 @@
#include "tglobal.h" #include "tglobal.h"
#include "tsclient.h" #include "tsclient.h"
#include "cJSON.h" #include "cJSON.h"
#include "shellAuto.h"
#include <regex.h> #include <regex.h>
...@@ -327,6 +328,12 @@ void shellRunCommandOnServer(TAOS *con, char command[]) { ...@@ -327,6 +328,12 @@ void shellRunCommandOnServer(TAOS *con, char command[]) {
fprintf(stdout, "Database changed.\n\n"); fprintf(stdout, "Database changed.\n\n");
fflush(stdout); fflush(stdout);
#ifndef WINDOWS
// call back auto tab module
callbackAutoTab(command, pSql, true);
#endif
atomic_store_64(&result, 0); atomic_store_64(&result, 0);
freeResultWithRid(oresult); freeResultWithRid(oresult);
return; return;
...@@ -365,6 +372,11 @@ void shellRunCommandOnServer(TAOS *con, char command[]) { ...@@ -365,6 +372,11 @@ void shellRunCommandOnServer(TAOS *con, char command[]) {
int num_rows_affacted = taos_affected_rows(pSql); int num_rows_affacted = taos_affected_rows(pSql);
et = taosGetTimestampUs(); et = taosGetTimestampUs();
printf("Query OK, %d of %d row(s) in database (%.6fs)\n", num_rows_affacted, num_rows_affacted, (et - st) / 1E6); printf("Query OK, %d of %d row(s) in database (%.6fs)\n", num_rows_affacted, num_rows_affacted, (et - st) / 1E6);
#ifndef WINDOWS
// call auto tab
callbackAutoTab(command, pSql, false);
#endif
} }
printf("\n"); printf("\n");
......
src/kit/shell/src/shellLinux.c
浏览文件 @ f017c4dd
...@@ -20,6 +20,7 @@ ...@@ -20,6 +20,7 @@
#include "shellCommand.h" #include "shellCommand.h"
#include "tkey.h" #include "tkey.h"
#include "tulog.h" #include "tulog.h"
#include "shellAuto.h"
#define OPT_ABORT 1 /* �Cabort */ #define OPT_ABORT 1 /* �Cabort */
...@@ -283,7 +284,12 @@ int32_t shellReadCommand(TAOS *con, char *command) { ...@@ -283,7 +284,12 @@ int32_t shellReadCommand(TAOS *con, char *command) {
utf8_array[k] = c; utf8_array[k] = c;
} }
insertChar(&cmd, utf8_array, count); insertChar(&cmd, utf8_array, count);
pressOtherKey(c);
} else if (c == TAB_KEY) {
// press TAB key
pressTabKey(con, &cmd);
} else if (c < '\033') { } else if (c < '\033') {
pressOtherKey(c);
// Ctrl keys. TODO: Implement ctrl combinations // Ctrl keys. TODO: Implement ctrl combinations
switch (c) { switch (c) {
case 1: // ctrl A case 1: // ctrl A
...@@ -329,8 +335,12 @@ int32_t shellReadCommand(TAOS *con, char *command) { ...@@ -329,8 +335,12 @@ int32_t shellReadCommand(TAOS *con, char *command) {
case 21: // Ctrl + U; case 21: // Ctrl + U;
clearLineBefore(&cmd); clearLineBefore(&cmd);
break; break;
case 23: // Ctrl + W;
positionCursorMiddle(&cmd);
break;
} }
} else if (c == '\033') { } else if (c == '\033') {
pressOtherKey(c);
c = (char)getchar(); c = (char)getchar();
switch (c) { switch (c) {
case '[': case '[':
...@@ -405,9 +415,11 @@ int32_t shellReadCommand(TAOS *con, char *command) { ...@@ -405,9 +415,11 @@ int32_t shellReadCommand(TAOS *con, char *command) {
break; break;
} }
} else if (c == 0x7f) { } else if (c == 0x7f) {
pressOtherKey(c);
// press delete key // press delete key
backspaceChar(&cmd); backspaceChar(&cmd);
} else { } else {
pressOtherKey(c);
insertChar(&cmd, &c, 1); insertChar(&cmd, &c, 1);
} }
} }
......
src/kit/shell/src/shellMain.c
浏览文件 @ f017c4dd
...@@ -17,6 +17,8 @@ ...@@ -17,6 +17,8 @@
#include "shell.h" #include "shell.h"
#include "tconfig.h" #include "tconfig.h"
#include "tnettest.h" #include "tnettest.h"
#include "shellCommand.h"
#include "shellAuto.h"
pthread_t pid; pthread_t pid;
static tsem_t cancelSem; static tsem_t cancelSem;
...@@ -162,10 +164,16 @@ int main(int argc, char* argv[]) { ...@@ -162,10 +164,16 @@ int main(int argc, char* argv[]) {
/* Get grant information */ /* Get grant information */
shellGetGrantInfo(args.con); shellGetGrantInfo(args.con);
#ifndef WINDOWS
shellAutoInit();
#endif
/* Loop to query the input. */ /* Loop to query the input. */
while (1) { while (1) {
pthread_create(&pid, NULL, shellLoopQuery, args.con); pthread_create(&pid, NULL, shellLoopQuery, args.con);
pthread_join(pid, NULL); pthread_join(pid, NULL);
} }
#ifndef WINDOWS
shellAutoExit();
#endif
} }
src/kit/shell/src/tire.c 0 → 100644
浏览文件 @ f017c4dd
/*
* Copyright (c) 2019 TAOS Data, Inc. <jhtao@taosdata.com>
*
* This program is free software: you can use, redistribute, and/or modify
* it under the terms of the GNU Affero General Public License, version 3
* or later ("AGPL"), as published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#define __USE_XOPEN
#include "os.h"
#include "tire.h"
// ----------- interface -------------
// create prefix search tree
STire* createTire(char type) {
STire* tire = malloc(sizeof(STire));
memset(tire, 0, sizeof(STire));
tire->ref = 1; // init is 1
tire->type = type;
tire->root.d = (STireNode **)calloc(CHAR_CNT, sizeof(STireNode *));
return tire;
}
// free tire node
void freeTireNode(STireNode* node) {
if (node == NULL)
return ;
// nest free sub node on array d
if(node->d) {
for (int i = 0; i < CHAR_CNT; i++) {
freeTireNode(node->d[i]);
}
tfree(node->d);
}
// free self
tfree(node);
}
// destroy prefix search tree
void freeTire(STire* tire) {
// free nodes
for (int i = 0; i < CHAR_CNT; i++) {
freeTireNode(tire->root.d[i]);
}
tfree(tire->root.d);
// free from list
StrName * item = tire->head;
while (item) {
StrName * next = item->next;
// free string
tfree(item->name);
// free node
tfree(item);
// move next
item = next;
}
tire->head = tire->tail = NULL;
// free tire
tfree(tire);
}
// insert a new word to list
bool insertToList(STire* tire, char* word) {
StrName * p = (StrName *)malloc(sizeof(StrName));
p->name = strdup(word);
p->next = NULL;
if(tire->head == NULL) {
tire->head = p;
tire->tail = p;
}else {
tire->tail->next = p;
tire->tail = p;
}
return true;
}
// insert a new word to tree
bool insertToTree(STire* tire, char* word, int len) {
int m = 0;
STireNode ** nodes = tire->root.d;
for (int i = 0; i < len; i++) {
m = word[i] - FIRST_ASCII;
if (m < 0 || m > CHAR_CNT) {
return false;
}
if (nodes[m] == NULL) {
// no pointer
STireNode* p = (STireNode* )tmalloc(sizeof(STireNode));
memset(p, 0, sizeof(STireNode));
nodes[m] = p;
if (i == len - 1) {
// is end
p->end = true;
break;
}
}
if (nodes[m]->d == NULL) {
// malloc d
nodes[m]->d = (STireNode **)calloc(CHAR_CNT, sizeof(STireNode *));
}
// move to next node
nodes = nodes[m]->d;
}
// add count
tire->count += 1;
return true;
}
// insert a new word
bool insertWord(STire* tire, char* word) {
int len = strlen(word);
if (len >= MAX_WORD_LEN) {
return false;
}
switch (tire->type) {
case TIRE_TREE:
return insertToTree(tire, word, len);
case TIRE_LIST:
return insertToList(tire, word);
default:
break;
}
return false;
}
// delete one word from list
bool deleteFromList(STire* tire, char* word) {
StrName * item = tire->head;
while (item) {
if (strcmp(item->name, word) == 0) {
// found, reset empty to delete
item->name[0] = 0;
}
// move next
item = item->next;
}
return true;
}
// delete one word from tree
bool deleteFromTree(STire* tire, char* word, int len) {
int m = 0;
bool del = false;
STireNode** nodes = tire->root.d;
for (int i = 0; i < len; i++) {
m = word[i] - FIRST_ASCII;
if (m < 0 || m >= CHAR_CNT) {
return false;
}
if (nodes[m] == NULL) {
// no found
return false;
} else {
// not null
if(i == len - 1) {
// this is last, only set end false , not free node
nodes[m]->end = false;
del = true;
break;
}
}
if(nodes[m]->d == NULL)
break;
// move to next node
nodes = nodes[m]->d;
}
// reduce count
if (del) {
tire->count -= 1;
}
return del;
}
// insert a new word
bool deleteWord(STire* tire, char* word) {
int len = strlen(word);
if (len >= MAX_WORD_LEN) {
return false;
}
switch (tire->type) {
case TIRE_TREE:
return deleteFromTree(tire, word, len);
case TIRE_LIST:
return deleteFromList(tire, word);
default:
break;
}
return false;
}
void addWordToMatch(SMatch* match, char* word){
// malloc new
SMatchNode* node = (SMatchNode* )tmalloc(sizeof(SMatchNode));
memset(node, 0, sizeof(SMatchNode));
node->word = strdup(word);
// append to match
if (match->head == NULL) {
match->head = match->tail = node;
} else {
match->tail->next = node;
match->tail = node;
}
match->count += 1;
}
// enum all words from node
void enumAllWords(STireNode** nodes, char* prefix, SMatch* match) {
STireNode * c;
char word[MAX_WORD_LEN];
int len = strlen(prefix);
for (int i = 0; i < CHAR_CNT; i++) {
c = nodes[i];
if (c == NULL) {
// chain end node
continue;
} else {
// combine word string
memset(word, 0, sizeof(word));
strcpy(word, prefix);
word[len] = FIRST_ASCII + i; // append current char
// chain middle node
if (c->end) {
// have end flag
addWordToMatch(match, word);
}
// nested call next layer
if (c->d)
enumAllWords(c->d, word, match);
}
}
}
// match prefix from list
void matchPrefixFromList(STire* tire, char* prefix, SMatch* match) {
StrName * item = tire->head;
int len = strlen(prefix);
while (item) {
if ( strncmp(item->name, prefix, len) == 0) {
// prefix matched
addWordToMatch(match, item->name);
}
// move next
item = item->next;
}
}
// match prefix words, if match is not NULL , put all item to match and return match
void matchPrefixFromTree(STire* tire, char* prefix, SMatch* match) {
SMatch* root = match;
int m = 0;
STireNode* c = 0;
int len = strlen(prefix);
if (len >= MAX_WORD_LEN) {
return;
}
STireNode** nodes = tire->root.d;
for (int i = 0; i < len; i++) {
m = prefix[i] - FIRST_ASCII;
if (m < 0 || m > CHAR_CNT) {
return;
}
// match
c = nodes[m];
if (c == NULL) {
// arrive end
break;
}
// previous items already matched
if (i == len - 1) {
// malloc match if not pass by param match
if (root == NULL) {
root = (SMatch* )tmalloc(sizeof(SMatch));
memset(root, 0, sizeof(SMatch));
strcpy(root->pre, prefix);
}
// prefix is match to end char
if (c->d)
enumAllWords(c->d, prefix, root);
} else {
// move to next node continue match
if(c->d == NULL)
break;
nodes = c->d;
}
}
// return
return ;
}
SMatch* matchPrefix(STire* tire, char* prefix, SMatch* match) {
if(match == NULL) {
match = (SMatch* )tmalloc(sizeof(SMatch));
memset(match, 0, sizeof(SMatch));
}
switch (tire->type) {
case TIRE_TREE:
matchPrefixFromTree(tire, prefix, match);
case TIRE_LIST:
matchPrefixFromList(tire, prefix, match);
default:
break;
}
// return if need
if (match->count == 0) {
freeMatch(match);
match = NULL;
}
return match;
}
// get all items from tires tree
void enumFromList(STire* tire, SMatch* match) {
StrName * item = tire->head;
while (item) {
if (item->name[0] != 0) {
// not delete
addWordToMatch(match, item->name);
}
// move next
item = item->next;
}
}
// get all items from tires tree
void enumFromTree(STire* tire, SMatch* match) {
char pre[2] ={0, 0};
STireNode* c;
// enum first layer
for (int i = 0; i < CHAR_CNT; i++) {
pre[0] = FIRST_ASCII + i;
// each node
c = tire->root.d[i];
if (c == NULL) {
// this branch no data
continue;
}
// this branch have data
if(c->end)
addWordToMatch(match, pre);
else
matchPrefix(tire, pre, match);
}
}
// get all items from tires tree
SMatch* enumAll(STire* tire) {
SMatch* match = (SMatch* )tmalloc(sizeof(SMatch));
memset(match, 0, sizeof(SMatch));
switch (tire->type) {
case TIRE_TREE:
enumFromTree(tire, match);
case TIRE_LIST:
enumFromList(tire, match);
default:
break;
}
// return if need
if (match->count == 0) {
freeMatch(match);
match = NULL;
}
return match;
}
// free match result
void freeMatchNode(SMatchNode* node) {
// first free next
if (node->next)
freeMatchNode(node->next);
// second free self
if (node->word)
free(node->word);
free(node);
}
// free match result
void freeMatch(SMatch* match) {
// first free next
if (match->head) {
freeMatchNode(match->head);
}
// second free self
free(match);
}
src/tsdb/src/tsdbMain.c
浏览文件 @ f017c4dd
...@@ -128,7 +128,7 @@ int tsdbCloseRepo(STsdbRepo *repo, int toCommit) { ...@@ -128,7 +128,7 @@ int tsdbCloseRepo(STsdbRepo *repo, int toCommit) {
tsdbStopStream(pRepo); tsdbStopStream(pRepo);
if(pRepo->pthread){ if(pRepo->pthread){
taosDestoryThread(pRepo->pthread); taosDestroyThread(pRepo->pthread);
pRepo->pthread = NULL; pRepo->pthread = NULL;
} }
......
src/tsdb/src/tsdbRead.c
浏览文件 @ f017c4dd
...@@ -986,7 +986,9 @@ static SMemRow getSMemRowInTableMem(STableCheckInfo* pCheckInfo, int32_t order, ...@@ -986,7 +986,9 @@ static SMemRow getSMemRowInTableMem(STableCheckInfo* pCheckInfo, int32_t order,
return rmem; return rmem;
} else { } else {
pCheckInfo->chosen = CHECKINFO_CHOSEN_BOTH; pCheckInfo->chosen = CHECKINFO_CHOSEN_BOTH;
*extraRow = rimem; if (extraRow) {
*extraRow = rimem;
}
return rmem; return rmem;
} }
} else { } else {
......
src/util/inc/tthread.h
浏览文件 @ f017c4dd
...@@ -26,7 +26,7 @@ extern "C" { ...@@ -26,7 +26,7 @@ extern "C" {
// create new thread // create new thread
pthread_t* taosCreateThread( void *(*__start_routine) (void *), void* param); pthread_t* taosCreateThread( void *(*__start_routine) (void *), void* param);
// destory thread // destory thread
bool taosDestoryThread(pthread_t* pthread); bool taosDestroyThread(pthread_t* pthread);
// thread running return true // thread running return true
bool taosThreadRunning(pthread_t* pthread); bool taosThreadRunning(pthread_t* pthread);
......
src/util/src/tthread.c
浏览文件 @ f017c4dd
...@@ -38,7 +38,7 @@ pthread_t* taosCreateThread( void *(*__start_routine) (void *), void* param) { ...@@ -38,7 +38,7 @@ pthread_t* taosCreateThread( void *(*__start_routine) (void *), void* param) {
} }
// destory thread // destory thread
bool taosDestoryThread(pthread_t* pthread) { bool taosDestroyThread(pthread_t* pthread) {
if(pthread == NULL) return false; if(pthread == NULL) return false;
if(taosThreadRunning(pthread)) { if(taosThreadRunning(pthread)) {
pthread_cancel(*pthread); pthread_cancel(*pthread);
......
src/vnode/src/vnodeMain.c
浏览文件 @ f017c4dd
...@@ -461,7 +461,7 @@ void vnodeStopWaitingThread(SVnodeObj* pVnode) { ...@@ -461,7 +461,7 @@ void vnodeStopWaitingThread(SVnodeObj* pVnode) {
if(loop == 0) { if(loop == 0) {
vInfo("vgId:%d :SDEL force kill thread to quit. pthread=%p pWrite=%p", pVnode->vgId, pWaitThread->pthread, pWaitThread->param); vInfo("vgId:%d :SDEL force kill thread to quit. pthread=%p pWrite=%p", pVnode->vgId, pWaitThread->pthread, pWaitThread->param);
// thread not stop , so need kill // thread not stop , so need kill
taosDestoryThread(pWaitThread->pthread); taosDestroyThread(pWaitThread->pthread);
// write msg need remove from queue // write msg need remove from queue
SVWriteMsg* pWrite = (SVWriteMsg* )pWaitThread->param; SVWriteMsg* pWrite = (SVWriteMsg* )pWaitThread->param;
if (pWrite) if (pWrite)
......
tests/parallel_test/cases.task
浏览文件 @ f017c4dd
# 20,,pytest,python3 insert/retentionpolicy.py change date time # 20,,pytest,python3 insert/retentionpolicy.py change date time
500,,docs-examples-test,./test_node.sh
299,,pytest,python3 test.py -f update/merge_commit_data-0.py 299,,pytest,python3 test.py -f update/merge_commit_data-0.py
290,,pytest,python3 test.py -f update/merge_commit_data.py 290,,pytest,python3 test.py -f update/merge_commit_data.py
241,,pytest,python3 test.py -f update/merge_commit_data2.py 241,,pytest,python3 test.py -f update/merge_commit_data2.py
...@@ -593,7 +594,6 @@ ...@@ -593,7 +594,6 @@
8,,develop-test,python3 ./test.py -f 5-taos-tools/taosdump/taosdumpTestTypeInt.py 8,,develop-test,python3 ./test.py -f 5-taos-tools/taosdump/taosdumpTestTypeInt.py
8,,develop-test,python3 ./test.py -f 5-taos-tools/taosdump/taosdumpTestTypeDouble.py 8,,develop-test,python3 ./test.py -f 5-taos-tools/taosdump/taosdumpTestTypeDouble.py
8,,pytest,python3 test.py -f update/update2.py 8,,pytest,python3 test.py -f update/update2.py
7,,docs-examples-test,./test_node.sh
7,,system-test,python3 ./test.py -f 5-taos-tools/taosbenchmark/taosdemoTestInsertWithJsonSml-otherPara.py 7,,system-test,python3 ./test.py -f 5-taos-tools/taosbenchmark/taosdemoTestInsertWithJsonSml-otherPara.py
7,,pytest,python3 test.py -f tools/taosdumpTest2.py 7,,pytest,python3 test.py -f tools/taosdumpTest2.py
7,,pytest,python3 test.py -f tools/taosdemoTestdatatype.py 7,,pytest,python3 test.py -f tools/taosdemoTestdatatype.py
......
tests/pytest/table/tagNameCaseSensitive.py
浏览文件 @ f017c4dd
...@@ -65,7 +65,7 @@ class TDTestCase: ...@@ -65,7 +65,7 @@ class TDTestCase:
tdSql.query("select * from `STB6`") tdSql.query("select * from `STB6`")
tdSql.checkRows(6) tdSql.checkRows(6)
tdSql.execute("delete from `STB6` where ` ` = 1 and ts = '2022-06-24 11:17:31.000'") tdSql.execute("delete from `STB6` where ` ` = 1 and ts = 1656040651000")
tdSql.checkAffectedRows(1) tdSql.checkAffectedRows(1)
tdSql.query("select * from `STB6`") tdSql.query("select * from `STB6`")
tdSql.checkRows(5) tdSql.checkRows(5)
...@@ -74,6 +74,10 @@ class TDTestCase: ...@@ -74,6 +74,10 @@ class TDTestCase:
tdSql.query("select * from `STB6`") tdSql.query("select * from `STB6`")
tdSql.checkRows(2) tdSql.checkRows(2)
tdSql.execute("alter table `STB6` add tag `1` int")
tdSql.execute("create table t1 using `STB6`(`1`) tags(1)")
tdSql.error("alter table t1 set tag 1=2222")
tdSql.error("alter table `STB6` add tag `` nchar(20)") tdSql.error("alter table `STB6` add tag `` nchar(20)")
def stop(self): def stop(self):
...@@ -82,4 +86,4 @@ class TDTestCase: ...@@ -82,4 +86,4 @@ class TDTestCase:
tdCases.addWindows(__file__, TDTestCase()) tdCases.addWindows(__file__, TDTestCase())
tdCases.addLinux(__file__, TDTestCase()) tdCases.addLinux(__file__, TDTestCase())
\ No newline at end of file
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或