[](https://bestpractices.coreinfrastructure.org/projects/4201)
English | [简体中文](README-CN.md) | [Learn more about TSDB](https://tdengine.com/tsdb/)
English | [简体中文](README-CN.md) | [TDengine Cloud](https://cloud.tdengine.com) | [Learn more about TSDB](https://tdengine.com/tsdb/)
# What is TDengine？
...
...
@@ -33,7 +33,7 @@ TDengine is an open source, high-performance, cloud native [time-series database
-**[Open Source](https://tdengine.com/tdengine/open-source-time-series-database/)**: TDengine’s core modules, including cluster feature, are all available under open source licenses. It has gathered 18.8k stars on GitHub. There is an active developer community, and over 139k running instances worldwide.
For a full list of TDengine competitive advantages, please [check here](https://tdengine.com/tdengine/)
For a full list of TDengine competitive advantages, please [check here](https://tdengine.com/tdengine/). The easiest way to experience TDengine is through [TDengine Cloud](https://cloud.tdengine.com).
@@ -23,8 +23,8 @@ The major features are listed below:
4.[Stream Processing](../develop/stream/): Not only is the continuous query is supported, but TDengine also supports event driven stream processing, so Flink or Spark is not needed for time-series data processing.
5.[Data Subscription](../develop/tmq/): Application can subscribe a table or a set of tables. API is the same as Kafka, but you can specify filter conditions.
6. Visualization
- Supports seamless integration with [Grafana](../third-party/grafana/) for visualization.
- Supports seamless integration with Google Data Studio.
- Supports seamless integration with [Grafana](../third-party/grafana/).
- Supports seamless integration with [Google Data Studio](../third-party/google-data-studio/).
7. Cluster
- Supports [cluster](../deployment/) with the capability of increasing processing power by adding more nodes.
- Supports [deployment on Kubernetes](../deployment/k8s/).
...
...
@@ -33,7 +33,7 @@ The major features are listed below:
- Provides [monitoring](../operation/monitor) on running instances of TDengine.
- Provides many ways to [import](../operation/import) and [export](../operation/export) data.
9. Tools
- Provides an interactive [Command-line Interface (CLI)](../reference/taos-shell) for management, maintenance and ad-hoc queries.
- Provides an interactive [Command Line Interface (CLI)](../reference/taos-shell) for management, maintenance and ad-hoc queries.
- Provides a tool [taosBenchmark](../reference/taosbenchmark/) for testing the performance of TDengine.
10. Programming
- Provides [connectors](../reference/connector/) for [C/C++](../reference/connector/cpp), [Java](../reference/connector/java), [Python](../reference/connector/python), [Go](../reference/connector/go), [Rust](../reference/connector/rust), [Node.js](../reference/connector/node) and other programming languages.
This document describes how to install TDengine in a Docker container and perform queries and inserts.
- The easiest way to explore TDengine is through [TDengine Cloud](http://cloud.tdengine.com).
- The easiest way to explore TDengine is through [TDengine Cloud](http://cloud.tdengine.com).
- To get started with TDengine in a non-containerized environment, see [Quick Install from Package](../../get-started/package).
- If you want to view the source code, build TDengine yourself, or contribute to the project, see the [TDengine GitHub repository](https://github.com/taosdata/TDengine).
## Run TDengine
If Docker is already installed on your computer, run the following command:
If Docker is already installed on your computer, pull the latest TDengine Docker container image:
The first column of the returned result set is the subtable name, and the subsequent columns are the tag columns.
If you already know the name of the tag column, you can use the following statement to get the value of the specified tag column.
```
taos> SELECT DISTINCT TBNAME, id FROM st1;
tbname | id |
===============================================
st1s1 | 1 |
st1s2 | 2 |
st1s3 | 3 |
Query OK, 3 rows in database (0.002891s)
```
It should be noted that DISTINCT and TBNAME in the SELECT statement are essential, and TDengine will optimize the statement according to them, so that it can return the tag value correctly and quickly even when there is no data or a lot of data.
- `INTERP` is used to get the value that matches the specified time slice from a column. If no such value exists an interpolation value will be returned based on `FILL` parameter.
- The input data of `INTERP` is the value of the specified column and a `where` clause can be used to filter the original data. If no `where` condition is specified then all original data is the input.
- The output time range of `INTERP` is specified by `RANGE(timestamp1,timestamp2)` parameter, with timestamp1<=timestamp2. timestamp1 is the starting point of the output time range and must be specified. timestamp2 is the ending point of the output time range and must be specified.
- The number of rows in the result set of `INTERP` is determined by the parameter `EVERY`. Starting from timestamp1, one interpolation is performed for every time interval specified `EVERY` parameter.
- The number of rows in the result set of `INTERP` is determined by the parameter `EVERY`. Starting from timestamp1, one interpolation is performed for every time interval specified `EVERY` parameter. The parameter `EVERY` must be an integer, with no quotes, with a time unit of: b(nanosecond), u(microsecond), a(millisecond)), s(second), m(minute), h(hour), d(day), or w(week). For example, `EVERY(500a)` will interpolate every 500 milliseconds.
- Interpolation is performed based on `FILL` parameter.
- `INTERP` can only be used to interpolate in single timeline. So it must be used with `partition by tbname` when it's used on a STable.
- Pseudo column `_irowts` can be used along with `INTERP` to return the timestamps associated with interpolation points(support after version 3.0.1.4).
@@ -136,19 +136,3 @@ The parameters that you can modify through this statement are the same as those
```sql
SHOWLOCALVARIABLES;
```
## Combine Vgroups
```sql
MERGEVGROUPvgroup_no1vgroup_no2;
```
If load and data are not properly balanced among vgroups due to the data in different tim lines having different characteristics, you can combine or separate vgroups.
## Separate Vgroups
```sql
SPLITVGROUPvgroup_no;
```
This statement creates a new vgroup and migrates part of the data from the original vgroup to the new vgroup with consistent hashing. During this process, the original vgroup can continue to provide services normally.
| 1 | vnodes | SMALLINT | Current number of vnodes on the dnode |
| 2 | vnodes | SMALLINT | Maximum number of vnodes on the dnode |
| 1 | vnodes | SMALLINT | Current number of vnodes on the dnode. It should be noted that `vnodes` is a TDengine keyword and needs to be escaped with ` when used as a column name. |
| 2 | support_vnodes | SMALLINT | Maximum number of vnodes on the dnode |
| 3 | status | BINARY(10) | Current status |
| 4 | note | BINARY(256) | Reason for going offline or other information |
| 5 | id | SMALLINT | Dnode ID |
...
...
@@ -49,16 +49,6 @@ Provides information about mnodes. Similar to SHOW MNODES.
| 4 | role_time | TIMESTAMP | Time at which the current role was assumed |
| 5 | create_time | TIMESTAMP | Creation time |
## INS_MODULES
Provides information about modules. Similar to SHOW MODULES.
Provides information about qnodes. Similar to SHOW QNODES.
...
...
@@ -88,33 +78,33 @@ Provides information about user-created databases. Similar to SHOW DATABASES.
| 1| name| BINARY(32)| Database name |
| 2 | create_time | TIMESTAMP | Creation time |
| 3 | ntables | INT | Number of standard tables and subtables (not including supertables) |
| 4 | vgroups | INT | Number of vgroups |
| 6 | replica | INT | Number of replicas |
| 7 | quorum | BINARY(3) | Strong consistency |
| 8 | duration | INT | Duration for storage of single files |
| 9 | keep | INT | Data retention period |
| 10 | buffer | INT | Write cache size per vnode, in MB |
| 11 | pagesize | INT | Page size for vnode metadata storage engine, in KB |
| 12 | pages | INT | Number of pages per vnode metadata storage engine |
| 13 | minrows | INT | Maximum number of records per file block |
| 14 | maxrows | INT | Minimum number of records per file block |
| 15 | comp | INT | Compression method |
| 16 | precision | BINARY(2) | Time precision |
| 4 | vgroups | INT | Number of vgroups. It should be noted that `vnodes` is a TDengine keyword and needs to be escaped with ` when used as a column name. |
| 6 | replica | INT | Number of replicas. It should be noted that `replica` is a TDengine keyword and needs to be escaped with ` when used as a column name. |
| 7 | strict | BINARY(3) | Strong consistency. It should be noted that `strict` is a TDengine keyword and needs to be escaped with ` when used as a column name. |
| 8 | duration | INT | Duration for storage of single files. It should be noted that `duration` is a TDengine keyword and needs to be escaped with ` when used as a column name. |
| 9 | keep | INT | Data retention period. It should be noted that `keep` is a TDengine keyword and needs to be escaped with ` when used as a column name. |
| 10 | buffer | INT | Write cache size per vnode, in MB. It should be noted that `buffer` is a TDengine keyword and needs to be escaped with ` when used as a column name. |
| 11 | pagesize | INT | Page size for vnode metadata storage engine, in KB. It should be noted that `pagesize` is a TDengine keyword and needs to be escaped with ` when used as a column name. |
| 12 | pages | INT | Number of pages per vnode metadata storage engine. It should be noted that `pages` is a TDengine keyword and needs to be escaped with ` when used as a column name. |
| 13 | minrows | INT | Maximum number of records per file block. It should be noted that `minrows` is a TDengine keyword and needs to be escaped with ` when used as a column name. |
| 14 | maxrows | INT | Minimum number of records per file block. It should be noted that `maxrows` is a TDengine keyword and needs to be escaped with ` when used as a column name. |
| 15 | comp | INT | Compression method. It should be noted that `comp` is a TDengine keyword and needs to be escaped with ` when used as a column name. |
| 16 | precision | BINARY(2) | Time precision. It should be noted that `precision` is a TDengine keyword and needs to be escaped with ` when used as a column name. |
| 17 | status | BINARY(10) | Current database status |
| 18 | retention | BINARY (60) | Aggregation interval and retention period |
| 19 | single_stable | BOOL | Whether the database can contain multiple supertables |
| 20 | cachemodel | BINARY(60) | Caching method for the newest data |
| 21 | cachesize | INT | Memory per vnode used for caching the newest data |
| 22 | wal_level | INT | WAL level |
| 23 | wal_fsync_period | INT | Interval at which WAL is written to disk |
| 24 | wal_retention_period | INT | WAL retention period |
| 25 | wal_retention_size | INT | Maximum WAL size |
| 26 | wal_roll_period | INT | WAL rotation period |
| 28 | stt_trigger | SMALLINT | The threshold for number of files to trigger file merging |
| 29 | table_prefix | SMALLINT | The prefix length in the table name that is ignored when distributing table to vnode based on table name |
| 30 | table_suffix | SMALLINT | The suffix length in the table name that is ignored when distributing table to vnode based on table name |
| 31 | tsdb_pagesize | INT | The page size for internal storage engine, its unit is KB |
| 18 | retentions | BINARY (60) | Aggregation interval and retention period. It should be noted that `retentions` is a TDengine keyword and needs to be escaped with ` when used as a column name. |
| 19 | single_stable | BOOL | Whether the database can contain multiple supertables. It should be noted that `single_stable` is a TDengine keyword and needs to be escaped with ` when used as a column name. |
| 20 | cachemodel | BINARY(60) | Caching method for the newest data. It should be noted that `cachemodel` is a TDengine keyword and needs to be escaped with ` when used as a column name. |
| 21 | cachesize | INT | Memory per vnode used for caching the newest data. It should be noted that `cachesize` is a TDengine keyword and needs to be escaped with ` when used as a column name. |
| 22 | wal_level | INT | WAL level. It should be noted that `wal_level` is a TDengine keyword and needs to be escaped with ` when used as a column name. |
| 23 | wal_fsync_period | INT | Interval at which WAL is written to disk. It should be noted that `wal_fsync_period` is a TDengine keyword and needs to be escaped with ` when used as a column name. |
| 24 | wal_retention_period | INT | WAL retention period. It should be noted that `wal_retention_period` is a TDengine keyword and needs to be escaped with ` when used as a column name. |
| 25 | wal_retention_size | INT | Maximum WAL size. It should be noted that `wal_retention_size` is a TDengine keyword and needs to be escaped with ` when used as a column name. |
| 26 | wal_roll_period | INT | WAL rotation period. It should be noted that `wal_roll_period` is a TDengine keyword and needs to be escaped with ` when used as a column name. |
| 27 | wal_segment_size | BIGINT | WAL file size. It should be noted that `wal_segment_size` is a TDengine keyword and needs to be escaped with ` when used as a column name. |
| 28 | stt_trigger | SMALLINT | The threshold for number of files to trigger file merging. It should be noted that `stt_trigger` is a TDengine keyword and needs to be escaped with ` when used as a column name. |
| 29 | table_prefix | SMALLINT | The prefix length in the table name that is ignored when distributing table to vnode based on table name. It should be noted that `table_prefix` is a TDengine keyword and needs to be escaped with ` when used as a column name. |
| 30 | table_suffix | SMALLINT | The suffix length in the table name that is ignored when distributing table to vnode based on table name. It should be noted that `table_suffix` is a TDengine keyword and needs to be escaped with ` when used as a column name. |
| 31 | tsdb_pagesize | INT | The page size for internal storage engine, its unit is KB. It should be noted that `tsdb_pagesize` is a TDengine keyword and needs to be escaped with ` when used as a column name. |
## INS_FUNCTIONS
...
...
@@ -123,8 +113,8 @@ Provides information about user-defined functions.
| 2 | comment | BINARY(255) | Function description |
| 3 | aggregate | INT | Whether the UDF is an aggregate function |
| 2 | comment | BINARY(255) | Function description. It should be noted that `comment` is a TDengine keyword and needs to be escaped with ` when used as a column name. |
| 3 | aggregate | INT | Whether the UDF is an aggregate function. It should be noted that `aggregate` is a TDengine keyword and needs to be escaped with ` when used as a column name. |
| 4 | output_type | BINARY(31) | Output data type |
| 5 | create_time | TIMESTAMP | Creation time |
| 6 | code_len | INT | Length of the source code |
...
...
@@ -153,12 +143,12 @@ Provides information about supertables.
| 2 | db_name | BINARY(64) | All databases in the supertable |
| 3 | create_time | TIMESTAMP | Creation time |
| 4 | columns | INT | Number of columns |
| 5 | tags | INT | Number of tags |
| 5 | tags | INT | Number of tags. It should be noted that `tags` is a TDengine keyword and needs to be escaped with ` when used as a column name. |
| 6 | last_update | TIMESTAMP | Last updated time |
| 8 | watermark | BINARY(64) | Window closing time. It should be noted that `watermark` is a TDengine keyword and needs to be escaped with ` when used as a column name. |
| 9 | max_delay | BINARY(64) | Maximum delay for pushing stream processing results. It should be noted that `max_delay` is a TDengine keyword and needs to be escaped with ` when used as a column name. |
| 10 | rollup | BINARY(128) | Rollup aggregate function. It should be noted that `rollup` is a TDengine keyword and needs to be escaped with ` when used as a column name. |
## INS_TABLES
...
...
@@ -173,7 +163,7 @@ Provides information about standard tables and subtables.
| 5 | stable_name | BINARY(192) | Supertable name |
| 6 | uid | BIGINT | Table ID |
| 7 | vgroup_id | INT | Vgroup ID |
| 8 | ttl | INT | Table time-to-live |
| 8 | ttl | INT | Table time-to-live. It should be noted that `ttl` is a TDengine keyword and needs to be escaped with ` when used as a column name. |
| 1 | version | BINARY(9) | Whether the deployment is a licensed or trial version |
| 2 | cpu_cores | BINARY(9) | CPU cores included in license |
| 3 | dnodes | BINARY(10) | Dnodes included in license |
| 4 | streams | BINARY(10) | Streams included in license |
| 5 | users | BINARY(10) | Users included in license |
| 6 | streams | BINARY(10) | Accounts included in license |
| 7 | storage | BINARY(21) | Storage space included in license |
| 8 | connections | BINARY(21) | Client connections included in license |
| 9 | databases | BINARY(11) | Databases included in license |
| 3 | dnodes | BINARY(10) | Dnodes included in license. It should be noted that `dnodes` is a TDengine keyword and needs to be escaped with ` when used as a column name. |
| 4 | streams | BINARY(10) | Streams included in license. It should be noted that `streams` is a TDengine keyword and needs to be escaped with ` when used as a column name. |
| 5 | users | BINARY(10) | Users included in license. It should be noted that `users` is a TDengine keyword and needs to be escaped with ` when used as a column name. |
| 6 | accounts | BINARY(10) | Accounts included in license. It should be noted that `accounts` is a TDengine keyword and needs to be escaped with ` when used as a column name. |
| 7 | storage | BINARY(21) | Storage space included in license. It should be noted that `storage` is a TDengine keyword and needs to be escaped with ` when used as a column name. |
| 8 | connections | BINARY(21) | Client connections included in license. It should be noted that `connections` is a TDengine keyword and needs to be escaped with ` when used as a column name. |
| 9 | databases | BINARY(11) | Databases included in license. It should be noted that `databases` is a TDengine keyword and needs to be escaped with ` when used as a column name. |
| 10 | speed | BINARY(9) | Write speed specified in license (data points per second) |
| 11 | querytime | BINARY(9) | Total query time specified in license |
| 12 | timeseries | BINARY(21) | Number of metrics included in license |
...
...
@@ -227,7 +217,7 @@ Provides information about vgroups.
| 3 | tables | INT | Tables in vgroup. It should be noted that `tables` is a TDengine keyword and needs to be escaped with ` when used as a column name. |
| 4 | status | BINARY(10) | Vgroup status |
| 5 | v1_dnode | INT | Dnode ID of first vgroup member |
| 6 | v1_status | BINARY(10) | Status of first vgroup member |
...
...
@@ -246,7 +236,7 @@ Provides system configuration information.
| 9 | trigger | INT | Method of triggering the result push (see stream processing documentation) |
| 8 | watermark | BIGINT | Watermark (see stream processing documentation). It should be noted that `watermark` is a TDengine keyword and needs to be escaped with ` when used as a column name. |
| 9 | trigger | INT | Method of triggering the result push (see stream processing documentation). It should be noted that `trigger` is a TDengine keyword and needs to be escaped with ` when used as a column name. |
@@ -32,7 +32,7 @@ We recommend using the latest version of `taospy`, regardless of the version of
### Preparation
1. Install Python. Python >= 3.6 is recommended. If Python is not available on your system, refer to the [Python BeginnersGuide](https://wiki.python.org/moin/BeginnersGuide/Download) to install it.
1. Install Python. Python >= 3.7 is recommended. If Python is not available on your system, refer to the [Python BeginnersGuide](https://wiki.python.org/moin/BeginnersGuide/Download) to install it.
2. Install [pip](https://pypi.org/project/pip/). In most cases, the Python installer comes with the pip utility. If not, please refer to [pip documentation](https://pip.pypa.io/en/stable/installation/) to install it.
If you use a native connection, you will also need to [Install Client Driver](/reference/connector#Install-Client-Driver). The client install package includes the TDengine client dynamic link library (`libtaos.so` or `taos.dll`) and the TDengine CLI.
...
...
@@ -121,7 +121,7 @@ Before establishing a connection with the connector, we recommend testing the co
Ensure that the TDengine instance is up and that the FQDN of the machines in the cluster (the FQDN defaults to hostname if you are starting a standalone version) can be resolved locally, by testing with the `ping` command.
Ensure that the TDengine instance is up and that the FQDN of the machines in the cluster (the FQDN defaults to hostname if you are starting a stand-alone version) can be resolved locally, by testing with the `ping` command.
taosBenchmark (formerly taosdemo ) is a tool for testing the performance of TDengine products. taosBenchmark can test the performance of TDengine's insert, query, and subscription functions and simulate large amounts of data generated by many devices. taosBenchmark can be configured to generate user defined databases, supertables, subtables, and the time series data to populate these for performance benchmarking. taosBenchmark is highly configurable and some of the configurations include the time interval for inserting data, the number of working threads and the capability to insert disordered data. The installer provides taosdemo as a soft link to taosBenchmark for compatibility with past users.
...
...
@@ -23,7 +23,7 @@ There are two ways to install taosBenchmark:
TaosBenchmark needs to be executed on the terminal of the operating system, it supports two configuration methods: [Command-line arguments](#command-line-arguments-in-detail) and [JSON configuration file](#configuration-file-parameters-in-detail). These two methods are mutually exclusive. Users can use `-f <json file>` to specify a configuration file. When running taosBenchmark with command-line arguments to control its behavior, users should use other parameters for configuration, but not the `-f` parameter. In addition, taosBenchmark offers a special way of running without parameters.
taosBenchmark supports the complete performance testing of TDengine by providing functionally to write, query, and subscribe. These three functions are mutually exclusive, users can only select one of them each time taosBenchmark runs. The query and subscribe functionalities are only configurable using a json configuration file by specifying the parameter `filetype`, while write can be performed through both the command-line and a configuration file. If you want to test the performance of queries or data subscriptionm configure taosBenchmark with the configuration file. You can modify the value of the `filetype` parameter to specify the function that you want to test.
taosBenchmark supports the complete performance testing of TDengine by providing functionally to write, query, and subscribe. These three functions are mutually exclusive, users can only select one of them each time taosBenchmark runs. The query and subscribe functionalities are only configurable using a json configuration file by specifying the parameter `filetype`, while write can be performed through both the command-line and a configuration file. If you want to test the performance of queries or data subscription configure taosBenchmark with the configuration file. You can modify the value of the `filetype` parameter to specify the function that you want to test.
**Make sure that the TDengine cluster is running correctly before running taosBenchmark. **
...
...
@@ -334,13 +334,13 @@ The configuration parameters for specifying super table tag columns and data col
-**name** : The name of the column, if used together with count, e.g. "name": "current", "count":3, then the names of the 3 columns are current, current_2. current_3.
-**min**: The minimum value of the column/label of the data type.
-**min**: The minimum value of the column/label of the data type. The generated value will equal or large than the minimum value.
-**max**: The maximum value of the column/label of the data type.
-**max**: The maximum value of the column/label of the data type. The generated value will less than the maxium value.
-**values**: The value field of the nchar/binary column/label, which will be chosen randomly from the values.
-**sma**: Insert the column into the BSMA. Enter `yes` or `no`. The default is `no`.
-**sma**: Insert the column into the SMA. Enter `yes` or `no`. The default is `no`.
pRes=taos_query(pConn,"create stable if not exists st1 (ts timestamp, k int) tags(a int)");
pRes=taos_query(pConn,"create stable if not exists st1 (ts timestamp, k int, j varchar(20)) tags(a varchar(20))");
if(taos_errno(pRes)!=0){
printf("failed to create super table st1, reason:%s\n",taos_errstr(pRes));
return-1;
}
taos_free_result(pRes);
pRes=taos_query(pConn,"create table if not exists tu1 using st1 tags(1)");
pRes=taos_query(pConn,"create table if not exists tu1 using st1 tags('c1')");
if(taos_errno(pRes)!=0){
printf("failed to create child table tu1, reason:%s\n",taos_errstr(pRes));
return-1;
}
taos_free_result(pRes);
pRes=taos_query(pConn,"create table if not exists tu2 using st1 tags(2)");
pRes=taos_query(pConn,"create table if not exists tu2 using st1 tags('c2')");
if(taos_errno(pRes)!=0){
printf("failed to create child table tu2, reason:%s\n",taos_errstr(pRes));
return-1;
}
taos_free_result(pRes);
pRes=taos_query(pConn,"create table if not exists tu3 using st1 tags(3)");
pRes=taos_query(pConn,"create table if not exists tu3 using st1 tags('c3')");
if(taos_errno(pRes)!=0){
printf("failed to create child table tu3, reason:%s\n",taos_errstr(pRes));
return-1;
...
...
@@ -96,7 +96,8 @@ int32_t create_stream() {
taos_free_result(pRes);
pRes=taos_query(pConn,
"create stream stream1 trigger at_once watermark 10s into outstb as select _wstart start, avg(k) from st1 partition by tbname interval(10s)");
/*"create stream stream1 trigger at_once watermark 10s into outstb as select _wstart start, avg(k) from st1 partition by tbname interval(10s)");*/
"create stream stream2 into outstb subtable(concat(concat(concat('prefix_', tname), '_suffix_'), cast(k1 as varchar(20)))) as select _wstart wstart, avg(k) from st1 partition by tbname tname, a k1 interval(10s);");
if(taos_errno(pRes)!=0){
printf("failed to create stream stream1, reason:%s\n",taos_errstr(pRes));
