@@ -30,7 +30,7 @@ taosAdapter provides the following features.
### Install taosAdapter
taosAdapter has been part of TDengine server software since TDengine v2.4.0.0. If you use the TDengine server, you don't need additional steps to install taosAdapter. You can download taosAdapter from [TDengine official website](https://tdengine.com/all-downloads/) to download the TDengine server installation package (taosAdapter is included in v2.4.0.0 and later version). If you need to deploy taosAdapter separately on another server other than the TDengine server, you should install the full TDengine server package on that server to install taosAdapter. If you need to build taosAdapter from source code, you can refer to the [Building taosAdapter](https://github.com/taosdata/taosadapter/blob/develop/BUILD.md) documentation.
taosAdapter has been part of TDengine server software since TDengine v2.4.0.0. If you use the TDengine server, you don't need additional steps to install taosAdapter. You can download taosAdapter from [TDengine official website](https://tdengine.com/all-downloads/) to download the TDengine server installation package (taosAdapter is included in v2.4.0.0 and later version). If you need to deploy taosAdapter separately on another server other than the TDengine server, you should install the full TDengine server package on that server to install taosAdapter. If you need to build taosAdapter from source code, you can refer to the [Building taosAdapter](https://github.com/taosdata/taosadapter/blob/3.0/BUILD.md) documentation.
--log.sqlRotationCount uint record sql log rotation count. Env "TAOS_ADAPTER_LOG_SQL_ROTATION_COUNT"(default 2)
--log.sqlRotationSize string record sql log rotation size(KB MB GB), must be a positive integer. Env "TAOS_ADAPTER_LOG_SQL_ROTATION_SIZE"(default "1GB")
--log.sqlRotationTime duration record sql log rotation time. Env "TAOS_ADAPTER_LOG_SQL_ROTATION_TIME"(default 24h0m0s)
--statsd.allowPendingMessages int statsd allow pending messages. Env "TAOS_ADAPTER_STATSD_ALLOW_PENDING_MESSAGES"(default 50000)
--statsd.db string statsd db name. Env "TAOS_ADAPTER_STATSD_DB"(default "statsd")
--statsd.deleteCounters statsd delete counter cache after gather. Env "TAOS_ADAPTER_STATSD_DELETE_COUNTERS"(default true)
...
...
@@ -149,11 +151,12 @@ You do not need to care about these configurations if you do not make interface
For details on the CORS protocol, please refer to: [https://www.w3.org/wiki/CORS_Enabled](https://www.w3.org/wiki/CORS_Enabled) or [https://developer.mozilla.org/zh-CN/docs/Web/HTTP/CORS](https://developer.mozilla.org/zh-CN/docs/Web/HTTP/CORS).
See [example/config/taosadapter.toml](https://github.com/taosdata/taosadapter/blob/develop/example/config/taosadapter.toml) for sample configuration files.
See [example/config/taosadapter.toml](https://github.com/taosdata/taosadapter/blob/3.0/example/config/taosadapter.toml) for sample configuration files.
## Feature List
- Compatible with RESTful interfaces [REST API](/reference/rest-api/)
- Compatible with OpenTSDB JSON and telnet format writes
...
...
@@ -176,13 +179,7 @@ See [example/config/taosadapter.toml](https://github.com/taosdata/taosadapter/bl
### TDengine RESTful interface
You can use any client that supports the http protocol to write data to or query data from TDengine by accessing the REST interface address `http://<fqdn>:6041/<APIEndPoint>`. See the [official documentation](/reference/connector#restful) for details. The following EndPoint is supported.
```text
/rest/sql
/rest/sqlt
/rest/sqlutc
```
You can use any client that supports the http protocol to write data to or query data from TDengine by accessing the REST interface address `http://<fqdn>:6041/rest/sql`. See the [official documentation](/reference/rest-api/) for details.
### InfluxDB
...
...
@@ -203,7 +200,7 @@ Note: InfluxDB token authorization is not supported at present. Only Basic autho
### OpenTSDB
You can use any client that supports the http protocol to access the RESTful interface address `http://<fqdn>:6041/<APIEndPoint>` to write data in OpenTSDB compatible format to TDengine.
You can use any client that supports the http protocol to access the RESTful interface address `http://<fqdn>:6041/<APIEndPoint>` to write data in OpenTSDB compatible format to TDengine. The EndPoint is as follows:
```text
/opentsdb/v1/put/json/<db>
...
...
@@ -254,7 +251,7 @@ HTTP response content.
Stops processing all write and query requests when the `pauseAllMemoryThreshold` threshold is exceeded.
HTTP response: code 503
HTTP response content.
- code 503
- body "memory exceeds threshold"
...
...
@@ -269,7 +266,7 @@ Status check interface `http://<fqdn>:6041/-/ping`
Corresponding configuration parameter
``text
```text
monitor.collectDuration monitoring interval environment variable `TAOS_MONITOR_COLLECT_DURATION` (default value 3s)
monitor.incgroup whether to run in cgroup (set to true for running in container) environment variable `TAOS_MONITOR_INCGROUP`
monitor.pauseAllMemoryThreshold memory threshold for no more inserts and queries environment variable `TAOS_MONITOR_PAUSE_ALL_MEMORY_THRESHOLD` (default 80)
...
...
@@ -297,11 +294,11 @@ taosAdapter supports writing the metrics of HTTP monitoring, CPU percentage, and
@@ -328,10 +323,10 @@ You can also adjust the level of the taosAdapter log output by setting the `--lo
In TDengine server 2.2.x.x or earlier, the TDengine server process (taosd) contains an embedded HTTP service. As mentioned earlier, taosAdapter is a standalone software managed using `systemd` and has its own process ID. There are some configuration parameters and behaviors that are different between the two. See the following table for details.
| 2 | httpMaxThreads | n/a | taosAdapter Automatically manages thread pools without this parameter |
| 3 | telegrafUseFieldNum | See the taosAdapter telegraf configuration method | |
| 4 | restfulRowLimit | restfulRowLimit | Embedded httpd outputs 10240 rows of data by default, the maximum allowed is 102400. taosAdapter also provides restfulRowLimit but it is not limited by default. You can configure it according to the actual scenario.
| 5 | httpDebugFlag | Not applicable | httpdDebugFlag does not work for taosAdapter |
| 6 | httpDBNameMandatory | N/A | taosAdapter requires the database name to be specified in the URL |
| 6 | httpDBNameMandatory | N/A | taosAdapter requires the database name to be specified in the URL |
@@ -23,11 +23,7 @@ There are two ways to install taosBenchmark:
TaosBenchmark needs to be executed on the terminal of the operating system, it supports two configuration methods: [Command-line arguments](#command-line-arguments-in-detail) and [JSON configuration file](#configuration-file-parameters-in-detail). These two methods are mutually exclusive. Users can use `-f <json file>` to specify a configuration file. When running taosBenchmark with command-line arguments to control its behavior, users should use other parameters for configuration, but not the `-f` parameter. In addition, taosBenchmark offers a special way of running without parameters.
<<<<<<< HEAD
taosBenchmark supports complete performance testing of TDengine. taosBenchmark supports the TDengine functions in three categories: write, query, and subscribe. These three functions are mutually exclusive, and users can select only one of them each time taosBenchmark runs. It is important to note that the type of functionality to be tested is not configurable when using the command-line configuration method, which can only test writing performance. To test the query and subscription performance of the TDengine, you must use the configuration file method and specify the function type to test via the parameter `filetype` in the configuration file.
=======
taosBenchmark supports the complete performance testing of TDengine by providing functionally to write, query, and subscribe. These three functions are mutually exclusive, users can only select one of them each time taosBenchmark runs. The query and subscribe functionalities are only configurable using a json configuration file by specifying the parameter `filetype`, while write can be performed through both the command-line and a configuration file.
>>>>>>> 108548b4d6 (docs: typo)
taosBenchmark supports the complete performance testing of TDengine by providing functionally to write, query, and subscribe. These three functions are mutually exclusive, users can only select one of them each time taosBenchmark runs. The query and subscribe functionalities are only configurable using a json configuration file by specifying the parameter `filetype`, while write can be performed through both the command-line and a configuration file. If you want to test the performance of queries or data subscriptionm configure taosBenchmark with the configuration file. You can modify the value of the `filetype` parameter to specify the function that you want to test.
**Make sure that the TDengine cluster is running correctly before running taosBenchmark. **
...
...
@@ -61,8 +57,9 @@ Use the following command-line to run taosBenchmark and control its behavior via
taosBenchmark -f <json file>
```
**Sample configuration files**
#### Configuration file examples
##### Example of inserting a scenario JSON configuration file
<details>
<summary>insert.json</summary>
...
...
@@ -73,7 +70,7 @@ taosBenchmark -f <json file>
</details>
##### Query Scenario JSON Profile Example
#### Query Scenario JSON Profile Example
<details>
<summary>query.json</summary>
...
...
@@ -84,7 +81,7 @@ taosBenchmark -f <json file>
</details>
##### Subscription JSON configuration example
#### Subscription JSON configuration example
<details>
<summary>subscribe.json</summary>
...
...
@@ -128,7 +125,7 @@ taosBenchmark -f <json file>
Enables interleaved insertion mode and specifies the number of rows of data to be inserted into each child table. Interleaved insertion mode means inserting the number of rows specified by this parameter into each sub-table and repeating the process until all sub-tables have been inserted. The default value is 0, i.e., data is inserted into one sub-table before the next sub-table is inserted.
-**-i/--insert-interval <timeInterval\>** :
Specify the insert interval in `ms` for interleaved insert mode. The default value is 0. It only works if `-B/--interlace-rows` is greater than 0. That means that after inserting interlaced rows for each child table, the data insertion with multiple threads will wait for the interval specified by this value before proceeding to the next round of writes.
Specify the insert interval in `ms` for interleaved insert mode. The default value is 0. It only works if `-B/--interlace-rows` is greater than 0. After inserting interlaced rows for each child table, the data insertion thread will wait for the interval specified by this value before proceeding to the next round of writes.
-**-r/--rec-per-req <rowNum\>** :
Writing the number of rows of records per request to TDengine, the default value is 30000.
...
...
@@ -184,7 +181,7 @@ taosBenchmark -A INT,DOUBLE,NCHAR,BINARY\(16\)
This parameter indicates writing data with random values. The default is false. If users use this parameter, taosBenchmark will generate the random values. For tag/data columns of numeric type, the value is a random value within the range of values of that type. For NCHAR and BINARY type tag columns/data columns, the value is the random string within the specified length range.
-**-x/--aggr-func** :
Switch parameter to indicate query aggregation function after insertion. The default value is false.
Switch parameter to indicate query aggregation function after insertion. The default is false.
-**-y/--answer-yes** :
Switch parameter that requires the user to confirm at the prompt to continue. The default value is false.
...
...
@@ -230,45 +227,34 @@ The parameters listed in this section apply to all function modes.
#### Database related configuration parameters
The parameters related to database creation are configured in `dbinfo` in the json configuration file, as follows. These parameters correspond to the database parameters specified when `create database` in TDengine.
The parameters related to database creation are configured in `dbinfo` in the json configuration file, as follows. The other parameters correspond to the database parameters specified when `create database` in [../../taos-sql/database].
-**name**: specify the name of the database.
-**drop**: indicate whether to delete the database before inserting. The default is true.
-**replica**: specify the number of replicas when creating the database.
-**days**: specify the time span for storing data in a single data file. The default is 10.
-**cache**: specify the size of the cache blocks in MB. The default value is 16.
-**blocks**: specify the number of cache blocks in each vnode. The default is 6.
-**precision**: specify the database time precision. The default value is "ms".
-**keep**: specify the number of days to keep the data. The default value is 3650.
#### Stream processing related configuration parameters
-**minRows**: specify the minimum number of records in the file block. The default value is 100.
The parameters for creating streams are configured in `stream` in the json configuration file, as shown below.
-**maxRows**: specify the maximum number of records in the file block. The default value is 4096.
-**stream_name**: Name of the stream. Mandatory.
-**comp**: specify the file compression level. The default value is 2.
-**stream_stb**: Name of the supertable for the stream. Mandatory.
-**walLevel** : specify WAL level, default is 1.
-**stream_sql**: SQL statement for the stream to process. Mandatory.
-**cacheLast**: indicate whether to allow the last record of each table to be kept in memory. The default value is 0. The value can be 0, 1, 2, or 3.
-**trigger_mode**: Triggering mode for stream processing. Optional.
-**quorum**: specify the number of writing acknowledgments in multi-replica mode. The default value is 1.
-**watermark**: Watermark for stream processing. Optional.
-**fsync**: specify the interval of fsync in ms when users set WAL to 2. The default value is 3000.
-**update** : indicate whether to support data update, default value is 0, optional values are 0, 1, 2.
-**drop**: Whether to create the stream. Specify yes to create the stream or no to not create the stream.
#### Super table related configuration parameters
The parameters for creating super tables are configured in `super_tables` in the json configuration file, as shown below.
-**name**: Super table name, mandatory, no default value.
-**child_table_exists** : whether the child table already exists, default value is "no", optional value is "yes" or "no".
-**child_table_count** : The number of child tables, the default value is 10.
...
...
@@ -319,6 +305,22 @@ The parameters for creating super tables are configured in `super_tables` in the
-**tags_file** : only works when insert_mode is taosc, rest. The final tag value is related to the childtable_count. Suppose the tag data rows in the CSV file are smaller than the given number of child tables. In that case, taosBenchmark will read the CSV file data cyclically until the number of child tables specified by childtable_count is generated. Otherwise, taosBenchmark will read the childtable_count rows of tag data only. The final number of child tables generated is the smaller of the two.
#### TSMA configuration parameters
The configuration parameters for specifying TSMAs are in `tsmas` in `super_tables`.
-**sliding**: Specifies time offset for TSMA window. Mandatory.
-**custom**: Specifies custom configurations to attach to the end of the TSMA creation statement. Optional.
-**start_when_inserted**: Specifies the number of inserted rows after which TSMA is started. Optional. The default value is 0.
#### Tag and Data Column Configuration Parameters
The configuration parameters for specifying super table tag columns and data columns are in `columns` and `tag` in `super_tables`, respectively.
...
...
@@ -338,6 +340,8 @@ The configuration parameters for specifying super table tag columns and data col
-**values**: The value field of the nchar/binary column/label, which will be chosen randomly from the values.
-**sma**: Insert the column into the BSMA. Enter `yes` or `no`. The default is `no`.
#### insertion behavior configuration parameters
-**thread_count**: specify the number of threads to insert data. Default is 8.
...
...
@@ -350,17 +354,17 @@ The configuration parameters for specifying super table tag columns and data col
-**confirm_parameter_prompt**: The switch parameter requires the user to confirm after the prompt to continue. The default value is false.
-**interlace_rows**: Enables interleaved insertion mode and specifies the number of rows of data to be inserted into each child table at a time. Interleaved insertion mode means inserting the number of rows specified by this parameter into each sub-table and repeating the process until all sub-tables are inserted. The default value is 0, which means that data will be inserted into the following child table only after data is inserted into one child table.
-**interlace_rows**: Enables interleaved insertion mode and specifies the number of rows of data to be inserted into each child table at a time. Staggered insertion mode means inserting the number of rows specified by this parameter into each sub-table and repeating the process until all sub-tables have been inserted. The default value is 0, i.e., data is inserted into one sub-table before the next sub-table is inserted.
This parameter can also be configured in `super_tables`, and if so, the configuration in `super_tables` takes precedence and overrides the global setting.
-**insert_interval** :
Specifies the insertion interval in ms for interleaved insertion mode. The default value is 0. Only works if `-B/--interlace-rows` is greater than 0. It means that after inserting interlace rows for each child table, the data insertion thread will wait for the interval specified by this value before proceeding to the next round of writes.
This parameter can also be configured in `super_tables`, and if configured, the configuration in `super_tables` takes high priority, overriding the global setting.
Specify the insert interval in `ms` for interleaved insert mode. The default value is 0. It only works if `-B/--interlace-rows` is greater than 0. After inserting interlaced rows for each child table, the data insertion thread will wait for the interval specified by this value before proceeding to the next round of writes.
This parameter can also be configured in `super_tables`, and if so, the configuration in `super_tables` takes precedence and overrides the global setting.
-**num_of_records_per_req** :
The number of rows of data to be written per request to TDengine, the default value is 30000. When it is set too large, the TDengine client driver will return the corresponding error message, so you need to lower the setting of this parameter to meet the writing requirements.
Writing the number of rows of records per request to TDengine, the default value is 30000. When it is set too large, the TDengine client driver will return the corresponding error message, so you need to lower the setting of this parameter to meet the writing requirements.
-**prepare_rand**: The number of unique values in the generated random data. A value of 1 means that all data are the same. The default value is 10000.
-**prepare_rand**: The number of unique values in the generated random data. A value of 1 means that all data are equal. The default value is 10000.
### Query scenario configuration parameters
...
...
@@ -388,7 +392,7 @@ The configuration parameters of the super table query are set in `super_table_qu
-**threads**: The number of threads to execute the query SQL, the default value is 1.
-**sqls** : The default value is 1.
-**sqls**:
-**sql**: The SQL command to be executed. For the query SQL of super table, keep "xxxx" in the SQL command. The program will automatically replace it with all the sub-table names of the super table.
Replace it with all the sub-table names in the super table.
-**result**: The file to save the query result. If not specified, taosBenchmark will not save result.
...
...
@@ -411,9 +415,9 @@ The configuration parameters for subscribing to a sub-table or a generic table a
-**resubAfterConsume**: "yes" means cancel the previous subscription and then subscribe again, "no" means continue the previous subscription, and the default value is "no".
-**sqls** : The default value is "no".
-**sqls**:
-**sql** : The SQL command to be executed, required.
-**result** : The file to save the query result, unspecified is not saved.
-**result**: The file to save the query result. If not specified, taosBenchmark will not save result.
#### Configuration parameters for subscribing to supertables
...
...
@@ -431,7 +435,7 @@ The configuration parameters for subscribing to a super table are set in `super_
-**resubAfterConsume**: "yes" means cancel the previous subscription and then subscribe again, "no" means continue the previous subscription, and the default value is "no".
-**sqls** : The default value is "no".
-**sql**: SQL command to be executed, required; for the query SQL of the super table, keep "xxxx" in the SQL command, and the program will replace it with all the sub-table names of the super table automatically.
-**sqls**:
-**sql**: The SQL command to be executed. For the query SQL of super table, keep "xxxx" in the SQL command. The program will automatically replace it with all the sub-table names of the super table.
Replace it with all the sub-table names in the super table.
-**result**: The file to save the query result, if not specified, it will not be saved.
-**result**: The file to save the query result. If not specified, taosBenchmark will not save result.
@@ -8,7 +8,7 @@ The TDengine command-line interface (hereafter referred to as `TDengine CLI`) is
## Installation
If executed on the TDengine server-side, there is no need for additional installation steps to install TDengine CLI as it is already included and installed automatically. To run TDengine CLI in an environment where no TDengine server is running, the TDengine client installation package needs to be installed first. For details, please refer to [connector](/reference/connector/).
If executed on the TDengine server-side, there is no need for additional installation steps to install TDengine CLI as it is already included and installed automatically. To run TDengine CLI in an environment where no TDengine server is running, the TDengine client installation package needs to be installed first. For details, please refer to [Connector](/reference/connector/).
## Execution
...
...
@@ -23,6 +23,7 @@ TDengine CLI will display a welcome message and version information if it succes
```cmd
taos>
```
After entering the TDengine CLI, you can execute various SQL commands, including inserts, queries, or administrative commands.
## Execute SQL script file
...
...
@@ -51,32 +52,33 @@ You can change the behavior of TDengine CLI by specifying command-line parameter
- -P PORT: Specify the port number to be used by the server. Default is `6030`
- -u USER: the user name to use when connecting. Default is `root`
- -p PASSWORD: the password to use when connecting to the server. Default is `taosdata`
- -?, --help: print out all command-line arguments
- -?, --help: print out all command-line arguments
And many more parameters.
- -a AUTHSTR: The auth string to use when connecting to the server
- -A: Generate auth string from password
- -a AUTHSTR: Authorization information to connect to the server.
- -A: Obtain authorization information from username and password.
- -c CONFIGDIR: Specify the directory where configuration file exists. The default is `/etc/taos`, and the default name of the configuration file in this directory is `taos.cfg`
- -C: Print the configuration parameters of `taos.cfg` in the default directory or specified by -c
- -d DATABASE: Specify the database to use when connecting to the server
- -f FILE: Execute the SQL script file in non-interactive mode
- -k: Check the service status, 0: unavailable,1: network ok,2: service ok,3: service degraded,4: exiting
- -l PKTLEN: Test package length to be used for network testing
- -n NETROLE: test scope for network connection test, default is `client`. The value can be `client`, `server`
- -N PKTNUM: Test package numbers to be used for network testing
- -f FILE: Execute the SQL script file in non-interactive mode Note that each SQL statement in the script file must be only one line.
- -k: Test the operational status of the server. 0: unavailable; 1: network ok; 2: service ok; 3: service degraded; 4: exiting
- -l PKTLEN: Test package size to be used for network testing
- -n NETROLE: test scope for network connection test, default is `client`. The value can be `client` or `server`.
- -N PKTNUM: Number of packets used for network testing
- -r: output the timestamp format as unsigned 64-bits integer (uint64_t in C language)
- -s COMMAND: execute SQL commands in non-interactive mode
- -t: Check the details of the service status,status same as -k
- -w DISPLAYWIDTH: 客户端列显示宽度
- -z, --timezone=TIMEZONE: Specify time zone. Default is the value of current configuration file
- -V, --version: Print out the current version number
- -t: Test the boot status of the server. The statuses of -k apply.
- -w DISPLAYWIDTH: Specify the number of columns of the server display.
- -zTIMEZONE: Specify time zone. Default is the value of current configuration file
- -V: Print out the current version number
Example.
For example:
```bash
taos -h h1.taos.com -s"use db; show tables;"
```
## TDengine CLI tips
- You can use the up and down keys to iterate the history of commands entered
description:Instructions and tips for using taosKeeper
---
## Introduction
taosKeeper is a tool for TDengine that exports monitoring metrics. With taosKeeper, you can easily monitor the operational status of your TDengine deployment. taosKeeper uses the TDengine REST API. It is not necessary to install TDengine Client to use taosKeeper.
## Installation
<!-- There are two ways to install taosKeeper: -->
Methods of installing taosKeeper:
<!--- Installing the official TDengine installer will automatically install taosKeeper. Please refer to [TDengine installation](/operation/pkg-install) for details. -->
- You can compile taosKeeper separately and install it. Please refer to the [taosKeeper](https://github.com/taosdata/taoskeeper) repository for details. -->
You can compile taosKeeper separately and install it. Please refer to the [taosKeeper](https://github.com/taosdata/taoskeeper) repository for details.
## Run
### Configuration and running methods
<!-- taosKeeper needs to be executed on the terminal of the operating system, it supports two configuration methods: [Command-line arguments](#command-line-arguments-in-detail) and [configuration file](#configuration-file-parameters-in-detail). Command-line arguments take precedence over values in the configuration file. -->
taosKeeper needs to be executed on the terminal of the operating system. To run taosKeeper, see [configuration file](#configuration-file-parameters-in-detail).
**Make sure that the TDengine cluster is running correctly before running taosKeeper. ** Ensure that the monitoring service in TDengine has been started. For more information, see [TDengine Monitoring Configuration](../config/#monitoring).
<!--
### Command-Line Parameters
You can use command-line parameters to run taosBenchmark and control its behavior:
```shell
taosKeeper
```
-->
### Configuration File
You can quickly launch taosKeeper with the following commands. If you do not specify a configuration file, `/etc/taos/keeper.toml` is used by default. If this file does not specify configurations, the default values are used.
```shell
taoskeeper -c <keeper config file>
```
**Sample configuration files**
```toml
# enable debug in gin framework
debug=false
# listen to server port, default 6043
port=6043
# set log level to panic, error, info, debug, or trace
loglevel="info"
# set pool size
gopoolsize=50000
# query rotation period for TDengine monitoring data
RotationInterval="15s"
[tdengine]
host="127.0.0.1"
port=6041
username="root"
password="taosdata"
# set taosAdapter to monitor
[taosAdapter]
address=["127.0.0.1:6041","192.168.1.95:6041"]
[metrics]
# monitoring metric prefix
prefix="taos"
# cluster data identifier
cluster="production"
# database to store monitoring data
database="log"
# standard tables to monitor
tables=["normal_table"]
```
### Obtain Monitoring Metrics
taosKeeper records monitoring metrics generated by TDengine in a specified database and provides an interface through which you can export the data.
The reference guide is a detailed introduction to TDengine including various TDengine connectors in different languages, and the tools that come with TDengine.
This section describes the TDengine connectors and utilities.
```mdx-code-block
import DocCardList from '@theme/DocCardList';
import {useCurrentSidebarCategory} from '@docusaurus/theme-common';