description:Install, Uninstall, Start, Stop and Upgrade
---
import Tabs from "@theme/Tabs";
import TabItem from "@theme/TabItem";
TDengine 开源版本提供 deb 和 rpm 格式安装包，用户可以根据自己的运行环境选择合适的安装包。其中 deb 支持 Debian/Ubuntu 及衍生系统，rpm 支持 CentOS/RHEL/SUSE 及衍生系统。同时我们也为企业用户提供 tar.gz 格式安装包。
TDengine community version provides dev and rpm package for users to choose based on the system environment. deb supports Debian, Ubuntu and systems derived from them. rpm supports CentOS, RHEL, SUSE and systems derived from them. Furthermore, tar.gz package is provided for enterprise customers.
## 安装
## Install
<Tabs>
<TabItemlabel="Deb 安装"value="debinst">
<TabItemlabel="Install Deb"value="debinst">
1、从官网下载获得 deb 安装包，例如 TDengine-server-2.4.0.7-Linux-x64.deb；
1. Download the tar.gz package, for example TDengine-server-2.4.0.7-Linux-x64.tar.gz；
2、In the directory where the package is located, firstly decompress the file, then switch to the sub-directory generated in decompressing, i.e. "TDengine-enterprise-server-2.4.0.7/" in this example, and execute the `install.sh` script.
```
```bash
$ tar xvzf TDengine-enterprise-server-2.4.0.7-Linux-x64.tar.gz
TDengine-enterprise-server-2.4.0.7/
TDengine-enterprise-server-2.4.0.7/driver/
...
...
@@ -124,7 +124,7 @@ taoskeeper is installed, enable it by `systemctl enable taoskeeper`
```
:::info
install.sh 安装脚本在执行过程中，会通过命令行交互界面询问一些配置信息。如果希望采取无交互安装方式，那么可以用 -e no 参数来执行 install.sh 脚本。运行 `./install.sh -h` 指令可以查看所有参数的详细说明信息。
Some configuration will be prompted for users to provide when install.sh is executing, the interactive mode can be disabled by executing `./install.sh -e no`. `./install -h` can show all parameters and detailed explanation.
当安装第一个节点时，出现 Enter FQDN：提示的时候，不需要输入任何内容。只有当安装第二个或以后更多的节点时，才需要输入已有集群中任何一个可用节点的 FQDN，支持该新节点加入集群。当然也可以不输入，而是在新节点启动前，配置到新节点的配置文件中。
When installing on the first node in the cluster, when "Enter FQDN:" is prompted, nothing needs to be provided. When installing on following nodes, when "Enter FQDN:" is prompted, the end point of the first dnode in the cluster can be input if it has been already up; or just ignore it and configure later after installation is done.
:::
## 卸载
## Uninstall
<Tabs>
<TabItemlabel="Deb 卸载"value="debuninst">
<TabItemlabel="Uninstall Deb"value="debuninst">
卸载命令如下:
Deb package of TDengine can be uninstalled as below:
```
```bash
$ sudo dpkg -r tdengine
(Reading database ... 137504 files and directories currently installed.)
Removing tdengine (2.4.0.7) ...
...
...
@@ -153,9 +153,9 @@ TDengine is removed successfully!
</TabItem>
<TabItemlabel="RPM 卸载"value="rpmuninst">
<TabItemlabel="Uninstall RPM"value="rpmuninst">
卸载命令如下:
RPM package of TDengine can be uninstalled as below:
```
$ sudo rpm -e tdengine
...
...
@@ -164,9 +164,9 @@ TDengine is removed successfully!
tar.gz package of TDengine can be uninstalled as below:
```
$ rmtaos
...
...
@@ -179,32 +179,28 @@ taosKeeper is removed successfully!
</TabItem>
</Tabs>
:::info
- TDengine 提供了多种安装包，但最好不要在一个系统上同时使用 tar.gz 安装包和 deb 或 rpm 安装包。否则会相互影响，导致在使用时出现问题。
:::note
- 对于 deb 包安装后，如果安装目录被手工误删了部分，出现卸载、或重新安装不能成功。此时，需要清除 TDengine 包的安装信息，执行如下命令：
- It's strongly suggested not to use multiple kinds of installation packages on single host TDengine
- After deb package is installed, if the installation directory is removed manually so that uninstall or reinstall can't succeed, it can be resolved by cleaning up TDengine package information as below command and then reinstalling.
-After rpm package is installed, if the installation directory is removed manually so that uninstall or reinstall can't succeed, it can be resolved by cleaning up TDengine package information as below command and then reinstalling.
```
```bash
$ sudo rpm -e--noscripts tdengine
```
然后再重新进行安装就可以了。
```
:::
## 安装目录说明
## Installation Directory
TDengine 成功安装后，主安装目录是 /usr/local/taos，目录内容如下：
TDengine is installed at /usr/local/taos if successful.
- When TDengine is uninstalled, the configuration /etc/taos/taos.cfg, data directory /var/lib/taos, log directory /var/log/taos are kept. They can be deleted manually with caution because data can't be recovered once
- When reinstalling TDengine, if the default configuration file /etc/taos/taos.cfg exists, it will be kept and the configuration file in the installation package will be renamed to taos.cfg.orig and stored at /usr/loca/taos/cfg to be used as configuration sample. Otherwise the configuration file in the installation package will be installed to /etc/taos/taos.cfg and used.
Linux system services `systemd`, `systemctl` or `service` is used to start, stop and restart TDengine. The server process of TDengine is `taosd`, which is started automatically after the Linux system is started. System operator can use `systemd`, `systemctl` or `service` to start, stop or restart TDengine server.
以 systemctl 为例，命令如下：
For example, if using `systemctl` , the commands to start, stop, restart and check TDengine server are as below:
From version 2.4.0.0, a new independent component named as `taosAdapter` has been included in TDengine. `taosAdapter` should be started and stopped using `systemctl`.
如果服务进程处于活动状态，则 status 指令会显示如下的相关信息：
If the server process is OK, the output of `systemctl status` is like below:
```
Active: active (running)
```
```
Active: active (running)
```
如果后台服务进程处于停止状态，则 status 指令会显示如下的相关信息：
Otherwise, the output is as below:
```
Active: inactive (dead)
```
```
Active: inactive (dead)
```
## Upgrade
## 升级
升级分为两个层面：升级安装包 和 升级运行中的实例。
There are two aspects in upgrade operation: upgrade installation package and upgrade a running server.
升级安装包请遵循前述安装和卸载的步骤先卸载旧版本再安装新版本。
Upgrading package should follow the steps mentioned previously to firstly uninstall old version then install new version.
Upgrading a running server is much more complex. Firstly please check the version number of old version and new version. The version number of TDengine consists of 4 sections, only the first 3 section match can the old version be upgraded to the new version. The steps of upgrading a running server are as below:
- Stop inserting data
- Make sure all data persisted into disk
- Stop the cluster of TDengine
- Uninstall old version and install new version
- Start the cluster of TDengine
- Make some simple queries to make sure no data loss
- Make some simple data insertion to make sure the cluster works well
- Restore business data
:::warning
TDengine 不保证低版本能够兼容高版本的数据，所以任何时候都不推荐降级
TDengine doesn't guarantee any lower version is compatible with the data generated by a higher version, so it's never recommended to downgrade the version.
The computing and storage resources need to be planned if using TDengine to build an IoT platform. How to plan the CPU, memory and disk required will be described in this chapter.
## 内存需求
## Memory Requirement of Server Side
每个 Database 可以创建固定数目的 vgroup，默认与 CPU 核数相同，可通过 maxVgroupsPerDb 配置；vgroup 中的每个副本会是一个 vnode；每个 vnode 会占用固定大小的内存（大小与数据库的配置参数 blocks 和 cache 有关)；每个 Table 会占用与标签总长度有关的内存；此外，系统会有一些固定的内存开销。因此，每个 DB 需要的系统内存可通过如下公式计算：
The number of vgroups created for each database is same as the number of CPU cores by default and can be configured by parameter `maxVgroupsPerDb`, each vnode in a vgroup stores one replica. Each vnode consumes fixed size of memory, i.e. `blocks` \* `cache`. Besides, some memory is required for tag values associated with each table. A fixed amount of memory is required for each cluster. So, the memory required for each DB can be calculated using below formula:
For example, assuming the default value of `maxVgroupPerDB` is 64, the default value of `cache` 16M, the default value of `blocks` is 6, there are 100,000 tables in a DB, the replica number is 1, total length of tag values is 256 bytes, the total memory required for this DB is: 并且一个 DB 中有 10 万张表，单副本，标签总长度是 256 字节，则这个 DB 总的内存需求为：64 \* 1 \* (16 \* 6 + 10) + 100000 \* (0.25 + 0.5) / 1000 = 6792M.
在实际的系统运维中，我们通常会更关心 TDengine 服务进程（taosd）会占用的内存量。
In real operation of TDengine, we are more concerned about the memory used by each TDengine server process `taosd`.在
1. "vnode_memory" of a `taosd` process is the memory used by all vnodes hosted by this `taosd` process. It can be roughly calculated by firstly adding up the total memory of all DBs whose memory usage can be derived according to the formula mentioned previously then dividing by number of dnodes and multiplying the number of replicas.
2. "mnode_memory" of a `taosd` process is the memory consumed by a mnode. If there is one (and only one) mnode hosted in a `taosd` process, the memory consumed by "mnode" is "0.2KB \* the total number of tables in the cluster".
3. "query_memory" is the memory used when processing query requests. Each ongoing query consumes at least "0.2 KB \* total number of involved tables".
### 客户端内存需求
Please be noted that the above formulas can only be used to estimate the minimum memory requirement, instead of maximum memory usage. In a real production environment, it's better to preserve some redundance beyond the estimated minimum memory requirement. If memory is abundant, it's suggested to increase the value of parameter `blocks` to speed up data insertion and data query.
客户端应用采用 taosc 客户端驱动连接服务端，会有内存需求的开销。
## Memory Requirement of Client Side
客户端的内存开销主要由写入过程中的 SQL 语句、表的元数据信息缓存、以及结构性开销构成。系统最大容纳的表数量为 N（每个通过超级表创建的表的 meta data 开销约 256 字节），最大并行写入线程数量 T，最大 SQL 语句长度 S（通常是 1 Mbytes）。由此可以进行客户端内存开销的估算（单位 MBytes）：
The client programs use TDengine client driver `taosc` to connect to the server side, there is also memory requirement for a client program.
The memory consumed by a client program is mainly about the SQL statements for data insertion, caching of table metadata, and some internal use. Assuming maximum number of tables is N (the memory consumed by the metadata of each table is 256 bytes), maximum number of threads for parallel insertion is T, maximum length of a SQL statement is S (normally 1 MB), the memory required by a client program can be estimated using below formula:
For example, if the number of parallel data insertion threads is 100, total number of tables is 10,000,000, then minimum memory requirement of a client program is:
```
100 * 3 + (10000000 / 4096) + 100 = 2741 (MBytes)
```
即配置 3 GBytes 内存是最低要求。
So, at least 3GB needs to be reserved for such a client.
- **Data Insertion** Each dnode of TDengine can process at least 10,000 insertion requests in one second, while each insertion request can have multiple rows. The computing resource consumed between inserting 1 row one time and inserting 10 rows one time is very small. So, the more the rows to insert one time, the higher the efficiency. Inserting in bach also exposes requirement for the client side which needs to cache rows and insert in batch once the cached rows reaches a threshold.
- **Data Query** High efficiency query is provided in TDengine, but it's hard to estimate the CPU resource required because the queries used in different use cases and the frequency of queries vary significantly. It can only be verified with the query statements, query frequency, data size to be queried, etc provided by user.
因此仅对数据插入而言，CPU 是可以估算出来的，但查询所耗的计算资源无法估算。在实际运营过程中，不建议 CPU 使用率超过 50%，超过后，需要增加新的节点，以获得更多计算资源。
In short words, the CPU resource required for data insertion can be estimated but it's hard to do so for query use cases. In real operation, it's suggested to control CPU usage below 50%. If this threshold is exceeded, it's a reminder for system operator to add more nodes in the cluster to expand resources.
The compression ratio in TDengine is much higher than that in RDBMS. In most cases, the compression ratio in TDengine is bigger than 5, or even 10 in some cases, depending on the characteristics of the original data. The data size before compression can be calculated based on below formula:
```
Raw DataSize = numOfTables * rowSizePerTable * rowsPerTable
For example, there are 10,000,000 meters, while each meter collects data every 15 minutes and the data size of each collection si 128 bytes, so the raw data size of one year is: 10000000 \* 128 \* 24 \* 60 / 15 \* 365 = 44.8512(TB). Assuming compression ratio is 5, the actual disk size is: 44.851 / 5 = 8.97024(TB).
Parameter `keep` can be used to set how long the data will be kept on disk. To further reduce storage cost, multiple storage levels can be enabled in TDengine, with the coldest data stored on the cheapest storage device, and this is transparent to application programs.
## 物理机或虚拟机台数
To increase the performance, multiple disks can be setup for parallel data reading or data inserting. Please be noted that expensive disk array is not necessary because replications are used in TDengine to provide high availability.
A host can be either physical or virtual. The total memory, total CPU, total disk required can be estimated according to the formulas mentioned previously. Then, according to the system resources that a single host can provide, assuming all hosts are same in resources, the number of hosts can be derived easily.
When a data block is received by TDengine, the original data block is firstly written into WAL. The log in WAL will be deleted only after the data has been written into data files in the database. Data can be recovered from WAL in case the server is stopped abnormally due to any reason and then restarted.
涉及的系统配置参数有两个：
There are 2 configuration parameters related to WAL:
- walLevel：0：wal is disabled; 1：wal is enabled without fsync; 2：wal is enabled with fsync.
- fsync：only valid when walLevel is set to 2, it specified the interval of invoking fsync. If set to 0, it means fsync is invoked immediately once WAL is written.
To achieve absolutely no data loss, walLevel needs to be set to 2 and fsync needs to be set to 1. The penalty is the speed of data insertion downgrades. However, if the concurrent threads of data insertion on the client side can reach a big enough number, for example 50, the data insertion performance would be still good enough, our verification shows that the drop is only 30% compared to fsync is set to 3,000 milliseconds.
## 灾备
## Disaster Recovery
TDengine 的集群通过多个副本的机制，来提供系统的高可用性，实现灾备能力。
TDengine uses replications to provide high availability and disaster recovery capability.
TDengine cluster is managed by mnode. To make sure the high availability of mnode, multiple replicas can be configured by system parameter `numOfMnodes`. The data replication between mnode replicas is in synchronous way to guarantee the metadata consistency.
The number of replicas for time series data in TDengine is associated with each database, there can be a lot of databases in a cluster while each database can be configured with a different number of replicas. When creating a database, parameter `replica` is used to configure the number of replications. To achieve high availability, `replica` needs to be higher than 1.
TDengine 集群的节点数必须大于等于副本数，否则创建表时将报错。
The number of dnodes in a TDengine cluster must NOT be lower than the number of replicas for any database, otherwise it would fail when trying to create table.集
As long as the dnodes of a TDengine cluster are deployed on different physical machines and replica number is set to bigger than 1, high availability can be achieved without any other assistance. If dnodes of TDengine cluster are deployed in geographically different data centers, disaster recovery can be achieved too.
TDengine CLI `taos` supports `source <filename>` command for executing the SQL statements in the file in batch. The SQL statements for creating databases, creating tables, and inserting rows can be written in single file with one statement on each line, then the file can be executed using `source` command in TDengine CLI `taos` to execute the SQL statements in order and in batch. In the script file, any line beginning with "#" is treated as comments and ignored silently.
In TDengine CLI, data can be imported from a CSV file into an existing table. The data in single CSV must belong to same table and must be consistent with the schema of that table. The SQL statement is as below:也
If there is description in the first line of a CSV file, please remove it before importing. If there is no value for a column, please use `NULL` without quotes.
:::
例如，现在存在一个子表 d1001, 其表结构如下：
For example, there is a sub table d1001 whose schema is as below:
```sql
taos>DESCRIBEd1001
...
...
@@ -35,7 +36,7 @@ taos> DESCRIBE d1001
groupid|INT|4|TAG|
```
要导入的 data.csv 的格式如下：
The format of the CSV file to be imported, data.csv, is as below:
```csv
'2018-10-04 06:38:05.000',10.30000,219,0.31000
...
...
@@ -49,13 +50,13 @@ taos> DESCRIBE d1001
'2018-10-12 06:38:05.000',18.30000,219,0.31000
```
那么可以用如下命令导入数据：
Then, below SQL statement can be used to import data from file "data.csv", assuming the file is located under the home directory of current Linux user.
A convenient tool for importing and exporting data is provided by TDengine, `taosdump`, which can used to export data from one TDengine cluster and import into another one. For the details of using `taosdump` please refer to [Tool for exporting and importing data: taosdump](/reference/taosdump).
With `taosdump`, you can choose to export the data of all databases, a database, a table or a stable, you can also choose export the data within a time range, or even only export the schema definition of a table. For the details of using `taosdump` please refer to [Tool for exporting and importing data: taosdump](/reference/taosdump).
System operator can use TDengine CLI to show the connections, ongoing queries, stream computing, and can close connection or stop ongoing query task or stream computing.
## 显示数据库的连接
## Show Connections
```sql
SHOWCONNECTIONS;
```
其结果中的一列显示 ip:port, 为连接的 IP 地址和端口号。
One column of the output of the above SQL command is "ip:port", which is the end point of the client.
## 强制关闭数据库连接
## Close Connections Forcedly
```sql
KILLCONNECTION<connection-id>;
```
其中的 connection-id 是 SHOW CONNECTIONS 中显示的第一列的数字。
In the above SQL command, `connection-id` is from the first column of the output of `SHOW CONNECTIONS`.
The first column of the output is query ID, which is composed of the corresponding connection ID and the sequence number of the current query task started on this connection, in format of "connection-id:query-no".
## 强制关闭数据查询
## Close Queries Forcedly
```sql
KILLQUERY<query-id>;
```
其中 query-id 是 SHOW QUERIES 中显示的 connection-id:query-no 字串，如“105:2”，拷贝粘贴即可。
In the above SQL command, `query-id` is from the first column of the output of `SHOW QUERIES `.
The first column of the output is stream ID, which is composed of the connection ID and the sequence number of the current stream started on this connection, in the format of "connection-id:stream-no".
## 强制关闭连续查询
## Close Continuous Query Forcedly
```sql
KILLSTREAM<stream-id>;
```
其中的 stream-id 是 SHOW STREAMS 中显示的 connection-id:stream-no 字串，如 103:2，拷贝粘贴即可。
The the above SQL command, `stream-id` is from the first column of the output of `SHOW STREAMS`.
After TDengine is started, a database named `log` for monitoring is created automatically. The information about CPU, memory, disk, bandwidth, number of requests, disk I/O speed, slow query is written into `log` database on the basis of a predefined interval. Besides, some important system operations, like logon, create user, drop database, and alerts and warnings generated in TDengine are written into `log` database too. System operator can view the data in `log` database from TDengine CLI or from a web console.
这些监测信息的采集缺省是打开的，但可以修改配置文件里的选项 monitor 将其关闭或打开。
Collection of the monitoring information is enabled by default, but can be disabled by parameter `monitor` in configuration file.
TDinsight is a total solution which uses the monitor database `log` mentioned previously and Grafana to monitor a TDengine cluster.
我们提供了一个自动化脚本 `TDinsight.sh` 对 TDinsight 进行部署。
From version 2.3.3.0, more monitoring data has been added in the `log` database. Please refer to [TDinsight Grafana Dashboard](https://grafana.com/grafana/dashboards/15167) to learn more details about using TDinsight to monitor TDengine.
下载 `TDinsight.sh`：
A script `TDinsight.sh` is provided to deploy TDinsight in automatic way.
- An existing Grafana Notification Channel can be specified with parameter `-E`, the notifier uid of the channel can be obtained by `curl -u admin:admin localhost:3000/api/alert-notifications |jq`
When the client is unable to access the server, the network connection between the client side and the server side needs to be checked to find out the root cause and resolve problems.
目前网络连接诊断支持在：Linux 与 Linux，Linux 与 Windows 之间进行诊断测试。
The diagnostic for network connection can be executed between Linux and Linux or between Linux and Windows.
诊断步骤：
Diagnostic steps：
1.如拟诊断的端口范围与服务器 taosd 实例的端口范围相同，须先停掉 taosd 实例
2.服务端命令行输入：`taos -n server -P <port> -l <pktlen>` 以服务端身份启动对端口 port 为基准端口的监听
1.If the port range to be diagnosed are being occupied by a `taosd` server process, please firstly stop `taosd.
2. On the server side, execute command `taos -n server -P <port> -l <pktlen>` to monitor the port range starting from the port specified by `-P` parameter with the role of "server.
3. On the client side, execute command `taos -n client -h <fqdnofserver> -P <port> -l <pktlen>` to send testing package to the specified server and port.
-l <pktlen\>： The size of the testing package, in bytes. The value range is [11, 64,000] and default value is 1,000. Please be noted that the package length must be same in the above 2 commands executed on server side and client side respectively.
服务端运行正常的话会输出以下信息：
Output of the server side is as below for example:
```bash
# taos -n server -P 6000
...
...
@@ -48,7 +48,7 @@ title: 诊断及其他
12/21 14:50:22.721261 0x7f53427ec700 UTL UDP: send:1000 bytes to 172.27.0.8 at 6011
```
客户端运行正常会输出以下信息：
Output of the client side is as below for example:
```bash
# taos -n client -h 172.27.0.7 -P 6000
...
...
@@ -66,66 +66,58 @@ title: 诊断及其他
12/21 14:50:22.721274 0x7fc95d859200 UTL successed to test UDP port:6011
```
仔细阅读打印出来的错误信息，可以帮助管理员找到原因，以解决问题。
The output needs to be checked carefully for the system operator to find out root cause and solve the problem.
## 启动状态及 RPC 诊断
## Startup Status and RPC Diagnostic
`taos -n startup -h <fqdn of server>`
`taos -n startup -h <fqdnofserver>` can be used to check the startup status of a `taosd` process. This is a comman task for a system operator to do to determine whether `taosd` has been started successfully, especially in case of cluster.
`taos -n rpc -h <fqdnofserver>` can be used to check whether the port of a started `taosd` can be accessed or not. If `taosd` process doesn't respond or work abnormally, this command can be used to initiate a rpc communication with the specified fqdn to determine whether it's network problem or `taosd` is abnormal.
The above commands can be executed on Linux Shell to check whether the port for sync works well and whether the sync module of the server side works well. Besides, `-P 6042` is used to check whether the arbitrator is configured properly and works well.
From version 2.2.0.0, the above command can be executed on Linux Shell to test the network speed, it sends uncompressed package to a running `taosd` server process or a simulated server process started by `taos -n server` to test the network speed. Parameters can be used when testing network speed are as below:
-n：设为“speed”时，表示对网络速度进行诊断。
-h：所要连接的服务端的 FQDN 或 ip 地址。如果不设置这一项，会使用本机 taos.cfg 文件中 FQDN 参数的设置作为默认值。
From version 2.2.0.0, the above command can be executed on Linux Shell to test the resolution speed of FQDN. It can be used to try to resolve a FQDN to an IP address and record the time spent in this process. The parameters that can be used for this purpose are as below:
## 服务端日志
-n：When set to "fqdn", it means testing the speed of resolving FQDN
-h：The FQDN to be resolved. If not set, the `FQDN` parameter in `taos.cfg` is used by default.
The parameter `debugFlag` is used to control the log level of `taosd` server process. The default value is 131, for debug purpose it needs to be escalated to 135 or 143.
Once this parameter is set to 135 or 143, the log file grows very quickly especially when there is huge volume of data insertion and data query requests. If all the logs are stored together, some important information may be missed very easily, so on server side important information is stored at different place from other logs.一
## 客户端日志
- The log at level of INFO, WARNING and ERROR is stored in `taosinfo` so that it is easy to find important information
- The log at level of DEBUG (135) and TRACE (143) and other information not handled by `taosinfo` are stored in `taosdlog`
An independent log file, named as "taoslog+<seq num\>" is generated for each client program, i.e. a client process. The default value of `debugfalg` is also 131 and only log at level of INFO/ERROR/WARNING is recorded, it and needs to be changed to 135 or 143 so that log at DEBUG or TRACE level can be recorded for debugging purpose.
