@@ -125,7 +125,7 @@ configuration files are under "conf" folder
* system config module (`tsfile-format.properties`, `iotdb-engine.properties`)
* log config module (`logback.xml`).
For more, see [Chapter4: Deployment and Management](https://iotdb.apache.org/#/Documents/0.8.0/chap4/sec1) in detail.
For more, see [Chapter3: Server](https://iotdb.apache.org/#/Documents/progress/chap3/sec1) and [Chapter4: Client](https://iotdb.apache.org/#/Documents/progress/chap4/sec1) in detail.
*[PISA: An Index for Aggregating Big Time Series Data](https://dl.acm.org/citation.cfm?id=2983775&dl=ACM&coll=DL), Xiangdong Huang and Jianmin Wang and Raymond K. Wong and Jinrui Zhang and Chen Wang. CIKM 2016.
*[Matching Consecutive Subpatterns over Streaming Time Series](https://link.springer.com/chapter/10.1007/978-3-319-96893-3_8), Rong Kang and Chen Wang and Peng Wang and Yuting Ding and Jianmin Wang. APWeb/WAIM 2018.
*[KV-match: A Subsequence Matching Approach Supporting Normalization and Time Warping](https://www.semanticscholar.org/paper/KV-match%3A-A-Subsequence-Matching-Approach-and-Time-Wu-Wang/9ed84cb15b7e5052028fc5b4d667248713ac8592), Jiaye Wu and Peng Wang and Chen Wang and Wei Wang and Jianmin Wang. ICDE 2019.
select status,temperature from root.ln.wf01.wt01 where (time > 2017-11-01T00:05:00.000 and time < 2017-11-01T00:12:00.000) or (time >= 2017-11-01T16:35:00.000 and time <= 2017-11-01T16:37:00.000);
select wf01.wt01.status,wf02.wt02.hardware from root.ln where (time > 2017-11-01T00:05:00.000 and time < 2017-11-01T00:12:00.000) or (time >= 2017-11-01T16:35:00.000 and time <= 2017-11-01T16:37:00.000);
@@ -48,8 +48,7 @@ This short guide will walk you through the basic process of using IoTDB. For a m
To use IoTDB, you need to have:
1. Java >= 1.8 (Please make sure the environment path has been set)
2. Maven >= 3.1 (If you want to compile and install IoTDB from source code)
3. Set the max open files num as 65535 to avoid "too many open files" problem.
2. Set the max open files num as 65535 to avoid "too many open files" problem.
## Installation
...
...
@@ -60,27 +59,14 @@ IoTDB provides you three installation methods, you can refer to the following su
* Using Docker：The path to the dockerfile is https://github.com/apache/incubator-iotdb/blob/master/docker/Dockerfile
Here in the Quick Start, we give a brief introduction of using source code to install IoTDB. For further information, please refer to Chapter 4 of the User Guide.
Here in the Quick Start, we give a brief introduction to install IoTDB. For further information, please refer to Chapter 3 of the User Guide.
Then the binary version (including both server and client) can be found at **distribution/target/apache-iotdb-{project.version}-incubating-bin.zip**
> NOTE: Directories "service-rpc/target/generated-sources/thrift" and "server/target/generated-sources/antlr3" need to be added to sources roots to avoid compilation errors in IDE.
### Configurations
## Configurations
configuration files are under "conf" folder
...
...
@@ -88,7 +74,7 @@ configuration files are under "conf" folder
* system config module (`tsfile-format.properties`, `iotdb-engine.properties`)
* log config module (`logback.xml`).
For more, see [Chapter3: Deployment](https://iotdb.apache.org/#/Documents/progress/chap3/sec1) in detail.
For more, see [Chapter3: Server](https://iotdb.apache.org/#/Documents/progress/chap3/sec1) in detail.
## Start
...
...
@@ -239,7 +225,7 @@ or
IoTDB> exit
```
For more on what commands are supported by IoTDB SQL, see [IoTDB SQL Language](https://iotdb.apache.org/#/Documents/progress/chap4/sec7).
For more on what commands are supported by IoTDB SQL, see [SQL Reference](https://iotdb.apache.org/#/Documents/progress/chap5/sec4).
### Stop IoTDB
...
...
@@ -253,16 +239,6 @@ The server can be stopped with ctrl-C or the following script:
> sbin\stop-server.bat
```
## Only build server
Under the root path of incubator-iotdb:
```
> mvn clean package -pl server -am -DskipTests
```
After build, the IoTDB server will be at the folder "server/target/iotdb-server-{project.version}".
@@ -113,7 +113,7 @@ Yes. IoTDB has intense integration with Open Source Ecosystem. IoTDB supports [H
## How does IoTDB handle duplicate points?
A data point is uniquely identified by a full time series path (e.g. ```root.vehicle.d0.s0```) and timestamp. If you submit a new point with the same path and timestamp as an existing point,
A data point is uniquely identified by a full time series path (e.g. ```root.vehicle.d0.s0```) and timestamp. If you submit a new point with the same path and timestamp as an existing point, IoTDB will update the value of this point instead of inserting a new point.
## How can I tell what type of the specific timeseries?
@@ -27,7 +27,7 @@ Besides IoTDB engine, we also developed several components to provide better IoT
IoTDB suite can provide a series of functions in the real situation such as data collection, data writing, data storage, data query, data visualization and data analysis. Figure 1.1 shows the overall application architecture brought by all the components of the IoTDB suite.
As shown in Figure 1.1, users can use JDBC to import timeseries data collected by sensor on the device to local/remote IoTDB. These timeseries data may be system state data (such as server load and CPU memory, etc.), message queue data, timeseries data from applications, or other timeseries data in the database. Users can also write the data directly to the TsFile (local or on HDFS).
To make this manual more practical, we will use a specific scenario example to illustrate how to operate IoTDB databases at all stages of use. See [this page](https://github.com/apache/incubator-iotdb/blob/master/docs/Documentation/OtherMaterial-Sample%20Data.txt) for a look. For convenience, we also provide you with a sample data file in real scenario to import into the IoTDB system for trial and operation.
The following basic concepts are involved in IoTDB:
According to the data attribute layers described in [sample data](https://raw.githubusercontent.com/apache/incubator-iotdb/master/docs/Documentation/OtherMaterial-Sample%20Data.txt), we can express it as an attribute hierarchy structure based on the coverage of attributes and the subordinate relationship between them, as shown in Figure 2.1 below. Its hierarchical relationship is: power group layer - power plant layer - device layer - sensor layer. ROOT is the root node, and each node of sensor layer is called a leaf node. In the process of using IoTDB, you can directly connect the attributes on the path from ROOT node to each leaf node with ".", thus forming the name of a timeseries in IoTDB. For example, The left-most path in Figure 2.1 can generate a timeseries named `ROOT.ln.wf01.wt01.status`.
After getting the name of the timeseries, we need to set up the storage group according to the actual scenario and scale of the data. Because in the scenario of this chapter data is usually arrived in the unit of groups (i.e., data may be across electric fields and devices), in order to avoid frequent switching of IO when writing data, and to meet the user's requirement of physical isolation of data in the unit of groups, we set the storage group at the group layer.
Here are the basic concepts of the model involved in IoTDB:
* Device
...
...
@@ -208,14 +219,3 @@ Relative time refers to the time relative to the server time ```now()``` and ```
```
> Note：There must be spaces on the left and right of '+' and '-'.
* Value
The value of a time series is actually the value sent by a sensor to IoTDB. This value can be stored by IoTDB according to the data type. At the same time, users can select the compression mode and the corresponding encoding mode according to the data type of this value. See [Data Type](/#/Documents/progress/chap2/sec2), [Encoding](/#/Documents/progress/chap2/sec3) and [Compression](/#/Documents/progress/chap2/sec4) of this document for details on data type and corresponding encoding.
* Point
A data point is made up of a timestamp value pair (timestamp, value).
* Column
A column of data contains all values belonging to a time series and the timestamps corresponding to these values. When there are multiple columns of data, IoTDB merges the timestamps into multiple <timestamp-value> pairs (timestamp, value, value,...).
@@ -31,7 +31,7 @@ IoTDB supports six data types in total:
* TEXT (String).
The time series of **FLOAT** and **DOUBLE** type can specify (MAX\_POINT\_NUMBER, see [this page](/#/Documents/progress/chap4/sec7) for more information on how to specify), which is the number of digits after the decimal point of the floating point number, if the encoding method is [RLE](/#/Documents/progress/chap2/sec3) or [TS\_2DIFF](/#/Documents/progress/chap2/sec3)(Refer to [Create Timeseries Statement](/#/Documents/progress/chap5/sec1) for more information on how to specify). If MAX\_POINT\_NUMBER is not specified, the system will use [float\_precision](/#/Documents/progress/chap4/sec2) in the configuration file `tsfile-format.properties`.
The time series of **FLOAT** and **DOUBLE** type can specify (MAX\_POINT\_NUMBER, see [this page](/#/Documents/progress/chap5/sec4) for more information on how to specify), which is the number of digits after the decimal point of the floating point number, if the encoding method is [RLE](/#/Documents/progress/chap2/sec3) or [TS\_2DIFF](/#/Documents/progress/chap2/sec3)(Refer to [Create Timeseries Statement](/#/Documents/progress/chap5/sec4) for more information on how to specify). If MAX\_POINT\_NUMBER is not specified, the system will use [float\_precision](/#/Documents/progress/chap3/sec4) in the configuration file `tsfile-format.properties`.
* For Float data value, The data range is (-Integer.MAX_VALUE, Integer.MAX_VALUE), rather than Float.MAX_VALUE, and the max_point_number is 19, it is because of the limition of function Math.round(float) in Java.
* For Double data value, The data range is (-Long.MAX_VALUE, Long.MAX_VALUE), rather than Double.MAX_VALUE, and the max_point_number is 19, it is because of the limition of function Math.round(double) in Java (Long.MAX_VALUE=9.22E18).
@@ -32,13 +32,13 @@ PLAIN encoding, the default encoding mode, i.e, no encoding, supports multiple d
Second-order differential encoding is more suitable for encoding monotonically increasing or decreasing sequence data, and is not recommended for sequence data with large fluctuations.
Second-order differential encoding can also be used to encode floating-point numbers, but it is necessary to specify reserved decimal digits (MAX\_POINT\_NUMBER, see [this page](/#/Documents/progress/chap5/sec1) for more information on how to specify) when creating time series. It is more suitable for storing sequence data where floating-point values appear continuously, monotonously increase or decrease, and it is not suitable for storing sequence data with high precision requirements after the decimal point or with large fluctuations.
Second-order differential encoding can also be used to encode floating-point numbers, but it is necessary to specify reserved decimal digits (MAX\_POINT\_NUMBER, see [this page](/#/Documents/progress/chap5/sec4) for more information on how to specify) when creating time series. It is more suitable for storing sequence data where floating-point values appear continuously, monotonously increase or decrease, and it is not suitable for storing sequence data with high precision requirements after the decimal point or with large fluctuations.
* RLE
Run-length encoding is more suitable for storing sequence with continuous integer values, and is not recommended for sequence data with most of the time different values.
Run-length encoding can also be used to encode floating-point numbers, but it is necessary to specify reserved decimal digits (MAX\_POINT\_NUMBER, see [this page](/#/Documents/progress/chap4/sec7) for more information on how to specify) when creating time series. It is more suitable for storing sequence data where floating-point values appear continuously, monotonously increasing or decreasing, and it is not suitable for storing sequence data with high precision requirements after the decimal point or with large fluctuations.
Run-length encoding can also be used to encode floating-point numbers, but it is necessary to specify reserved decimal digits (MAX\_POINT\_NUMBER, see [this page](/#/Documents/progress/chap5/sec4) for more information on how to specify) when creating time series. It is more suitable for storing sequence data where floating-point values appear continuously, monotonously increasing or decreasing, and it is not suitable for storing sequence data with high precision requirements after the decimal point or with large fluctuations.
When the time series is written and encoded as binary data according to the specified type, IoTDB compresses the data using compression technology to further improve space storage efficiency. Although both encoding and compression are designed to improve storage efficiency, encoding techniques are usually only available for specific data types (e.g., second-order differential encoding is only suitable for INT32 or INT64 data type, and storing floating-point numbers requires multiplying them by 10m to convert to integers), after which the data is converted to a binary stream. The compression method (SNAPPY) compresses the binary stream, so the use of the compression method is no longer limited by the data type.
IoTDB allows you to specify the compression method of the column when creating a time series. IoTDB now supports several kinds of compression: UNCOMPRESSED (no compression), SNAPPY, GZIP, LZ0, SDT, PAA and PLA. The specified syntax for compression is detailed in [Create Timeseries Statement](/#/Documents/progress/chap4/sec7).
\ No newline at end of file
IoTDB allows you to specify the compression method of the column when creating a time series, and now supports two compression methods:
* UMCOMPRESSED
* SNAPPY
The specified syntax for compression is detailed in [Create Timeseries Statement](/#/Documents/progress/chap5/sec4).
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
# Chapter 3: Deployment
## Installation
IoTDB provides you two installation methods, you can refer to the following suggestions, choose one of them:
* Installation from binary files. Download the binary files from the official website. This is the recommended method, in which you will get a binary released package which is out-of-the-box.
* Installation from source code. If you need to modify the code yourself, you can use this method.
### Prerequisites
To install and use IoTDB, you need to have:
1. Java >= 1.8 (Please make sure the environment path has been set)
2. Maven >= 3.1 (If you want to compile and install IoTDB from source code)
TODO: TsFile and IoTDB-JDBC dependencies will be removed after the project reconstruct.
### Installation from binary files
IoTDB provides you binary files which contains all the necessary components for the IoTDB system to run. You can get them on our website [http://tsfile.org/download](http://tsfile.org/download).
```
NOTE:
iotdb-<version>.tar.gz # For Linux or MacOS
iotdb-<version>.zip # For Windows
```
After downloading, you can extract the IoTDB tarball using the following operations:
```
Shell > uzip iotdb-<version>.zip # For Windows
Shell > tar -zxf iotdb-<version>.tar.gz # For Linux or MacOS
```
The IoTDB project will be at the subfolder named iotdb. The folder will include the following contents:
```
iotdb/ <-- root path
|
+- sbin/ <-- script files
|
+- conf/ <-- configuration files
|
+- lib/ <-- project dependencies
|
+- LICENSE <-- LICENSE
```
### Installation from source code
You can get the released source code from https://iotdb.apache.org/#/Download, or from the git repository https://github.com/apache/incubator-iotdb/tree/master
Now suppose your directory is like this:
```
> pwd
/workspace/incubator-iotdb
> ls -l
incubator-iotdb/ <-- root path
|
+- server/
|
+- jdbc/
|
+- client/
|
...
|
+- pom.xml
```
Let `$IOTDB_HOME = /workspace/incubator-iotdb/server/target/iotdb-server-{project.version}`
Let `$IOTDB_CLI_HOME = /workspace/incubator-iotdb/client/target/iotdb-client-{project.version}`
Note:
* if `IOTDB_HOME` is not explicitly assigned,
then by default `IOTDB_HOME` is the direct parent directory of `sbin/start-server.sh` on Unix/OS X
(or that of `sbin\start-server.bat` on Windows).
* if `IOTDB_CLI_HOME` is not explicitly assigned,
then by default `IOTDB_CLI_HOME` is the direct parent directory of `sbin/start-cli.sh` on
Unix/OS X (or that of `sbin\start-cli.bat` on Windows).
If you are not the first time that building IoTDB, remember deleting the following files:
```
> rm -rf $IOTDB_HOME/data/
> rm -rf $IOTDB_HOME/lib/
```
Then under the root path of incubator-iotdb, you can build IoTDB using Maven:
```
> pwd
/workspace/incubator-iotdb
> mvn clean package -pl server -am -Dmaven.test.skip=true
```
If successful, you will see the the following text in the terminal:
Otherwise, you may need to check the error statements and fix the problems.
After building, the IoTDB project will be at the subfolder named iotdb. The folder will include the following contents:
```
$IOTDB_HOME/
|
+- sbin/ <-- script files
|
+- conf/ <-- configuration files
|
+- lib/ <-- project dependencies
```
<!-- > NOTE: We also provide already built JARs and project at [http://tsfile.org/download](http://tsfile.org/download) instead of build the jar package yourself. -->
### Installation by Docker (Dockerfile)
You can build and run a IoTDB docker image by following the guide of [Deployment by Docker](/#/Documents/progress/chap3/sec3)
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
# Chapter 3: Server
## Download
IoTDB provides you three installation methods, you can refer to the following suggestions, choose one of them:
* Installation from source code. If you need to modify the code yourself, you can use this method.
* Installation from binary files. Download the binary files from the official website. This is the recommended method, in which you will get a binary released package which is out-of-the-box.(Comming Soon...)
* Using Docker：The path to the dockerfile is https://github.com/apache/incubator-iotdb/blob/master/docker/Dockerfile
### Prerequisites
To use IoTDB, you need to have:
1. Java >= 1.8 (Please make sure the environment path has been set)
2. Maven >= 3.1 (Optional)
3. Set the max open files num as 65535 to avoid "too many open files" problem.
>Note: If you don't have maven installed, you should replace 'mvn' in the following commands with 'mvnw.sh' or 'mvnw.cmd'.
### Installation from binary files
You can download the binary file from:
[Here](https://iotdb.apache.org/#/Download)
### Installation from source code
You can get the released source code from https://iotdb.apache.org/#/Download, or from the git repository https://github.com/apache/incubator-iotdb/tree/master
Then the binary version (including both server and client) can be found at **distribution/target/apache-iotdb-{project.version}-incubating-bin.zip**
> NOTE: Directories "service-rpc/target/generated-sources/thrift" and "server/target/generated-sources/antlr3" need to be added to sources roots to avoid compilation errors in IDE.
If you would like to build the IoTDB server, you can run the following command under the root path of incubator-iotdb:
```
> mvn clean package -pl server -am -DskipTests
```
After build, the IoTDB server will be at the folder "server/target/iotdb-server-{project.version}".
### Installation by Docker (Dockerfile)
You can build and run a IoTDB docker image by following the guide of [Deployment by Docker](/#/Documents/progress/chap3/sec5)
Before starting to use IoTDB, you need to config the configuration files first. For your convenience, we have already set the default config in the files.
In total, we provide users three kinds of configurations module:
* environment configuration file (`iotdb-env.bat`, `iotdb-env.sh`). The default configuration file for the environment configuration item. Users can configure the relevant system configuration items of JAVA-JVM and set environment variable in the file.
* environment configuration file (`iotdb-env.bat`, `iotdb-env.sh`). The default configuration file for the environment configuration item. Users can configure the relevant system configuration items of JAVA-JVM in the file.
* system configuration file (`tsfile-format.properties`, `iotdb-engine.properties`).
*`tsfile-format.properties`: The default configuration file for the IoTDB file layer configuration item. Users can configure the information about the TsFile, such as the data size written to the disk per time(`group_size_in_byte`).
*`iotdb-engine.properties`: The default configuration file for the IoTDB engine layer configuration item. Users can configure the IoTDB engine related parameters in the file, such as JDBC service listening port (`rpc_port`), unsequence data storage directory (`unsequence_data_dir`), etc.
IoTDB provides Cli/shell tools for users to interact with IoTDB server in command lines. This document will show how Cli/shell tool works and what does it parameters mean.
> Note: In this document, \$IOTDB\_HOME represents the path of the IoTDB installation directory.
## Build client from source code
Under the root path of incubator-iotdb:
```
> mvn clean package -pl client -am -DskipTests
```
After build, the IoTDB client will be at the folder "client/target/iotdb-client-{project.version}".
## Running Cli/Shell
After installation, there is a default user in IoTDB: `root`, and the
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
# Chapter 4: Operation Manual
## Data Import
### Import Historical Data
This feature is not supported in version 0.7.0.
### Import Real-time Data
IoTDB provides users with a variety of ways to insert real-time data, such as directly inputting [INSERT SQL statement](/#/Documents/progress/chap4/sec7) in [Client/Shell tools](/#/Tools/Client), or using [Java JDBC](/#/Documents/progress/chap4/sec7) to perform single or batch execution of [INSERT SQL statement](/#/Documents/progress/chap4/sec7).
This section mainly introduces the use of [INSERT SQL statement](/#/Documents/progress/chap4/sec7) for real-time data import in the scenario. See Section 4.7 for a detailed syntax of [INSERT SQL statement](/#/Documents/progress/chap4/sec7).
#### Use of INSERT Statements
The [INSERT SQL statement](/#/Documents/progress/chap4/sec7) statement can be used to insert data into one or more specified timeseries that have been created. For each point of data inserted, it consists of a [timestamp](/#/Documents/progress/chap2/sec1) and a sensor acquisition value of a numerical type (see [Data Type](/#/Documents/progress/chap2/sec2)).
In the scenario of this section, take two timeseries `root.ln.wf02.wt02.status` and `root.ln.wf02.wt02.hardware` as an example, and their data types are BOOLEAN and TEXT, respectively.
The sample code for single column data insertion is as follows:
```
IoTDB > insert into root.ln.wf02.wt02(timestamp,status) values(1,true)
IoTDB > insert into root.ln.wf02.wt02(timestamp,hardware) values(1, "v1")
```
The above example code inserts the long integer timestamp and the value "true" into the timeseries `root.ln.wf02.wt02.status` and inserts the long integer timestamp and the value "v1" into the timeseries `root.ln.wf02.wt02.hardware`. When the execution is successful, cost time is shown to indicate that the data insertion has been completed.
> Note: In IoTDB, TEXT type data can be represented by single and double quotation marks. The insertion statement above uses double quotation marks for TEXT type data. The following example will use single quotation marks for TEXT type data.
The INSERT statement can also support the insertion of multi-column data at the same time point. The sample code of inserting the values of the two timeseries at the same time point '2' is as follows:
After inserting the data, we can simply query the inserted data using the SELECT statement:
```
IoTDB > select * from root.ln.wf02 where time < 3
```
The result is shown below. From the query results, it can be seen that the insertion statements of single column and multi column data are performed correctly.
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
# Chapter 4: Operation Manual
## Data Maintenance
<!-- >
### Data Update
Users can use [UPDATE statements](/#/Documents/0.8.0/chap5/sec1) to update data over a period of time in a specified timeseries. When updating data, users can select a timeseries to be updated (version 0.8.0 does not support multiple timeseries updates) and specify a time point or period to be updated (version 0.8.0 must have time filtering conditions).
In a JAVA programming environment, you can use the [Java JDBC](/#/Documents/0.8.0/chap6/sec1) to execute single or batch UPDATE statements.
#### Update Single Timeseries
Taking the power supply status of ln group wf02 plant wt02 device as an example, there exists such a usage scenario:
After data access and analysis, it is found that the power supply status from 2017-11-01 15:54:00 to 2017-11-01 16:00:00 is true, but the actual power supply status is abnormal. You need to update the status to false during this period. The SQL statement for this operation is:
```
update root.ln.wf02 SET wt02.status = false where time <=2017-11-01T16:00:00 and time >= 2017-11-01T15:54:00
```
It should be noted that when the updated data type does not match the actual data type, IoTDB will give the corresponding error prompt as shown below:
```
IoTDB> update root.ln.wf02 set wt02.status = 1205 where time < now()
error: The BOOLEAN data type should be true/TRUE or false/FALSE
```
When the updated path does not exist, IoTDB will give the corresponding error prompt as shown below:
```
IoTDB> update root.ln.wf02 set wt02.sta = false where time < now()
Msg: do not select any existing series
```
-->
### Data Deletion
Users can delete data that meet the deletion condition in the specified timeseries by using the [DELETE statement](/#/Documents/progress/chap4/sec7). When deleting data, users can select one or more timeseries paths, prefix paths, or paths with star to delete data before a certain time (version 0.8.0 does not support the deletion of data within a closed time interval).
In a JAVA programming environment, you can use the [Java JDBC](/#/Documents/progress/chap6/sec1) to execute single or batch UPDATE statements.
#### Delete Single Timeseries
Taking ln Group as an example, there exists such a usage scenario:
The wf02 plant's wt02 device has many segments of errors in its power supply status before 2017-11-01 16:26:00, and the data cannot be analyzed correctly. The erroneous data affected the correlation analysis with other devices. At this point, the data before this time point needs to be deleted. The SQL statement for this operation is
```
delete from root.ln.wf02.wt02.status where time<=2017-11-01T16:26:00;
```
#### Delete Multiple Timeseries
When both the power supply status and hardware version of the ln group wf02 plant wt02 device before 2017-11-01 16:26:00 need to be deleted, [the prefix path with broader meaning or the path with star](/#/Documents/progress/chap2/sec1) can be used to delete the data. The SQL statement for this operation is:
```
delete from root.ln.wf02.wt02 where time <= 2017-11-01T16:26:00;
```
or
```
delete from root.ln.wf02.wt02.* where time <= 2017-11-01T16:26:00;
```
It should be noted that when the deleted path does not exist, IoTDB will give the corresponding error prompt as shown below:
```
IoTDB> delete from root.ln.wf03.wt02.status where time < now()
Msg: TimeSeries does not exist and its data cannot be deleted
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
# Chapter 5: Management
## Performance Monitor
### Introduction
In order to grasp the performance of iotdb, we add this module to count the time-consuming of each operation. This module can statistic the avg time-consuming of each operation and the proportion of each operation fall into a time range. The output is in log_measure.log file. A output example is in below.
1. EnableStat：Whether the statistics are enable or not, if it is true, the module records the time-consuming of each operation and prints the results; It can not be set dynamically but changed by function in below.
2. DisplayIntervalInMs：The interval between print results. It can be set dynamically, but will take effect after restart.( First call stopStatistic(), then call startContinuousStatistics() or startOneTimeStatistics()）
3. OperationSwitch：It's a map to indicate whether stat the operation, the key is operation name and the value is stat state. This parameter cannot be changed directly, it's change by operation 'changeOperationSwitch()'.
**Operation**
1. startContinuousStatistics： Start the statistics and output at interval of ‘DisplayIntervalInMs’.
2. startOneTimeStatistics：Start the statistics and output in delay of ‘DisplayIntervalInMs’.
3. stopStatistic：Stop the statistics.
4. clearStatisticalState(): clear current stat result, reset statistical result.
5. changeOperationSwitch(String operationName, Boolean operationState):set whether to monitor operation status. The param 'operationName' is the name of operation, defined in attribute operationSwitch. The param operationState is the state of operation. If state-switch successful the function will return true, else return false.
### Adding Custom Monitoring Items for developer of IOTDB
**Add Operation**
Add an enumeration in org.apache.iotdb.db.cost.statistic.Operation.
To improve query performance, IOTDB caches ChunkMetaData and TsFileMetaData. Users can view the cache hit rate through debug level log and MXBean, and adjust the memory occupied by the cache according to the cache hit rate and system memory. The method of using MXBean to view cache hit ratio is as follows:
1. Connect to jconsole with port 31999 and select 'MBean' in the menu item above.
2. Expand the sidebar and select 'org.apache.iotdb.db.service'. You will get the results shown in the following figure:
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
# Chapter 5: Management
## System log
IoTDB allows users to configure IoTDB system logs (such as log output level) by modifying the log configuration file. The default location of the system log configuration file is in \$IOTDB_HOME/conf folder.
The default log configuration file is named logback.xml. The user can modify the configuration of the system running log by adding or changing the xml tree node parameters. It should be noted that the configuration of the system log using the log configuration file does not take effect immediately after the modification, instead, it will take effect after restarting the system. The usage of logback.xml is just as usual.
At the same time, in order to facilitate the debugging of the system by the developers and DBAs, we provide several JMX interface to dynamically modify the log configuration, and configure the Log module of the system in real time without restarting the system. For detailed usage, see [Dynamic System Log Configuration](/#/Documents/progress/chap5/sec4) section.
### Dynamic System Log Configuration
#### Connect JMX
Here we use JConsole to connect with JMX.
Start the JConsole, establish a new JMX connection with the IoTDB Server (you can select the local process or input the IP and PORT for remote connection, the default operation port of the IoTDB JMX service is 31999). Fig 4.1 shows the connection GUI of JConsole.
This method is to reload the default logback configuration file. The user can modify the default configuration file first, and then call this method to reload the modified configuration file into the system to take effect.
* reloadByFileName
This method loads a logback configuration file with the specified path and name, and then makes it take effect. This method accepts a parameter of type String named p1, which is the path to the configuration file that needs to be specified for loading.
* getLoggerEffectiveLevel
This method is to obtain the current log level of the specified Logger. This method accepts a String type parameter named p1, which is the name of the specified Logger. This method returns the log level currently in effect for the specified Logger.
* getLoggerLevel
This method is to obtain the log level of the specified Logger. This method accepts a String type parameter named p1, which is the name of the specified Logger. This method returns the log level of the specified Logger.
It should be noted that the difference between this method and the `getLoggerEffectiveLevel` method is that the method returns the log level that the specified Logger is set in the configuration file. If the user does not set the log level for the Logger. , then return empty. According to Logre's log-level inheritance mechanism, if a Logger is not displayed to set the log level, it will inherit the log level settings from its nearest ancestor. At this point, calling the `getLoggerEffectiveLevel` method will return the log level in which the Logger is in effect; calling the methods described in this section will return null.
To make this manual more practical, we will use a specific scenario example to illustrate how to operate IoTDB databases at all stages of use. See [this page](https://github.com/apache/incubator-iotdb/blob/master/docs/Documentation/OtherMaterial-Sample%20Data.txt) for a look. For your convenience, we also provide you with a sample data file in real scenario to import into the IoTDB system for trial and operation.
Before importing data to IoTDB, we first select the appropriate data storage model according to the [sample data](https://raw.githubusercontent.com/apache/incubator-iotdb/master/docs/Documentation/OtherMaterial-Sample%20Data.txt), and then create the storage group and timeseries using [SET STORAGE GROUP](/#/Documents/progress/chap4/sec7) statement and [CREATE TIMESERIES](/#/Documents/progress/chap4/sec7) statement respectively.
### Storage Model Selection
According to the data attribute layers described in [sample data](https://raw.githubusercontent.com/apache/incubator-iotdb/master/docs/Documentation/OtherMaterial-Sample%20Data.txt), we can express it as an attribute hierarchy structure based on the coverage of attributes and the subordinate relationship between them, as shown in Figure 3.1 below. Its hierarchical relationship is: power group layer - power plant layer - device layer - sensor layer. ROOT is the root node, and each node of sensor layer is called a leaf node. In the process of using IoTDB, you can directly connect the attributes on the path from ROOT node to each leaf node with ".", thus forming the name of a timeseries in IoTDB. For example, The left-most path in Figure 3.1 can generate a timeseries named `ROOT.ln.wf01.wt01.status`.
After getting the name of the timeseries, we need to set up the storage group according to the actual scenario and scale of the data. Because in the scenario of this chapter data is usually arrived in the unit of groups (i.e., data may be across electric fields and devices), in order to avoid frequent switching of IO when writing data, and to meet the user's requirement of physical isolation of data in the unit of groups, we set the storage group at the group layer.
### Storage Group Creation
After selecting the storage model, according to which we can set up the corresponding storage group. The SQL statements for creating storage groups are as follows:
### Create Storage Group
According to the storage model we can set up the corresponding storage group. The SQL statements for creating storage groups are as follows:
After the storage group is created, we can use the [SHOW STORAGE GROUP](/#/Documents/progress/chap4/sec7) statement to view all the storage groups. The SQL statement is as follows:
After the storage group is created, we can use the [SHOW STORAGE GROUP](/#/Documents/progress/chap5/sec4) statement to view all the storage groups. The SQL statement is as follows:
According to the storage model selected before, we can create corresponding timeseries in the two storage groups respectively. The SQL statements for creating timeseries are as follows:
```
...
...
@@ -107,10 +90,48 @@ The results are shown below respectly:
It is worth noting that when the queried path does not exist, the system will return no timeseries.
### Precautions
### Count Timeseries
IoTDB are able to use `COUNT TIMESERIES<Path>` to count the amount of timeseries in the path. SQL statements are as follows:
```
IoTDB > COUNT TIMESERIES root
IoTDB > COUNT TIMESERIES root.ln
IoTDB > COUNT TIMESERIES root.ln.*.*.status
IoTDB > COUNT TIMESERIES root.ln.wf01.wt01.status
```
### Delete Timeseries
To delete the timeseries we created before, we are able to use `DELETE TimeSeries <PrefixPath>` statement.
Version 0.7.0 imposes some limitations on the scale of data that users can operate:
IoTDB supports users to check the devices using `SHOW DEVICES` statement.
SQL statement is as follows:
```
IoTDB> show devices
```
## TTL
IoTDB supports storage-level TTL settings, which means it is able to delete old data automatically and periodically. The benefit of using TTL is that hopefully you can control the total disk space usage and prevent the machine from running out of disks. Moreover, the query performance may downgrade as the total number of files goes up and the memory usage also increase as there are more files. Timely removing such files helps to keep at a high query performance level and reduce memory usage.
### Set TTL
The SQL Statement for setting TTL is as follow:
```
IoTDB> set ttl to root.ln 3600000
```
This example means that for data in `root.ln`, only that of the latest 1 hour will remain, the older one is removed or made invisible.
### Unset TTL
To unset TTL, we can use follwing SQL statement:
```
IoTDB> unset ttl to root.ln
```
After unset TTL, all data will be accepted in `root.ln`
Limit 1: Assuming that the JVM memory allocated to IoTDB at runtime is p and the user-defined size of data in memory written to disk ([group\_size\_in\_byte](/#/Documents/progress/chap3/sec2)) is q, then the number of storage groups should not exceed p/q.
Limit 2: The number of timeseries should not exceed the ratio of JVM memory allocated to IoTDB at run time to 20KB.
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
...
...
@@ -19,12 +19,73 @@
-->
# Chapter 4: Operation Manual
# Chapter 5: Operation Manual
# DML (Data Manipulation Language)
## INSERT
### Insert Real-time Data
## Data Query
IoTDB provides users with a variety of ways to insert real-time data, such as directly inputting [INSERT SQL statement](/#/Documents/progress/chap5/sec4) in [Client/Shell tools](/#/Tools/Client), or using [Java JDBC](/#/Documents/progress/chap4/sec2) to perform single or batch execution of [INSERT SQL statement](/#/Documents/progress/chap5/sec4).
This section mainly introduces the use of [INSERT SQL statement](/#/Documents/progress/chap5/sec4) for real-time data import in the scenario. See Section 5.4 for a detailed syntax of [INSERT SQL statement](/#/Documents/progress/chap5/sec4).
#### Use of INSERT Statements
The [INSERT SQL statement](/#/Documents/progress/chap5/sec4) statement can be used to insert data into one or more specified timeseries that have been created. For each point of data inserted, it consists of a [timestamp](/#/Documents/progress/chap2/sec1) and a sensor acquisition value (see [Data Type](/#/Documents/progress/chap2/sec2)).
In the scenario of this section, take two timeseries `root.ln.wf02.wt02.status` and `root.ln.wf02.wt02.hardware` as an example, and their data types are BOOLEAN and TEXT, respectively.
The sample code for single column data insertion is as follows:
```
IoTDB > insert into root.ln.wf02.wt02(timestamp,status) values(1,true)
IoTDB > insert into root.ln.wf02.wt02(timestamp,hardware) values(1, "v1")
```
The above example code inserts the long integer timestamp and the value "true" into the timeseries `root.ln.wf02.wt02.status` and inserts the long integer timestamp and the value "v1" into the timeseries `root.ln.wf02.wt02.hardware`. When the execution is successful, cost time is shown to indicate that the data insertion has been completed.
> Note: In IoTDB, TEXT type data can be represented by single and double quotation marks. The insertion statement above uses double quotation marks for TEXT type data. The following example will use single quotation marks for TEXT type data.
The INSERT statement can also support the insertion of multi-column data at the same time point. The sample code of inserting the values of the two timeseries at the same time point '2' is as follows:
After inserting the data, we can simply query the inserted data using the SELECT statement:
```
IoTDB > select * from root.ln.wf02 where time < 3
```
The result is shown below. From the query results, it can be seen that the insertion statements of single column and multi column data are performed correctly.
If the user inserts data into a non-existent timeseries, for example, execute the following commands:
```
IoTDB > insert into root.ln.wf02.wt02(timestamp, temperature) values(1,"v1")
```
Because `root.ln.wf02.wt02. temperature` does not exist, the system will return the following ERROR information:
```
Msg: Current deviceId[root.ln.wf02.wt02] does not contain measurement:temperature
```
If the data type inserted by the user is inconsistent with the corresponding data type of the timeseries, for example, execute the following command:
```
IoTDB > insert into root.ln.wf02.wt02(timestamp,hardware) values(1,100)
```
The system will return the following ERROR information:
```
error: The TEXT data type should be covered by " or '
```
## SELECT
### Time Slice Query
This chapter mainly introduces the relevant examples of time slice query using IoTDB SELECT statements. Detailed SQL syntax and usage specifications can be found in [SQL Documentation](/#/Documents/progress/chap4/sec7). You can also use the [Java JDBC](/#/Documents/progress/chap6/sec1) standard interface to execute related queries.
This chapter mainly introduces the relevant examples of time slice query using IoTDB SELECT statements. Detailed SQL syntax and usage specifications can be found in [SQL Documentation](/#/Documents/progress/chap5/sec4). You can also use the [Java JDBC](/#/Documents/progress/chap4/sec2) standard interface to execute related queries.
#### Select a Column of Data Based on a Time Interval
...
...
@@ -83,7 +144,7 @@ The execution result of this SQL statement is as follows:
This section mainly introduces the related examples of down-frequency aggregation query, using the [GROUP BY clause](/#/Documents/progress/chap4/sec7), which is used to partition the result set according to the user's given partitioning conditions and aggregate the partitioned result set. IoTDB supports partitioning result sets according to time intervals, and by default results are sorted by time in ascending order. You can also use the [Java JDBC](/#/Documents/progress/chap6/sec1) standard interface to execute related queries.
This section mainly introduces the related examples of down-frequency aggregation query, using the [GROUP BY clause](/#/Documents/progress/chap5/sec4), which is used to partition the result set according to the user's given partitioning conditions and aggregate the partitioned result set. IoTDB supports partitioning result sets according to time intervals, and by default results are sorted by time in ascending order. You can also use the [Java JDBC](/#/Documents/progress/chap4/sec2) standard interface to execute related queries.
The GROUP BY statement provides users with three types of specified parameters:
...
...
@@ -95,7 +156,7 @@ The actual meanings of the three types of parameters are shown in Figure 3.2 bel
**Figure 4.2 The actual meanings of the three types of parameters**</center>
**Figure 5.2 The actual meanings of the three types of parameters**</center>
#### Down-Frequency Aggregate Query without Specifying the Time Axis Origin Position
The SQL statement is:
...
...
@@ -167,7 +228,7 @@ In the actual use of IoTDB, when doing the query operation of timeseries, situat
Automated fill function refers to filling empty values according to the user's specified method and effective time range when performing timeseries queries for single or multiple columns. If the queried point's value is not null, the fill function will not work.
> Note: In the current version 0.7.0, IoTDB provides users with two methods: Previous and Linear. The previous method fills blanks with previous value. The linear method fills blanks through linear fitting. And the fill function can only be used when performing point-in-time queries.
> Note: In the current version, IoTDB provides users with two methods: Previous and Linear. The previous method fills blanks with previous value. The linear method fills blanks through linear fitting. And the fill function can only be used when performing point-in-time queries.
#### Fill Function
* Previous Function
...
...
@@ -273,9 +334,9 @@ When the fill method is not specified, each data type bears its own default fill
### Row and Column Control over Query Results
IoTDB provides [LIMIT/SLIMIT](/#/Documents/progress/chap5/sec1) clause and [OFFSET/SOFFSET](/#/Documents/progress/chap5/sec1) clause in order to make users have more control over query results. The use of LIMIT and SLIMIT clauses allows users to control the number of rows and columns of query results, and the use of OFFSET and SOFSET clauses allows users to set the starting position of the results for display.
IoTDB provides [LIMIT/SLIMIT](/#/Documents/progress/chap5/sec4) clause and [OFFSET/SOFFSET](/#/Documents/progress/chap5/sec4) clause in order to make users have more control over query results. The use of LIMIT and SLIMIT clauses allows users to control the number of rows and columns of query results, and the use of OFFSET and SOFSET clauses allows users to set the starting position of the results for display.
This chapter mainly introduces related examples of row and column control of query results. You can also use the [Java JDBC](/#/Documents/progress/chap6/sec1) standard interface to execute queries.
This chapter mainly introduces related examples of row and column control of query results. You can also use the [Java JDBC](/#/Documents/progress/chap4/sec2) standard interface to execute queries.
#### Row Control over Query Results
By using LIMIT and OFFSET clauses, users can control the query results in a row-related manner. We will demonstrate how to use LIMIT and OFFSET clauses through the following examples.
...
...
@@ -483,3 +544,38 @@ select * from root.ln.wf01.wt01 where time > 2017-11-01T00:05:00.000 and time <
```
The SQL statement will not be executed and the corresponding error prompt is given as follows:
Users can delete data that meet the deletion condition in the specified timeseries by using the [DELETE statement](/#/Documents/progress/chap5/sec4). When deleting data, users can select one or more timeseries paths, prefix paths, or paths with star to delete data before a certain time (current version does not support the deletion of data within a closed time interval).
In a JAVA programming environment, you can use the [Java JDBC](/#/Documents/progress/chap4/sec2) to execute single or batch UPDATE statements.
### Delete Single Timeseries
Taking ln Group as an example, there exists such a usage scenario:
The wf02 plant's wt02 device has many segments of errors in its power supply status before 2017-11-01 16:26:00, and the data cannot be analyzed correctly. The erroneous data affected the correlation analysis with other devices. At this point, the data before this time point needs to be deleted. The SQL statement for this operation is
```
delete from root.ln.wf02.wt02.status where time<=2017-11-01T16:26:00;
```
### Delete Multiple Timeseries
When both the power supply status and hardware version of the ln group wf02 plant wt02 device before 2017-11-01 16:26:00 need to be deleted, [the prefix path with broader meaning or the path with star](/#/Documents/progress/chap2/sec1) can be used to delete the data. The SQL statement for this operation is:
```
delete from root.ln.wf02.wt02 where time <= 2017-11-01T16:26:00;
```
or
```
delete from root.ln.wf02.wt02.* where time <= 2017-11-01T16:26:00;
```
It should be noted that when the deleted path does not exist, IoTDB will give the corresponding error prompt as shown below:
```
IoTDB> delete from root.ln.wf03.wt02.status where time < now()
Msg: TimeSeries does not exist and its data cannot be deleted
IoTDB provides users with priviledge management operations, so as to ensure data security.
## Account Management Statements
IoTDB provides users with account priviledge management operations, so as to ensure data security.
We will show you basic user priviledge management operations through the following specific examples. Detailed SQL syntax and usage details can be found in [SQL Documentation](/#/Documents/progress/chap4/sec7). At the same time, in the JAVA programming environment, you can use the [Java JDBC](/#/Documents/progress/chap6/sec1) to execute priviledge management statements in a single or batch mode.
We will show you basic user priviledge management operations through the following specific examples. Detailed SQL syntax and usage details can be found in [SQL Documentation](/#/Documents/progress/chap5/sec4). At the same time, in the JAVA programming environment, you can use the [Java JDBC](/#/Documents/progress/chap4/sec2) to execute priviledge management statements in a single or batch mode.
In this part, we will introduce you IoTDB's Query Language. IoTDB offers you a SQL-like query language for interacting with IoTDB, the query language can be devided into 4 major parts:
* Schema Statement: statements about schema management are all listed in this section.
...
...
@@ -29,8 +30,6 @@ In this part, we will introduce you IoTDB's Query Language. IoTDB offers you a S
All of these statements are write in IoTDB's own syntax, for details about the syntax composition, please check the `Reference` section.
## IoTDB Query Statement
### Schema Statement
...
...
@@ -138,6 +137,13 @@ Note: The path can be prefix path or timeseries path.
Note: This statement can be used in IoTDB Client and JDBC.
```
* Show Devices Statement
```
SHOW DEVICE
Eg: IoTDB > SHOW DEVICES
Note: This statement can be used in IoTDB Client and JDBC.
```
### Data Management Statement
* Insert Record Statement
...
...
@@ -524,8 +530,8 @@ Eg: IoTDB > LIST ALL USER OF ROLE roleuser;
```
ALTER USER <username> SET PASSWORD <password>;
roleName:=identifier
password:=string
Eg: IoTDB > UPDATE USER tempuser SET PASSWORD 'newpwd';
password:=identifier
Eg: IoTDB > ALTER USER tempuser SET PASSWORD 'newpwd';
Currently, IoTDB provides users to use Java's JConsole tool to monitor system status or use IoTDB's open API to check data status.
...
...
@@ -357,3 +357,115 @@ Here are the file size statistics:
|Timeseries Name| root.stats.file\_size.SCHEMA |
|Reset After Restarting System| No |
|Example| select SCHEMA from root.stats.file\_size.SCHEMA|
## Performance Monitor
### Introduction
In order to grasp the performance of iotdb, we add this module to count the time-consumption of each operation. This module can compute the statistics of the avg time-consuming of each operation and the proportion of each operation whose time consumption falls into a time range. The output is in log_measure.log file. An output example is below.
1. EnableStat：Whether the statistics are enabled or not, if it is true, the module records the time-consuming of each operation and prints the results; It is non-editable but can be changed by the function below.
2. DisplayIntervalInMs：The interval between print results. The changes will not take effect instantly. To make the changes effective, you should call startContinuousStatistics() or startOneTimeStatistics().
3. OperationSwitch：It's a map to indicate whether the statistics of one kind of operation should be computed, the key is operation name and the value is true means the statistics of the operation are enabled, otherwise disabled. This parameter cannot be changed directly, it's changed by operation 'changeOperationSwitch()'.
**Operation**
1. startContinuousStatistics： Start the statistics and output at interval of ‘DisplayIntervalInMs’.
2. startOneTimeStatistics：Start the statistics and output in delay of ‘DisplayIntervalInMs’.
3. stopStatistic：Stop the statistics.
4. clearStatisticalState(): clear current stat result, reset statistical result.
5. changeOperationSwitch(String operationName, Boolean operationState):set whether to monitor a kind of operation. The param 'operationName' is the name of operation, defined in attribute operationSwitch. The param operationState is whether to enable the statistics or not. If the state is switched successfully, the function will return true, else return false.
### Adding Custom Monitoring Items for contributors of IOTDB
**Add Operation**
Add an enumeration in org.apache.iotdb.db.cost.statistic.Operation.
To improve query performance, IOTDB caches ChunkMetaData and TsFileMetaData. Users can view the cache hit ratio through debug level log and MXBean, and adjust the memory occupied by the cache according to the cache hit ratio and system memory. The method of using MXBean to view cache hit ratio is as follows:
1. Connect to jconsole with port 31999 and select 'MBean' in the menu item above.
2. Expand the sidebar and select 'org.apache.iotdb.db.service'. You will get the results shown in the following figure:
IoTDB allows users to configure IoTDB system logs (such as log output level) by modifying the log configuration file. The default location of the system log configuration file is in \$IOTDB_HOME/conf folder.
The default log configuration file is named logback.xml. The user can modify the configuration of the system running log by adding or changing the xml tree node parameters. It should be noted that the configuration of the system log using the log configuration file does not take effect immediately after the modification, instead, it will take effect after restarting the system. The usage of logback.xml is just as usual.
At the same time, in order to facilitate the debugging of the system by the developers and DBAs, we provide several JMX interfaces to dynamically modify the log configuration, and configure the Log module of the system in real time without restarting the system.
### Dynamic System Log Configuration
#### Connect JMX
Here we use JConsole to connect with JMX.
Start the JConsole, establish a new JMX connection with the IoTDB Server (you can select the local process or input the IP and PORT for remote connection, the default operation port of the IoTDB JMX service is 31999). Fig 4.1 shows the connection GUI of JConsole.
This method is to reload the default logback configuration file. The user can modify the default configuration file first, and then call this method to reload the modified configuration file into the system to take effect.
* reloadByFileName
This method loads a logback configuration file with the specified path and name, and then makes it take effect. This method accepts a parameter of type String named p1, which is the path to the configuration file that needs to be specified for loading.
* getLoggerEffectiveLevel
This method is to obtain the current log level of the specified Logger. This method accepts a String type parameter named p1, which is the name of the specified Logger. This method returns the log level currently in effect for the specified Logger.
* getLoggerLevel
This method is to obtain the log level of the specified Logger. This method accepts a String type parameter named p1, which is the name of the specified Logger. This method returns the log level of the specified Logger.
It should be noted that the difference between this method and the `getLoggerEffectiveLevel` method is that the method returns the log level that the specified Logger is set in the configuration file. If the user does not set the log level for the Logger. , then return empty. According to Logger's log-level inheritance mechanism, f a Logger's level is not explicitly set, it will inherit the log level settings from its nearest ancestor. At this point, calling the `getLoggerEffectiveLevel` method will return the log level in which the Logger is in effect; calling `getLoggerLevel` will return null.
In IoTDB, there are many kinds of data needed to be storage. In this section, we will introduce IoTDB's data storage strategy in order to give you an intuitive understanding of IoTDB's data management.
