提交 c4edb29a 编写于 作者: 一位用户's avatar 一位用户 🎨 提交者: hustjieke

docs(stonedb): update the latest documentation

上级 bfb209a5
......@@ -4,8 +4,7 @@ sidebar_position: 1.3
---
# Limits
StoneDB is built on MySQL by integrating a storage engine into MySQL. Therefore, StoneDB is highly compatible with the MySQL 5.6 and 5.7 protocols, and the ecosystem, common features, and common syntaxes of MySQL. However, due to characteristics of column-based storage, StoneDB is incompatible with certain MySQL operations and features.
As a conlumn-based storage engine, StoneDB is built on MySQL. Therefore, StoneDB is highly compatible with the MySQL 5.6 and 5.7 protocols, and the ecosystem, common features, and common syntaxes of MySQL. However, due to characteristics of column-based storage, StoneDB is incompatible with certain MySQL operations and features.
## Unsupported DDL operations
StoneDB does not support the following DDL operations:
......@@ -26,10 +25,10 @@ StoneDB does not support the following DDL operations:
- Add a unique constraint.
- Delete a unique constraint.
- Create an index.
- Delete an index.
- Remove an index.
- Modify a table comment.
StoneDB is a column-oriented database. Data in StoneDB is highly compressed. For this reason, table attributes and column attributes are difficult to modify. The character sets, data types, constraints, and indexes must be properly defined when tables are being created.
Data stored in StoneDB is highly compressed. For this reason, table attributes and column attributes are difficult to modify. The character sets, data types, constraints, and indexes must be properly defined when tables are being created.
## Unsupported DML operations
StoneDB does not support the following DML operations:
......@@ -64,13 +63,13 @@ StoneDB does not support the following binary log formats:
- row
- mixed
Column-oriented databases supports only statement-based binary logs. Row-based binary logs and mixed binary logs are not supported.
## Association queries across storage engines not supported
By default, StoneDB does not support association queries across storage engines. If an association query involves tables in both InnoDB and the StoneDB column-based storage engine, an error will be reported. You can set the **stonedb_ini_allowmysqlquerypath** parameter to **1 **in the **stonedb.cnf** configuration file to remove this limit.
Column-based storage engines support only statement-based binary logs. Row-based binary logs and mixed binary logs are not supported.
## Join queries across storage engines not supported
By default, StoneDB does not support join queries across storage engines. If a join query involves tables in both InnoDB and StoneDB, an error will be reported. You can set the **stonedb_ini_allowmysqlquerypath** parameter to **1** in the **stonedb.cnf** configuration file to remove this limit.
## Transactions not supported
Transactions must strictly comply with the ACID attributes. However, StoneDB does not support redo and undo logs and thus does not support transactions.
## Partitions not supported
Column-oriented databases do not support partitioning.
Column-based storage engines do not support partitioning.
## Column locking and table locking not supported
Column-oriented databases do not support column locking or table locking.
Column-based storage engines do not support column locking or table locking.
---
id: basic-operations
sidebar_position: 3.3
---
# Basic Operations
Structured Query Language (SQL) is a programming language for communicating with databases. You can use it to manage relational databases by performing insert, query, update, and other operations.
StoneDB is compatible with MySQL. You can use clients supported by MySQL to connect to StoneDB. In addition, StoneDB supports most SQL syntaxes. This section describes the basic SQL operations supported by StoneDB.
SQL can be classified into the following four parts by usage:
- Data Definition Language (DDL): is used to manage database objects, such as CREATE, ALTER, and DROP statements.
- Data Manipulation Language (DML): is used to manage data in tables, such as INSERT, DELETE, and UPDATE statements.
- Data Query Language (DQL): is used to query objects, such as SELECT statements.
- Data Control Language (DCL): is used to control access to data, such as GRANT and REVOKE statements.
## Operations on databases
This section provides examples of performing basic operations on databases.
### Create a database
Execute the following SQL statement to create a database named **test_db** and set the default character set to **utf8mb4**:
```sql
create database test_db DEFAULT CHARACTER SET utf8mb4;
```
### List databases
Execute the following SQL statement to list databases:
```sql
show databases;
```
### Use a database
Execute the following SQL statement to use database **test_db**:
```sql
use test_db;
```
### Drop a database
Execute the following SQL statement to drop database** test_db**:
```sql
drop database test_db;
```
## Operations on tables
This section provides examples of performing basic operations on tables.
### Create a StoneDB table
Execute the following SQL statement to create a table which is named **student** and consists of the **id**, **name**, **age**, and **birthday** fields:
```sql
create table student(
id int(11) primary key,
name varchar(255),
age smallint,
birthday DATE
) engine=stonedb;
```
:::info
If you do not specify **engine=stonedb** in the SQL statement, the storage engine on which the table is created is determined by the value of parameter **default_storage_engine**. For more information, see [Configure parameters](https://stoneatom.yuque.com/staff-ft8n1u/dghuxr/xg9czr).
:::
### Query the schema of a table
Execute the following SQL statement to query the schema of table **student**:
```sql
show create table student\G
```
### Drop a table
Execute the following SQL statement to drop table **student**:
```sql
drop table student;
```
## Operations on data
This section provides examples of performing basic operations on data.
### Insert data into a table
Execute the following SQL statement to insert a record to table **student**:
```sql
insert into student values(1,'Jack',15,'20220506');
```
### Modify data in a table
Execute the following UPDATE statement to modify data in table **student**:
```sql
update student set age=25 where id=1;
```
### Remove data from a table
#### Clear data in a table
Execute the following TRUNCATE statement to clear data in table **student**:
```sql
truncate table student ;
```
#### Remove specific data from a table
As a column-based storage engine, StoneDB does not support DELETE operations.
### Query data from a table
Execute a SELECT statement to query data from a table.
Example 1: Query the name and birthday of the student whose **id** is **1** from table **student**.
```sql
select name,birthday from student where id=1;
```
Example 2: Query the name and birthday of each student and sort the result by birthday from table **student**.
```sql
select name,birthday from student order by birthday;
```
## Operations on users
This section provides examples of performing basic operations on users.
### Create a user
Execute the following SQL statement to create a user named **tiger** and set the password to **123456**:
```sql
create user 'tiger'@'%' identified by '123456';
```
:::info
The username together with the hostname uniquely identify a user in the format of '_username_'@'_host_'. In this way, 'tiger'@'%' and 'tiger'@'localhost' are two users.
:::
### Grant a user permissions
Execute the following SQL statement to grant user **tiger** the permissions to query all tables in database **test_db**:
```sql
grant select on test_db.* to 'tiger'@'%';
```
### Query user permissions
Execute the following SQL statement to query permissions granted to user **tiger**:
```sql
show grants for 'tiger'@'%';
```
### Drop a user
Execute the following SQL statement to drop user '**tiger'@'%'**:
```sql
drop user 'tiger'@'%';
```
......@@ -5,61 +5,67 @@ sidebar_position: 3.1
# Quick Deployment
## Upload and decompress the TAR package
## 1. Download the latest package
Click [here](https://static.stoneatom.com/stonedb-ce-5.6-v1.0.0.el7.x86_64.tar.gz) to download the latest installation package of StoneDB.
## 2. Upload and decompress the TAR package
```shell
tar -zxvf stonedb-ce-5.6-v1.0.0.el7.x86_64.tar.gz
```
tar -zxvf stonedb-build_stonedb5.7_0.1_x86_64_CentOS7.9.2009_Release_2022-05-17_12_06.bin.tar.gz
```
Upload the installation package to the directory. The name of the folder extracted from the package is **stonedb**.
## Check dependencies
```
cd stonedb/install/bin
Upload the installation package to the directory. The name of the folder extracted from the package is **stonedb56**.
## 3. Check dependencies
```bash
cd /stonedb56/install/bin
ldd mysqld
ldd mysql
```
If the command output contains keywords **not found**, some dependencies are missing and must be installed.
## Modify the configuration file
```
cd stonedb/install/
## 4. Modify the configuration file
```bash
cd /stonedb56/install/
cp stonedb.cnf stonedb.cnf.bak
vi stonedb.cnf
```
Modify the path and parameters. If the installation folder is **stonedb**, you only need to modify the parameters.
## Create an account and directories
```
groupadd stonedb
useradd -g stonedb stonedb
passwd stonedb
cd stonedb/install/
mkdir binlog
mkdir log
mkdir tmp
mkdir redolog
mkdir undolog
chown -R stonedb:stonedb stonedb
```
## Initialize StoneDB
## 5. Create an account
```bash
groupadd mysql
useradd -g mysql mysql
passwd mysql
```
/stonedb/install/bin/mysqld --defaults-file=/stonedb/install/stonedb.cnf --initialize-insecure --user=stonedb
## 6. Execute reinstall.sh
```bash
cd /stonedb56/install
./reinstall.sh
```
The process of executing the script is to initialize and start the StoneDB.
## 7. Log in to StoneDB
```shell
/stonedb56/install/bin/mysql -uroot -p -S /stonedb56/install/tmp/mysql.sock
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 1
Server version: 5.6.24-StoneDB-log build-
When you initialize StoneDB, add** parameter --initialize-insecure** to allow the admin to initially log in without the need to enter a password. The admin is required to set a password after the initial login.
Copyright (c) 2000, 2022 StoneAtom Group Holding Limited
No entry for terminal type "xterm";
using dumb terminal settings.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
## Start or stop StoneDB
```
/stonedb/install/bin/mysqld_safe --defaults-file=/stonedb/install/stonedb.cnf --user=stonedb &
mysqladmin -uroot -p -S /stonedb/install/tmp/mysql.sock shutdown
```
## Log in as admin and reset the password
```
mysql -uroot -p -S /stonedb/install/tmp/mysql.sock
>set password = password('MYPASSWORD');
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| cache |
| innodb |
| mysql |
| performance_schema |
| sys_stonedb |
| test |
+--------------------+
7 rows in set (0.00 sec)
```
Replace **MYPASSWORD** with your password.
## 8. Stop StoneDB
```shell
/stonedb56/install/bin/mysqladmin -uroot -p -S /stonedb56/install/tmp/mysql.sock shutdown
```
\ No newline at end of file
......@@ -8,7 +8,7 @@ sidebar_position: 3.2
This topic presents some examples to show you that the StoneDB storage engine has superior performance than InnoDB on processing bulk insert of data, compressing data, and executing analytical queries.
## Step 1. Deploy a test environment
Before using StoneDB, prepare your test environment according to instructions provided in [Quick Deployment](https://stoneatom.yuque.com/staff-ft8n1u/dghuxr/pv8ath) and start StoneDB.
Before using StoneDB, prepare your test environment according to instructions provided in [Quick Deployment](quick-deployment.md) and start StoneDB.
## Step 2. Prepare test data
Perform the following steps to generate test data.
......@@ -25,7 +25,7 @@ In the test environment, create a StoneDB table and a InnoDB table. Ensure the f
Create a database named **test**.
```
```sql
create database test DEFAULT CHARACTER SET utf8mb4;
```
......@@ -33,7 +33,7 @@ create database test DEFAULT CHARACTER SET utf8mb4;
In database **test**, create a table named **t_test**.
```
```sql
use test
CREATE TABLE t_user(
id INT NOT NULL AUTO_INCREMENT,
......@@ -50,7 +50,7 @@ CREATE TABLE t_user(
Create a stored procedure that is used to generate a table containing randomly generated names of persons.
```
```sql
DELIMITER //
create PROCEDURE add_user(in num INT)
BEGIN
......@@ -83,7 +83,8 @@ DELIMITER ;
## Step 3. Test insert performance
Call the stored procedure to insert 10,000,000 rows of data.
```
```sql
> call add_user_innodb(10000000);
Query OK, 1 row affected (24 min 46.62 sec)
......@@ -94,7 +95,8 @@ According to the returned result, StoneDB takes 9 minutes and 21 seconds, while
## Step 4. Test data compression efficiency
Compress the inserted data.
```
```sql
> select count(*) from t_user_innodb;
+----------+
| count(*) |
......@@ -123,7 +125,8 @@ According to the returned result, the data size after compression in StoneDB is
## Step 5. Test performance on processing aggregate queries
Execute an aggregate query.
```
```sql
> select first_name,count(*) from t_user group by first_name order by 1;
+------------+----------+
| first_name | count(*) |
......
{
"position": 4.2,
"label": "Monitoring and Alerting",
"collapsible": true
}
\ No newline at end of file
---
id: prometheus+grafana-monitor
sidebar_position: 4.21
---
# Use Prometheus and Grafana to Monitor MySQL or StoneDB Databases
\ No newline at end of file
{
"position": 4.3,
"label": "Backup and Recovery",
"collapsible": true
}
\ No newline at end of file
---
id: use-mydumper-full-backup
sidebar_position: 4.32
---
# Use Mydumper for Full Backup
## Mydumper introduction
Mydumper is a logical backup tool for MySQL. It consists of two parts:
- mydumper: exports consistent backup files of MySQL databases.
- myloader: reads backups from mydumper, connects to destination databases, and imports backups.
Both parts require multithreading capacities.
### Benefits
- Parallelism and performance: The tool provides high backup rate. Expensive character set conversion routines are avoided and the overall high efficiency of code is ensured.
- Simplified output management: Separate files are used for tables and metadata is dumped, simplifying data view and parse.
- High consistency: The tool maintains snapshots across all threads and provides accurate positions of primary and secondary logs.
- Manageability: Perl Compatible Regular Expressions (PCRE) can be used to specify whether to include or exclude tables or databases.
### Features
- Multi-threaded backup, which generates multiple backup files
- Consistent snapshots for transactional and non-transactional tables
:::info
This feature is supported by versions later than 0.2.2.
:::
- Fast file compression
- Export of binlogs
- Multi-threaded recovery
:::info
This feature is supported by versions later than 0.2.1.
:::
- Function as a daemon to periodically perform snapshots and consistently records binlogs
:::info
This feature is supported by versions later than 0.5.0.
:::
- Open source (license: GNU GPLv3)
## Use Mydumper
### Parameters for mydumer
```bash
mydumper --help
Usage:
mydumper [OPTION…] multi-threaded MySQL dumping
Help Options:
-?, --help Show help options
Application Options:
-B, --database Database to dump
-o, --outputdir Directory to output files to
-s, --statement-size Attempted size of INSERT statement in bytes, default 1000000
-r, --rows Try to split tables into chunks of this many rows. This option turns off --chunk-filesize
-F, --chunk-filesize Split tables into chunks of this output file size. This value is in MB
--max-rows Limit the number of rows per block after the table is estimated, default 1000000
-c, --compress Compress output files
-e, --build-empty-files Build dump files even if no data available from table
-i, --ignore-engines Comma delimited list of storage engines to ignore
-N, --insert-ignore Dump rows with INSERT IGNORE
-m, --no-schemas Do not dump table schemas with the data and triggers
-M, --table-checksums Dump table checksums with the data
-d, --no-data Do not dump table data
--order-by-primary Sort the data by Primary Key or Unique key if no primary key exists
-G, --triggers Dump triggers. By default, it do not dump triggers
-E, --events Dump events. By default, it do not dump events
-R, --routines Dump stored procedures and functions. By default, it do not dump stored procedures nor functions
-W, --no-views Do not dump VIEWs
-k, --no-locks Do not execute the temporary shared read lock. WARNING: This will cause inconsistent backups
--no-backup-locks Do not use Percona backup locks
--less-locking Minimize locking time on InnoDB tables.
--long-query-retries Retry checking for long queries, default 0 (do not retry)
--long-query-retry-interval Time to wait before retrying the long query check in seconds, default 60
-l, --long-query-guard Set long query timer in seconds, default 60
-K, --kill-long-queries Kill long running queries (instead of aborting)
-D, --daemon Enable daemon mode
-X, --snapshot-count number of snapshots, default 2
-I, --snapshot-interval Interval between each dump snapshot (in minutes), requires --daemon, default 60
-L, --logfile Log file name to use, by default stdout is used
--tz-utc SET TIME_ZONE='+00:00' at top of dump to allow dumping of TIMESTAMP data when a server has data in different time zones or data is being moved between servers with different time zones, defaults to on use --skip-tz-utc to disable.
--skip-tz-utc
--use-savepoints Use savepoints to reduce metadata locking issues, needs SUPER privilege
--success-on-1146 Not increment error count and Warning instead of Critical in case of table doesn't exist
--lock-all-tables Use LOCK TABLE for all, instead of FTWRL
-U, --updated-since Use Update_time to dump only tables updated in the last U days
--trx-consistency-only Transactional consistency only
--complete-insert Use complete INSERT statements that include column names
--split-partitions Dump partitions into separate files. This options overrides the --rows option for partitioned tables.
--set-names Sets the names, use it at your own risk, default binary
-z, --tidb-snapshot Snapshot to use for TiDB
--load-data
--fields-terminated-by
--fields-enclosed-by
--fields-escaped-by Single character that is going to be used to escape characters in theLOAD DATA stament, default: '\'
--lines-starting-by Adds the string at the begining of each row. When --load-data is usedit is added to the LOAD DATA statement. Its affects INSERT INTO statementsalso when it is used.
--lines-terminated-by Adds the string at the end of each row. When --load-data is used it isadded to the LOAD DATA statement. Its affects INSERT INTO statementsalso when it is used.
--statement-terminated-by This might never be used, unless you know what are you doing
--sync-wait WSREP_SYNC_WAIT value to set at SESSION level
--where Dump only selected records.
--no-check-generated-fields Queries related to generated fields are not going to be executed.It will lead to restoration issues if you have generated columns
--disk-limits Set the limit to pause and resume if determines there is no enough disk space.Accepts values like: '<resume>:<pause>' in MB.For instance: 100:500 will pause when there is only 100MB free and willresume if 500MB are available
--csv Automatically enables --load-data and set variables to export in CSV format.
-t, --threads Number of threads to use, default 4
-C, --compress-protocol Use compression on the MySQL connection
-V, --version Show the program version and exit
-v, --verbose Verbosity of output, 0 = silent, 1 = errors, 2 = warnings, 3 = info, default 2
--defaults-file Use a specific defaults file
--stream It will stream over STDOUT once the files has been written
--no-delete It will not delete the files after stream has been completed
-O, --omit-from-file File containing a list of database.table entries to skip, one per line (skips before applying regex option)
-T, --tables-list Comma delimited table list to dump (does not exclude regex option)
-h, --host The host to connect to
-u, --user Username with the necessary privileges
-p, --password User password
-a, --ask-password Prompt For User password
-P, --port TCP/IP port to connect to
-S, --socket UNIX domain socket file to use for connection
-x, --regex Regular expression for 'db.table' matching
```
### Parameters for myloader
```bash
myloader --help
Usage:
myloader [OPTION…] multi-threaded MySQL loader
Help Options:
-?, --help Show help options
Application Options:
-d, --directory Directory of the dump to import
-q, --queries-per-transaction Number of queries per transaction, default 1000
-o, --overwrite-tables Drop tables if they already exist
-B, --database An alternative database to restore into
-s, --source-db Database to restore
-e, --enable-binlog Enable binary logging of the restore data
--innodb-optimize-keys Creates the table without the indexes and it adds them at the end
--set-names Sets the names, use it at your own risk, default binary
-L, --logfile Log file name to use, by default stdout is used
--purge-mode This specify the truncate mode which can be: NONE, DROP, TRUNCATE and DELETE
--disable-redo-log Disables the REDO_LOG and enables it after, doesn't check initial status
-r, --rows Split the INSERT statement into this many rows.
--max-threads-per-table Maximum number of threads per table to use, default 4
--skip-triggers Do not import triggers. By default, it imports triggers
--skip-post Do not import events, stored procedures and functions. By default, it imports events, stored procedures nor functions
--no-data Do not dump or import table data
--serialized-table-creation Table recreation will be executed in serie, one thread at a time
--resume Expect to find resume file in backup dir and will only process those files
-t, --threads Number of threads to use, default 4
-C, --compress-protocol Use compression on the MySQL connection
-V, --version Show the program version and exit
-v, --verbose Verbosity of output, 0 = silent, 1 = errors, 2 = warnings, 3 = info, default 2
--defaults-file Use a specific defaults file
--stream It will stream over STDOUT once the files has been written
--no-delete It will not delete the files after stream has been completed
-O, --omit-from-file File containing a list of database.table entries to skip, one per line (skips before applying regex option)
-T, --tables-list Comma delimited table list to dump (does not exclude regex option)
-h, --host The host to connect to
-u, --user Username with the necessary privileges
-p, --password User password
-a, --ask-password Prompt For User password
-P, --port TCP/IP port to connect to
-S, --socket UNIX domain socket file to use for connection
-x, --regex Regular expression for 'db.table' matching
--skip-definer Removes DEFINER from the CREATE statement. By default, statements are not modified
```
### Install and use Mydumper
```bash
# On GitHub, download the RPM package or source code package that corresponds to the machine that you use. We recommend you download the RPM package because it can be directly used while the source code package requires compilation. The OS used in the following example is CentOS 7. Therefore, download an el7 version.
[root@dev tmp]# wget https://github.com/mydumper/mydumper/releases/download/v0.12.1/mydumper-0.12.1-1-zstd.el7.x86_64.rpm
# Because the downloaded package is a ZSTD file, dependency 'libzstd' is required.
[root@dev tmp]# yum install libzstd.x86_64 -y
[root@dev tmp]#rpm -ivh mydumper-0.12.1-1-zstd.el7.x86_64.rpm
Preparing... ################################# [100%]
Updating / installing...
1:mydumper-0.12.1-1 ################################# [100%]
# Backup library
[root@dev home]# mydumper -u root -p ******** -P 3306 -h 127.0.0.1 -B zz -o /home/dumper/
# Recovery library
[root@dev home]# myloader -u root -p ******** -P 3306 -h 127.0.0.1 -S /stonedb/install/tmp/mysql.sock -B zz -d /home/dumper
```
#### Generated backup files
```bash
[root@dev home]# ll dumper/
total 112
-rw-r--r--. 1 root root 139 Mar 23 14:24 metadata
-rw-r--r--. 1 root root 88 Mar 23 14:24 zz-schema-create.sql
-rw-r--r--. 1 root root 97819 Mar 23 14:24 zz.t_user.00000.sql
-rw-r--r--. 1 root root 4 Mar 23 14:24 zz.t_user-metadata
-rw-r--r--. 1 root root 477 Mar 23 14:24 zz.t_user-schema.sql
[root@dev dumper]# cat metadata
Started dump at: 2022-03-23 15:51:40
SHOW MASTER STATUS:
Log: mysql-bin.000002
Pos: 4737113
GTID:
Finished dump at: 2022-03-23 15:51:40
[root@dev-myos dumper]# cat zz-schema-create.sql
CREATE DATABASE /*!32312 IF NOT EXISTS*/ `zz` /*!40100 DEFAULT CHARACTER SET utf8 */;
[root@dev dumper]# more zz.t_user.00000.sql
/*!40101 SET NAMES binary*/;
/*!40014 SET FOREIGN_KEY_CHECKS=0*/;
/*!40103 SET TIME_ZONE='+00:00' */;
INSERT INTO `t_user` VALUES(1,"e1195afd-aa7d-11ec-936e-00155d840103","kAMXjvtFJym1S7PAlMJ7",102,62,"2022-03-23 15:50:16")
,(2,"e11a7719-aa7d-11ec-936e-00155d840103","0ufCd3sXffjFdVPbjOWa",698,44,"2022-03-23 15:50:16")
.....# The content is not full displayed since it is too long.
[root@dev dumper]# cat zz.t_user-metadata
10000
[root@dev-myos dumper]# cat zz.t_user-schema.sql
/*!40101 SET NAMES binary*/;
/*!40014 SET FOREIGN_KEY_CHECKS=0*/;
/*!40103 SET TIME_ZONE='+00:00' */;
CREATE TABLE `t_user` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`c_user_id` varchar(36) NOT NULL DEFAULT '',
`c_name` varchar(22) NOT NULL DEFAULT '',
`c_province_id` int(11) NOT NULL,
`c_city_id` int(11) NOT NULL,
`create_time` datetime NOT NULL,
PRIMARY KEY (`id`),
KEY `idx_user_id` (`c_user_id`)
) ENGINE=InnoDB AUTO_INCREMENT=10001 DEFAULT CHARSET=utf8;
```
The directory contains the following files:
**metadata**: records the name and position of the binlog file of the backup database at the backup point in time.
:::info
If the backup is performed on the standby library, this file also records the name and position of the binlog file that has been synchronized from the active libary when the backup is performed.
:::
Each table has two backup files:
- **database-schema-create**: records the statements for creating the library.
- **database.table-schema.sql**: records the table schemas.
- **database.table.00000.sql**: records table data.
- **database.table-metadata**: records table metadata.
### Backup principles
1. The main thread executes **FLUSH TABLES WITH READ LOCK** to add a global read-only lock to ensure data consistency.
2. The name and position of the binlog file at the current point in time are obtained and recorded to the **metadata **file to support recovery performed later.
3. Multiple (4 by default, customizable) dump threads change the isolation level for transactions to Repeatable Read and enable read-consistent transactions.
4. Non-InnoDB tables are exported.
5. After data of the non-transaction engine is backed up, the main thread executes **UNLOCK TABLES** to release the global read-only lock.
6. InnoDB tables are exported.
......@@ -22,10 +22,10 @@ CREATE TABLE t_name(
) engine=STONEDB;
```
2. Execute a `CREATE TABLE... LIKE `statement to create another table that uses the same schema as the table created in the step 1.
2. Execute a `CREATE TABLE... LIKE `statement to create another table that uses the same schema as the table created in the step 1.
Code example:
```
```sql
create table t_other like t_name;
```
......@@ -33,7 +33,7 @@ create table t_other like t_name;
Execute a `TRUNCATE TABLE` statement to clear data in a table and retain the table schema.
Code example:
```
```sql
truncate table t_name;
```
......@@ -41,19 +41,19 @@ truncate table t_name;
Execute an `ALTER TABLE... ADD COLUMN` statement to add a field in a given table. The added field is the last field, by default.
Code example:
```
```sql
alter table t_name add column c_name varchar(10);
```
### Drop a field
Execute an `ALTER TABLE... DROP` statement to drop a field from a table.<br />Code example:
```
```sql
alter table t_name drop c_name;
```
### Rename a table
Execute an `ALTER TABLE... RENAME TO` statement to rename a given table.<br />Code example:
```
```sql
alter table t_name rename to t_name_new;
```
......@@ -61,21 +61,23 @@ alter table t_name rename to t_name_new;
### Create a user
Code example:
```
```sql
create user 'u_name'@'hostname' identified by 'xxx';
```
### Grant user permissions
Code example:
```
```sql
grant all on *.* to 'u_name'@'hostname';
grant select on db_name.* to 'u_name'@'hostname';
grant select(column_name) on db_name.t_name to 'u_name'@'hostname';
```
### Revoke permissions
Code example:
```
```sql
revoke all privileges on *.* from 'u_name'@'hostname';
revoke select on db_name.* from 'u_name'@'hostname';
revoke select(column_name) on db_name.t_name from 'u_name'@'hostname';
......@@ -83,6 +85,7 @@ revoke select(column_name) on db_name.t_name from 'u_name'@'hostname';
### Drop a user
Code example:
```
```sql
drop user 'u_name'@'hostname';
```
{
"position": 5.1,
"label": "Compiling Methods",
"collapsible": true,
"link": {
"type": "generated-index",
"slug": "/compiling-methods"
}
}
\ No newline at end of file
---
id: compile-overview
sidebar_position: 5.11
---
# Overview
The method to compile StoneDB varies with the environment. Choose the compiling method that suits your environment.
- [Compile StoneDB on CentOS 7](compile-using-centos7.md)
- [Compile StoneDB on RHEL 7](compile-using-redhat7.md)
- [Compile StoneDB on Ubuntu 20.04](compile-using-ubuntu2004.md)
- [Compile StoneDB Using a Docker Container](compile-using-docker.md)
---
id: compile-using-centos7
sidebar_position: 5.12
---
# Compile StoneDB on CentOS 7
This topic describes how to compile StoneDB on CentOS 7.
## Prerequisites
The source code of StoneDB has been downloaded.
Download link: [https://github.com/stoneatom/stonedb.git](https://github.com/stoneatom/stonedb.git)
## Procedure
### Step 1. Install the dependencies
Before installing the dependencies, ensure that the GCC version in use is 4.8.5.
```bash
yum install -y gcc
yum install -y g++
yum install -y make
yum install -y build-essential
yum install -y autoconf
yum install -y tree
yum install -y bison
yum install -y git
yum install -y cmake
yum install -y libtool
yum install -y numactl
yum install -y python
yum install -y openssl
yum install -y perl
yum install -y binutils
yum install -y libgmp-dev
yum install -y libmpfr-dev
yum install -y libmpc-dev
yum install -y libisl-dev
yum install -y zlib1g-dev
yum install -y liblz4-dev
yum install -y libbz2-dev
yum install -y libzstd-dev
yum install -y lz4
yum install -y ncurses-dev
yum install -y libsnappy-dev
```
### Step 2. Install CMake and third-party libraries
Before compiling StoneDB, install CMake 3.7 or later and the following third-party libraries: marisa, RocksDB, and Boost.
1. Install CMake.
```bash
wget https://cmake.org/files/v3.7/cmake-3.7.2.tar.gz
tar -zxvf cmake-3.7.2.tar.gz
cd cmake-3.7.2
./bootstrap && make && make install
/usr/local/bin/cmake --version
apt remove cmake -y
ln -s /usr/local/bin/cmake /usr/bin/
```
2. Install marisa.
```bash
git clone https://github.com/s-yata/marisa-trie.git
cd marisa-trie
autoreconf -i
./configure --enable-native-code --prefix=/usr/local/stonedb-marisa
make && make install
```
3. Install RocksDB.
```bash
wget https://github.com/facebook/rocksdb/archive/refs/tags/v4.13.tar.gz
tar -zxvf v4.13.tar.gz
cd rocksdb-4.13
make shared_lib
make install-shared INSTALL_PATH=/usr/local/stonedb-gcc-rocksdb
make static_lib
make install-static INSTALL_PATH=/usr/local/stonedb-gcc-rocksdb
```
:::info
Boost is automatically installed when the **stonedb_build.sh** script is executed in **Step 4**.
:::
### Step 3. Install GCC 7.3.0
Before executing **stonedb_build.sh** to compile StoneDB, you must ensure the GCC version is 7.3.0.
You can run the following command to check the GCC version.
```bash
gcc --version
```
If the version is earlier than 7.3.0, perform the following steps to upgrade GCC.
1. Install the SCL utility.
```bash
yum install centos-release-scl scl-utils-build -y
```
2. Install GCC, GCC-C++, or GDB of v7.3.0.
```bash
yum install devtoolset-7-gcc.x86_64 devtoolset-7-gcc-c++.x86_64 devtoolset-7-gcc-gdb-plugin.x86_64 -y
```
3. Switch the version to 7.3.0.
```bash
scl enable devtoolset-7 bash
```
4. Check that the version is switched to 7.3.0.
```bash
gcc --version
```
### Step 4. Compile StoneDB
Execute the following script to compile StoneDB:
```bash
cd /stonedb2022/scripts
./stonedb_build.sh
```
After the compilation is complete, a folder named **/stonedb56** is generated.
### Step 5. Start StoneDB
Perform the following steps to start StoneDB.
1. Create a group, a user, and relevant directories.
```sql
groupadd mysql
useradd -g mysql mysql
mkdir -p /stonedb56/install/{log/,tmp/,binlog/,data/innodb} && chown -R mysql:mysql /stonedb56
```
2. Start StoneDB.
```sql
cd /stonedb56/install/bin
./mysqld_safe --defaults-file=/stonedb56/install/stonedb.cnf --user=mysql &
```
3. Log in to StoneDB.
```sql
#./mysql -uroot -p -S /stonedb56/install/tmp/mysql.sock
Warning: Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 1
Server version: 5.6.24-StoneDB-log build-
Copyright (c) 2000, 2022 StoneAtom Group Holding Limited
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| cache |
| innodb |
| test |
+--------------------+
4 rows in set (0.08 sec)
```
---
id: compile-using-docker
sidebar_position: 5.15
---
# Compile StoneDB in a Docker Container
## Introduction
Compiling StoneDB on a physical server requires installation of third-party repositories, which is complicated. In addition, if the OS in your environment is Fedora or Ubuntu, you also need to install many dependencies. We recommend that you compile StoneDB in a Docker container. After StoneDB is compiled, you can directly run StoneDB in the container or copy the compilation files to your environment.
## Prerequisites
Docker has been installed. For information about how to install Docker, visit [https://docs.docker.com/engine/install/ubuntu/](https://docs.docker.com/engine/install/ubuntu/).
## Use a Dockerfile in a compilation environment
### Step 1. Download the source code of StoneDB and docker.zip
Download file **docker.zip**, save the file to the root directory of the source code of StoneDB, and then decompress the file.
*docker.zip(waiting)*
```bash
[root@testOS src]# cd /home/src/
[root@testOS src]# git clone https://github.com/stoneatom/stonedb.git
Cloning into 'stonedb'...
remote: Enumerating objects: 84350, done.
remote: Counting objects: 100% (84350/84350), done.
remote: Total 84350 (delta 19707), reused 83550 (delta 19707), pack-reused 0
Receiving objects: 100% (84350/84350), 402.19 MiB | 13.50 MiB/s, done.
Resolving deltas: 100% (19707/19707), done.
[root@testOS src]# cd atomstore2022/
#Use an FTP tool to upload 'docker.zip' to this directory for decompression.
[root@testOS atomstore2022]# unzip docker.zip
[root@testOS atomstore2022]# tree docker
docker
├── cmake.tar.gz
├── docker_build.sh
├── Dockerfile
├── stonedb-boost1.66.tar.gz
├── stonedb-gcc-rocksdb.tar.gz
└── stonedb-marisa.tar.gz
0 directories, 6 files
```
### Step 2. Build a Docker image
```bash
[root@testOS atomstore2022]# cd docke
[root@testOS docker]# chmod u+x docker_build.sh
# If an image has been created in your environment, you can use the cache. If this is the first image that is to be created in your environment, you must install dependencies. This may take a longer period of time.
# Run the './docker_build.sh <tag>' command to call the script. <tag> specifies the tag of the image.
# Example './docker_build.sh 0.1'
[root@testOS docker]# ./docker_build.sh v0.1
/home/src
Sending build context to Docker daemon 99.41MB
Step 1/14 : FROM centos:7
---> eeb6ee3f44bd
Step 2/14 : ENV container docker
---> Using cache
---> dc33c0e29f61
Step 3/14 : RUN (cd /lib/systemd/system/sysinit.target.wants/; for i in *; do [ $i == systemd-tmpfiles-setup.service ] || rm -f $i; done); rm -f /lib/systemd/system/multi-user.target.wants/*;rm -f /etc/systemd/system/*.wants/*;rm -f /lib/systemd/system/local-fs.target.wants/*; rm -f /lib/systemd/system/sockets.target.wants/*udev*; rm -f /lib/systemd/system/sockets.target.wants/*initctl*; rm -f /lib/systemd/system/basic.target.wants/*;rm -f /lib/systemd/system/anaconda.target.wants/*;
---> Using cache
---> 12ca4ee4c8b0
Step 4/14 : RUN yum install -y epel-release.noarch
---> Using cache
---> 8b9a0d9cb423
Step 5/14 : RUN yum install -y snappy-devel lz4-devel bzip2-devel libzstd-devel.x86_64 ncurses-devel make bison libaio ncurses-devel perl perl-DBI perl-DBD-MySQL perl-Time-HiRes readline-devel numactl zlib-devel curldevel openssl-devel redhat-lsb-core.x86_64 git
---> Using cache
---> c7cf717b95c4
Step 6/14 : RUN yum install -y centos-release-scl && yum install devtoolset-7-gcc* -y
---> Using cache
---> e512aca12c7e
Step 7/14 : RUN ln -s /opt/rh/devtoolset-7/root/bin/gcc /usr/bin/gcc && ln -s /opt/rh/devtoolset-7/root/bin/g++ /usr/bin/g++ && ln -s /opt/rh/devtoolset-7/root/bin/c++ /usr/bin/c++
---> Using cache
---> 39cb9ada4812
Step 8/14 : RUN yum clean all
---> Using cache
---> 1370d1dc1a8e
Step 9/14 : ADD cmake.tar.gz /usr/local/
---> Using cache
---> f93823785ade
Step 10/14 : RUN ln -s /usr/local/cmake/bin/cmake /usr/bin/
---> Using cache
---> f5f9d2b3c35b
Step 11/14 : ADD stonedb-marisa.tar.gz /usr/local/
---> Using cache
---> e787d2341307
Step 12/14 : ADD stonedb-boost1.66.tar.gz /usr/local/
---> Using cache
---> 5d115e2ddb34
Step 13/14 : ADD stonedb-gcc-rocksdb.tar.gz /usr/local/
---> Using cache
---> a338f6756d71
Step 14/14 : CMD ["/usr/sbin/init"]
---> Using cache
---> 38381cd2bf3d
Successfully built 38381cd2bf3d
Successfully tagged stonedb_buildenv:v0.1
Docker build success!you can run it:
docker run -d -p 23306:3306 -v /home/src:/home/ stonedb_buildenv:v0.1
```
### Step 3. Enter the container and compile StoneDB
```bash
# docker run parameter description
# -v Directory mounting. Specify the directory on the host first and then the directory in the container.
# -p Port mapping. Specify the port on the host first and then the port in the container.
# After configuring the port mapping, you can directly start StoneDB in the container. If you do not need the trial, you can skip this parameter.
# docker run You can refer to the commands used in 'Step 2. Build a Docker image'.
[root@testOS docker]# docker run -d -p 23306:3306 -v /home/src:/home/ stonedb_buildenv:v0.1
06f1f385d3b35c86c4ed324731a13785b2a66f8ef2c3423c9b4711c56de1910f
[root@testOS docker]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
06f1f385d3b3 stonedb_buildenv:v0.1 "/usr/sbin/init" 18 seconds ago Up 17 seconds 0.0.0.0:23306->3306/tcp confident_tesla
# Enter the Docker container and compile StoneDB.
[root@testOS docker]# docker exec -it 06f1f385d3b3 bash
[root@06f1f385d3b3 home]# cd /home/atomstore2022/
[root@06f1f385d3b3 atomstore2022]# git branch -a
* 0.4
remotes/origin/0.4
remotes/origin/0.5
remotes/origin/HEAD -> origin/0.4
[root@06f1f385d3b3 atomstore2022]# git checkout 0.5
Branch 0.5 set up to track remote branch 0.5 from origin.
Switched to a new branch '0.5'
[root@06f1f385d3b3 atomstore2022]# git branch -a
0.4
* 0.5
remotes/origin/0.4
remotes/origin/0.5
remotes/origin/HEAD -> origin/0.4
[root@06f1f385d3b3 atomstore2022]# mkdir build
[root@06f1f385d3b3 atomstore2022]# cd build/
[root@06f1f385d3b3 build]# cmake ../ \
-DCMAKE_BUILD_TYPE=RelWithDebInfo \
-DCMAKE_INSTALL_PREFIX=/stonedb56/install \
-DMYSQL_DATADIR=/stonedb56/install/data \
-DSYSCONFDIR=/stonedb56/install \
-DMYSQL_UNIX_ADDR=/stonedb56/install/tmp/mysql.sock \
-DWITH_EMBEDDED_SERVER=OFF \
-DWITH_STONEDB_STORAGE_ENGINE=1 \
-DWITH_MYISAM_STORAGE_ENGINE=1 \
-DWITH_INNOBASE_STORAGE_ENGINE=1 \
-DWITH_PARTITION_STORAGE_ENGINE=1 \
-DMYSQL_TCP_PORT=3306 \
-DENABLED_LOCAL_INFILE=1 \
-DEXTRA_CHARSETS=all \
-DDEFAULT_CHARSET=utf8 \
-DDEFAULT_COLLATION=utf8_general_ci \
-DDOWNLOAD_BOOST=0 \
-DWITH_BOOST=/usr/local/stonedb-boost/include/
#After the 'cmake' command is completed, run the 'make' and 'make install' commands.
[root@06f1f385d3b3 build]# make
[root@06f1f385d3b3 build]# make install
```
## (Optional) Follow-up operations
After the `make` commands are successful, you can choose either to compress the compilation files to a TAR file and copy the TAR file from the container or to directly run it in the container.
### Compress compilation files to a TAR file
```bash
# Compress the 'home' folder to a TAR file and mount the TAR file to a directory outside the container.
[root@06f1f385d3b3 build]# tar -zcPvf /home/stonedb56.tar.gz /stonedb56/
```
### Directly use StoneDB in the container
You can refer to [Quick Deployment](https://stoneatom.yuque.com/staff-ft8n1u/dghuxr/pv8ath) or the following code to deploy and use StoneDB in the container.
```bash
[root@06f1f385d3b3 build]# cd /stonedb56/install/
[root@06f1f385d3b3 install]# groupadd mysql
[root@06f1f385d3b3 install]# useradd -g mysql mysql
[root@06f1f385d3b3 install]# ll
total 180
-rw-r--r--. 1 root root 17987 Jun 8 03:41 COPYING
-rw-r--r--. 1 root root 102986 Jun 8 03:41 INSTALL-BINARY
-rw-r--r--. 1 root root 2615 Jun 8 03:41 README
drwxr-xr-x. 2 root root 4096 Jun 8 06:16 bin
drwxr-xr-x. 3 root root 18 Jun 8 06:16 data
drwxr-xr-x. 2 root root 55 Jun 8 06:16 docs
drwxr-xr-x. 3 root root 4096 Jun 8 06:16 include
-rwxr-xr-x. 1 root root 267 Jun 8 03:41 install.sh
drwxr-xr-x. 3 root root 272 Jun 8 06:16 lib
drwxr-xr-x. 4 root root 30 Jun 8 06:16 man
drwxr-xr-x. 10 root root 4096 Jun 8 06:16 mysql-test
-rwxr-xr-x. 1 root root 12516 Jun 8 03:41 mysql_server
-rwxr-xr-x. 1 root root 764 Jun 8 03:41 reinstall.sh
drwxr-xr-x. 2 root root 57 Jun 8 06:16 scripts
drwxr-xr-x. 28 root root 4096 Jun 8 06:16 share
drwxr-xr-x. 4 root root 4096 Jun 8 06:16 sql-bench
-rw-r--r--. 1 root root 5526 Jun 8 03:41 stonedb.cnf
drwxr-xr-x. 2 root root 136 Jun 8 06:16 support-files
[root@06f1f385d3b3 install]# ./reinstall.sh
# If the following information is returned, StoneDB is started.
+ log_success_msg
+ /etc/redhat-lsb/lsb_log_message success
/etc/redhat-lsb/lsb_log_message: line 3: /etc/init.d/functions: No such file or directory
/etc/redhat-lsb/lsb_log_message: line 11: success: command not found
+ return 0
+ return_value=0
+ test -w /var/lock/subsys
+ touch /var/lock/subsys/mysql
+ exit 0
# Reset the password of local user 'root'.
[root@06f1f385d3b3 install]# /stonedb56/install/bin/mysqladmin flush-privileges -u root password "*******"
Warning: Using a password on the command line interface can be insecure.
# Create a username and password for remote connection.
[root@06f1f385d3b3 install]# /stonedb56/install/bin/mysql -uroot -p*******
Warning: Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 5
Server version: 5.6.24-StoneDB-log build-
Copyright (c) 2000, 2022 StoneAtom Group Holding Limited
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> grant all ON *.* to root@'%' identified by '********';
Query OK, 0 rows affected (0.00 sec)
mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)
```
After you start StoneDB in the container, you can log in to and use StoneDB or run the `docker run -p <port mapping>` command to connect to StoneDB.
---
id: compile-using-redhat7
sidebar_position: 5.13
---
# Compile StoneDB on RHEL 7
This topic describes how to compile StoneDB on Red Hat Enterprise Linux (RHEL) 7.
## Prerequisites
The source code of StoneDB has been downloaded.
Download link: [https://github.com/stoneatom/stonedb.git](https://github.com/stoneatom/stonedb.git)
## Procedure
### Step 1. Install the dependencies
Before installing the dependencies, ensure that the GCC version in use is 4.8.5.
```bash
yum install -y gcc
yum install -y g++
yum install -y make
yum install -y build-essential
yum install -y autoconf
yum install -y tree
yum install -y bison
yum install -y git
yum install -y cmake
yum install -y libtool
yum install -y numactl
yum install -y python
yum install -y openssl
yum install -y perl
yum install -y binutils
yum install -y libgmp-dev
yum install -y libmpfr-dev
yum install -y libmpc-dev
yum install -y libisl-dev
yum install -y zlib1g-dev
yum install -y liblz4-dev
yum install -y libbz2-dev
yum install -y libzstd-dev
yum install -y lz4
yum install -y ncurses-dev
yum install -y libsnappy-dev
```
### Step 2. Install CMake and third-party libraries
Before compiling StoneDB, install CMake 3.7 or later and the following third-party libraries: marisa, RocksDB, and Boost.
1. Install CMake.
```bash
wget https://cmake.org/files/v3.7/cmake-3.7.2.tar.gz
tar -zxvf cmake-3.7.2.tar.gz
cd cmake-3.7.2
./bootstrap && make && make install
/usr/local/bin/cmake --version
apt remove cmake -y
ln -s /usr/local/bin/cmake /usr/bin/
```
2. Install marisa.
```bash
git clone https://github.com/s-yata/marisa-trie.git
cd marisa-trie
autoreconf -i
./configure --enable-native-code --prefix=/usr/local/stonedb-marisa
make && make install
```
3. Install RocksDB.
```bash
wget https://github.com/facebook/rocksdb/archive/refs/tags/v4.13.tar.gz
tar -zxvf v4.13.tar.gz
cd rocksdb-4.13
make shared_lib
make install-shared INSTALL_PATH=/usr/local/stonedb-gcc-rocksdb
make static_lib
make install-static INSTALL_PATH=/usr/local/stonedb-gcc-rocksdb
```
:::info
Boost is automatically installed when the **stonedb_build.sh** script is executed in step 4.
:::
### Step 3. Install GCC 7.3.0
Before executing **stonedb_build.sh** to compile StoneDB, you must ensure the GCC version is 7.3.0.
You can run the following command to check the GCC version.
```bash
gcc --version
```
If the version is earlier than 7.3.0, perform the following steps to upgrade GCC.
1. Install the SCL utility.
```bash
yum install centos-release-scl scl-utils-build -y
```
2. Install GCC, GCC-C++, or GDB of v7.3.0.
```bash
yum install devtoolset-7-gcc.x86_64 devtoolset-7-gcc-c++.x86_64 devtoolset-7-gcc-gdb-plugin.x86_64 -y
```
3. Switch the version to 7.3.0.
```bash
scl enable devtoolset-7 bash
```
4. Check that the version is switched to 7.3.0.
```bash
gcc --version
```
### Step 4. Compile StoneDB
Execute the following script to compile StoneDB:
```bash
cd /stonedb2022/scripts
./stonedb_build.sh
```
After the compilation is complete, a folder named **/stonedb56** is generated.
### **Step 5. Start StoneDB**
Perform the following steps to start StoneDB.
1. Create a group, a user, and relevant directories.
```bash
groupadd mysql
useradd -g mysql mysql
mkdir -p /stonedb56/install/{log/,tmp/,binlog/,data/innodb} && chown -R mysql:mysql /stonedb56
```
2. Start StoneDB.
```bash
cd /stonedb56/install/bin
./mysqld_safe --defaults-file=/stonedb56/install/stonedb.cnf --user=mysql &
```
3. Log in to StoneDB.
```bash
#./mysql -uroot -p -S /stonedb56/install/tmp/mysql.sock
Warning: Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 1
Server version: 5.6.24-StoneDB-log build-
Copyright (c) 2000, 2022 StoneAtom Group Holding Limited
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| cache |
| innodb |
| test |
+--------------------+
4 rows in set (0.08 sec)
```
---
id: compile-using-ubuntu20.04
sidebar_position: 5.14
---
# Compile StoneDB on Ubuntu 20.04
This topic describes how to compile StoneDB on Ubuntu 20.04.
## Step 1. Install GCC 7.3.0
Ubuntu 20.04 LTS uses GCC 9.4.0, by default. You must downgrade the GCC version to 7.3.0, because StoneDB can be complied only on GCC 7.3.0.
### 1. Install the dependencies
```shell
sudo apt install gcc
sudo apt install g++
sudo apt install make
sudo apt install build-essential
sudo apt install autoconf
sudo apt install tree
sudo apt install bison
sudo apt install git
sudo apt install cmake
sudo apt install libtool
sudo apt install numactl
sudo apt install python
sudo apt install openssl
sudo apt install perl
sudo apt install binutils
sudo apt install libgmp-dev
sudo apt install libmpfr-dev
sudo apt install libmpc-dev
sudo apt install libisl-dev
sudo apt install zlib1g-dev
sudo apt install liblz4-dev
sudo apt install libbz2-dev
sudo apt install libzstd-dev
sudo apt install lz4
sudo apt install ncurses-dev
sudo apt install libsnappy-dev
```
:::info
Ensure that all the dependencies are installed. Otherwise, a large number of errors will be reported.
:::
### 2. Decompress the source code package of GCC 7.3.0
[Download](http://ftp.gnu.org/gnu/gcc/), upload, and then decompress the source code package of GCC 7.3.0.
```shell
cd /home
tar -zxvf gcc-7.3.0.tar.gz
```
### 3. Prepare for compiling GCC
Modify the source code.
```shell
cd /home/gcc-7.3.0/libsanitizer/sanitizer_common
cp sanitizer_platform_limits_posix.cc sanitizer_platform_limits_posix.cc.bak
vim sanitizer_platform_limits_posix.cc
# 1. Comment out row 158.
//#include <sys/ustat.h>
# 2. Add the following code after row 250.
// Use pre-computed size of struct ustat to avoid <sys/ustat.h> which
// has been removed from glibc 2.28.
#if defined(__aarch64__) || defined(__s390x__) || defined (__mips64) \
|| defined(__powerpc64__) || defined(__arch64__) || defined(__sparcv9) \
|| defined(__x86_64__)
#define SIZEOF_STRUCT_USTAT 32
#elif defined(__arm__) || defined(__i386__) || defined(__mips__) \
|| defined(__powerpc__) || defined(__s390__)
#define SIZEOF_STRUCT_USTAT 20
#else
#error Unknown size of struct ustat
#endif
unsigned struct_ustat_sz = SIZEOF_STRUCT_USTAT;
```
*Here's a picture to add*
:::info
If GCC is compiled without the modification, an error will be reported, indicating that **sys/ustat.h** does not exist.
:::
### 4. Compile GCC
```
mkdir /gcc
cd /home/gcc-7.3.0
./contrib/download_prerequisites
./configure --prefix=/gcc --enable-bootstrap -enable-threads=posix --enable-checking=release --enable-languages=c,c++ --disable-multilib --disable-libsanitizer
sudo make && make install
```
### 5. Check the GCC version
```
# /gcc/bin/gcc --version
gcc (GCC) 7.3.0
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
```
### 6. Delete GCC and G++ versions that are later than 7.3.0
```
sudo rm /usr/bin/gcc
sudo ln -s /gcc/bin/gcc /usr/bin/gcc
sudo rm /usr/bin/g++
sudo ln -s /gcc/bin/g++ /usr/bin/g++
# gcc --version
gcc (GCC) 7.3.0
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
# g++ --version
g++ (GCC) 7.3.0
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
# c++ --version
c++ (GCC) 7.3.0
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
```
## Step 2. Compile StoneDB
### 1. Download the source code package of StoneDB
Download the source code from [https://github.com/stoneatom/stonedb.git](https://github.com/stoneatom/stonedb.git).
### 2. Install CMake and the third-party libraries
1. Install CMake.
```
wget https://cmake.org/files/v3.7/cmake-3.7.2.tar.gz
tar -zxvf cmake-3.7.2.tar.gz
cd cmake-3.7.2
./bootstrap && make && make install
/usr/local/bin/cmake --version
apt remove cmake -y
ln -s /usr/local/bin/cmake /usr/bin/
```
2. Install marisa.
```
git clone https://github.com/s-yata/marisa-trie.git
cd marisa-trie
autoreconf -i
./configure --enable-native-code --prefix=/usr/local/stonedb-marisa
make && make install
```
The directories and files shown in the following figure are generated in directory **/usr/local/stonedb-marisa**.
*Here's a picture to add*
1. Install GCC 4.8.5 in an offline manner and configure it to be the default version.
```shell
sudo vim /etc/apt/sources.list
# Append the image sources to the paths.
deb http://dk.archive.ubuntu.com/ubuntu/ xenial main
deb http://dk.archive.ubuntu.com/ubuntu/ xenial universe
# Install GCC 4.8.5.
sudo apt update
sudo apt install gcc-4.8
sudo apt install gcc-4.8-locales
sudo apt install gcc-4.8-multilib
sudo apt install gcc-4.8-doc
sudo apt install g++-4.8
# Switch the GCC version to 4.8.5.
update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.8 20
update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-4.8 20
# Check the GCC version.
# gcc --version
gcc (Ubuntu 4.8.5-4ubuntu2) 4.8.5
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
# c++ --version
c++ (Ubuntu 4.8.5-4ubuntu2) 4.8.5
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
# g++ --version
g++ (Ubuntu 4.8.5-4ubuntu2) 4.8.5
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
```
4. Install RocksDB.
```shell
wget https://github.com/facebook/rocksdb/archive/refs/tags/v4.13.tar.gz
tar -zxvf v4.13.tar.gz
cd rocksdb-4.13
make shared_lib
make install-shared INSTALL_PATH=/usr/local/stonedb-gcc-rocksdb
make static_lib
make install-static INSTALL_PATH=/usr/local/stonedb-gcc-rocksdb
```
The directories and files shown in the following figure are generated in directory **/usr/local/stonedb-gcc-rocksdb**.
*Here's a picture to add*
1. Switch the GCC version back to 7.3.0. Otherwise, errors will be reported.
```shell
sudo rm /usr/bin/gcc
sudo ln -s /gcc/bin/gcc /usr/bin/gcc
sudo rm /usr/bin/g++
sudo ln -s /gcc/bin/g++ /usr/bin/g++
```
6. Install Boost.
Boost can be automatically installed when you execute the **stonedb_build.sh** script stored in directory** /stonedb2022/scripts**. The following code shows how to manually install Boost.
```shell
tar -zxvf boost_1_66_0.tar.gz
cd boost_1_66_0
./bootstrap.sh --prefix=/usr/local/stonedb-boost
./b2 install --with=all
```
The files and directories shown in the following figure are generated in directory **/usr/local/stonedb-boost**.
*Here's a picture to add*
### 3. Compile StoneDB
1. Modify script** stonedb_build.sh**.
```shell
vim /stonedb2022/scripts/stonedb_build.sh
cmake ../../ \
-DCMAKE_BUILD_TYPE=${build_type} \
-DCMAKE_INSTALL_PREFIX=${install_target} \
-DMYSQL_DATADIR=${install_target}/data \
-DSYSCONFDIR=${install_target} \
-DMYSQL_UNIX_ADDR=${install_target}/tmp/mysql.sock \
-DWITH_EMBEDDED_SERVER=OFF \
-DWITH_STONEDB_STORAGE_ENGINE=1 \
-DWITH_MYISAM_STORAGE_ENGINE=1 \
-DWITH_INNOBASE_STORAGE_ENGINE=1 \
-DWITH_PARTITION_STORAGE_ENGINE=1 \
-DMYSQL_TCP_PORT=3306 \
-DENABLED_LOCAL_INFILE=1 \
-DEXTRA_CHARSETS=all \
-DDEFAULT_CHARSET=utf8 \
-DDEFAULT_COLLATION=utf8_general_ci \
-DDOWNLOAD_BOOST=0 \
-DWITH_BOOST=/usr/local/stonedb-boost/include \
-DCMAKE_CXX_FLAGS='-D_GLIBCXX_USE_CXX11_ABI=0'
cd /stonedb2022/scripts/
./stonedb_build.sh
```
After the compilation is complete, directory **/stonedb56** is generated.
:::info
- Because Boost in this example is manually installed, the value of **-DWITH_BOOST** must be set to **/usr/local/stonedb-boost/include**.
- For compatibility purposes, **-DCMAKE_CXX_FLAGS='-D_GLIBCXX_USE_CXX11_ABI=0** must be included in the script. Otherwise, an error will be reported when the complication progress reaches 82%.
:::
## Step 3. Start StoneDB
Perform the following steps to start StoneDB.
### 1. Create a user group, a user, and directories
```sql
groupadd mysql
useradd -g mysql mysql
mkdir -p /stonedb56/install/{log/,tmp/,binlog/,data/innodb} && chown -R mysql:mysql /stonedb56
```
### 2. Start StoneDB
```sql
cd /stonedb56/install/bin
./mysqld_safe --defaults-file=/stonedb56/install/stonedb.cnf --user=mysql &
```
### 3. Log in to StoneDB
```shell
#./mysql -uroot -p -S /stonedb56/install/tmp/mysql.sock
Warning: Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 1
Server version: 5.6.24-StoneDB-log build-
Copyright (c) 2000, 2022 StoneAtom Group Holding Limited
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| cache |
| innodb |
| test |
+--------------------+
4 rows in set (0.08 sec)
```
{
"position": 5.2,
"label": "Connect to StoneDB",
"collapsible": true
}
\ No newline at end of file
---
id: use-mysql-client
sidebar_position: 5.21
---
# Use mysql to Connect to StoneDB
This topic describes how to use the MySQL command-line client named mysql to connect to StoneDB.
## Prerequisites
- mysql has been installed and its version is 5.5, 5.6, or 5.7.
- The value of environment variable **PATH** contains the directory that stores mysql.
## Procedure
1. Open mysql.
2. Specify required parameters according to the following format.
```shell
mysql -u -p -h -P -S -A
```
## Parameter description
The following table describes required parameters.
| **Parameter** | **Description** |
| --- | --- |
| -h | The IP address of StoneDB. |
| -u | The username of the account. |
| -p | The password of the account. You can skip this parameter for security purposes and enter the password as prompted later. |
| -P | The port used to connect to StoneDB. By default, the port 3306 is used. You can specify another port as needed. |
| -A | The name of the database. |
| -S | The name of the .sock file that is used to connect to StoneDB. |
:::tip
- If you want to exit the command-line client, enter** exit** and press **Enter.**
- For more information about parameters, run `mysql --help`.
:::
---
id: use-navicat
sidebar_position: 5.22
---
# Use Navicat to Connect to StoneDB
Navicat is a database management tool that allows you to connect to databases. You can use Navicat to connect to StoneDB and other relational databases such as Oracle, MySQL, and PostgreSQL. After you connect to StoneDB using Navicat, you can create, manage and maintain StoneDB on the Navicat graphical user interface (GUI).
This topic shows you how to use Navicat to connect to StoneDB.
## Prerequisites
Navicat has been installed.
## Procedure
1. Open Navicat and choose **File** > **New Connection** > **MySQL**.
*Here's a picture to add*
2. In the dialog box that appears, click the **General** tab, and enter the connection name, server IP address, port, username, and password. The following figure provides an example.
*Here's a picture to add*
3. Click **Test Connection**. If message "Connection successful" appears, the connection to StoneDB is established.
*Here's a picture to add*
:::info
You cannot use Navicat to connect to StoneDB as a super administrator ('root'@'localhost').
:::
{
"position": 5.5,
"label": "Create and Manage Database Objects",
"collapsible": true
}
\ No newline at end of file
---
id: create-and-manage-database
sidebar_position: 5.51
---
# Create and Manage a Database
Create a database. For example, execute the following SQL statement to create a database named **test_db** that uses **utf8mb4** as the default character set:
```sql
create database test_db DEFAULT CHARACTER SET utf8mb4;
```
List databases by executing the following SQL statement:
```sql
show databases;
```
Use a database. For example, execute the following SQL statement to use database **test_db**:
```sql
use test_db;
```
Drop a datable. For example, execute the following SQL statement to drop database **test_db**:
```sql
drop database test_db;
```
---
id: create-and-manage-stored-procedure
sidebar_position: 5.54
---
# Create and Manage a View
Create a stored procedure. For example, perform the following two steps to create a stored procedure named **add_user**, used to insert 1,000,000 random data records.
1. Execute the following SQL statement to create a table:
```sql
CREATE TABLE t_test(
id INT NOT NULL AUTO_INCREMENT,
first_name VARCHAR(10) NOT NULL,
last_name VARCHAR(10) NOT NULL,
sex VARCHAR(5) NOT NULL,
score INT NOT NULL,
copy_id INT NOT NULL,
PRIMARY KEY (`id`)
) engine=STONEDB;
```
2. Execute the following SQL statement to create the stored procedure:
```sql
DELIMITER //
create PROCEDURE add_user(in num INT)
BEGIN
DECLARE rowid INT DEFAULT 0;
DECLARE firstname VARCHAR(10);
DECLARE name1 VARCHAR(10);
DECLARE name2 VARCHAR(10);
DECLARE lastname VARCHAR(10) DEFAULT '';
DECLARE sex CHAR(1);
DECLARE score CHAR(2);
WHILE rowid < num DO
SET firstname = SUBSTRING(md5(rand()),1,4);
SET name1 = SUBSTRING(md5(rand()),1,4);
SET name2 = SUBSTRING(md5(rand()),1,4);
SET sex=FLOOR(0 + (RAND() * 2));
SET score= FLOOR(40 + (RAND() *60));
SET rowid = rowid + 1;
IF ROUND(RAND())=0 THEN
SET lastname =name1;
END IF;
IF ROUND(RAND())=1 THEN
SET lastname = CONCAT(name1,name2);
END IF;
insert INTO t_user(first_name,last_name,sex,score,copy_id) VALUES (firstname,lastname,sex,score,rowid);
END WHILE;
END //
DELIMITER ;
```
Call a stored procedure. For example, execute the following SQL statement to call stored procedure **add_user**:
```sql
call add_user(1000000);
```
Drop a stored procedure. For example, execute the following SQL statement to drop stored procedure **add_user**:
```sql
drop PROCEDURE add_user;
```
---
id: create-and-manage-table
sidebar_position: 5.52
---
# Create and Manage a Table
Create a table. For example, execute the following SQL statement to create a table which is named **student** and consists of the **id**, **name**, **age**, and **birthday** fields:
```sql
create table student(
id int(11) primary key,
name varchar(255),
age smallint,
birthday DATE
) engine=stonedb;
```
Query the schema of a table. For example, execute the following SQL statement to query the schema of table **student**:
```sql
show create table student\G
```
Drop a table. For example, execute the following SQL statement to drop table **student**:
```sql
drop table student;
```
\ No newline at end of file
---
id: create-and-manage-view
sidebar_position: 5.53
---
# Create and Manage a View
Create a view. For example, execute the following SQL statement to create a view named **v_s**, used to query teachers aged over 18:
```sql
create view v_s as select name from teachers where age>18;
```
Check the statement used for creating a view. For example, execute the following SQL statement to check the statement used for creating view **v_s**:
```sql
show create view v_s\G
```
Drop a view. For example, execute the following SQL statement to drop view **v_s**:
```sql
drop view v_s;
```
{
"position": 5.6,
"label": "Appendix",
"collapsible": true
}
\ No newline at end of file
---
id: configuration-parameters
sidebar_position: 5.62
---
# Configure Parameters
By default, all parameters of the StoneDB storage engine are saved in **/stonedb/install/stonedb.cnf**. Parameters of other storage engines can also be saved in file **stonedb.cnf**. If you want to modify parameter settings of the StoneDB storage engine, you must modify them in file **stonedb.cnf**, and then restart the StoneDB instance to make the modification take effect. This is because the StoneDB storage engine supports only static modification of parameter settings, which is different from other storage engines.
You can configure parameters based on your environment requirements. The following examples show how to configure parameters respectively in a dynamic and static manner.
# Example 1: Change the storage engine type
Parameter **default_storage_engine** specifies the storage engine type. You can dynamically set this parameter at the session level or the global level. However, If the database is restarted, the value of this parameter is restored to the default value. If you want to make a permanent modification, change the value of this parameter in file **stonedb.cnf** and restart the StoneDB instance.
Code example of changing the default storage engine:
```shell
# mysql -uroot -p -P3308
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 926
Server version: 5.7.36-StoneDB-log build-
Copyright (c) 2000, 2022 StoneAtom Group Holding Limited
No entry for terminal type "xterm";
using dumb terminal settings.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> show variables like 'default_storage_engine';
+------------------------+--------+
| Variable_name | Value |
+------------------------+--------+
| default_storage_engine | MyISAM |
+------------------------+--------+
1 row in set (0.00 sec)
mysql> set global default_storage_engine=StoneDB;
Query OK, 0 rows affected (0.00 sec)
mysql> exit
Bye
# mysql -uroot -p -P3308
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 927
Server version: 5.7.36-StoneDB-log build-
Copyright (c) 2000, 2022 StoneAtom Group Holding Limited
No entry for terminal type "xterm";
using dumb terminal settings.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> show variables like 'default_storage_engine';
+------------------------+---------+
| Variable_name | Value |
+------------------------+---------+
| default_storage_engine | STONEDB |
+------------------------+---------+
1 row in set (0.00 sec)
```
The default storage engine of the database is modified from **MyISAM** to **STONEDB** at a global level.
Code example of restarting the StoneDB instance:
```sql
mysql> shutdown;
Query OK, 0 rows affected (0.00 sec)
mysql> exit
Bye
# mysql -uroot -p -P3308
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.7.36-StoneDB-log build-
Copyright (c) 2000, 2022 StoneAtom Group Holding Limited
No entry for terminal type "xterm";
using dumb terminal settings.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> show variables like 'default_storage_engine';
+------------------------+--------+
| Variable_name | Value |
+------------------------+--------+
| default_storage_engine | MyISAM |
+------------------------+--------+
1 row in set (0.00 sec)
```
After the StoneDB instance is restarted, the value of **default_storage_engine** is restored to **MyISAM**. If you want to make your change persistently effective, edit file **stonedb.cnf** to modify the parameter setting and then restart the StoneDB instance.
# Example 2: Change the insert buffer size
The parameters of the StoneDB storage engine support only static modification. After the parameter settings are modified, restart the StoneDB instance to make the modification take effect.
Code example:
```sql
mysql> show variables like 'stonedb_insert_buffer_size';
+----------------------------+-------+
| Variable_name | Value |
+----------------------------+-------+
| stonedb_insert_buffer_size | 512 |
+----------------------------+-------+
1 row in set (0.00 sec)
# vi /stonedb/install/stonedb.cnf
stonedb_insert_buffer_size = 1024
mysql> shutdown;
Query OK, 0 rows affected (0.00 sec)
# /stonedb/install//bin/mysqld_safe --datadir=/stonedb/install/data/ --user=mysql &
# mysql -uroot -p -P3308
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.7.36-StoneDB-log build-
Copyright (c) 2000, 2022 StoneAtom Group Holding Limited
No entry for terminal type "xterm";
using dumb terminal settings.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> show variables like 'stonedb_insert_buffer_size';
+----------------------------+-------+
| Variable_name | Value |
+----------------------------+-------+
| stonedb_insert_buffer_size | 1024 |
+----------------------------+-------+
1 row in set (0.00 sec)
```
**stonedb_insert_buffer_size** is set to 1024 MB.
---
id: error-codes
sidebar_position: 5.63
---
# Error Codes
This topic describes the common error codes that may be returned in StoneDB.
| **Error code** | **Error message** | **Description** |
| --- | --- | --- |
| 2233 (HY000) | Be disgraceful to storage engine, operating is forbidden! | The error message is returned because the DDL operation is not supported. |
| 1031 (HY000) | Table storage engine for 'xxx' doesn't have this option | The error message is because the DML operation is not supported. |
| 1040 (HY000) | Too many connections | The error message is because the number of connections has reached the maximum number. |
| 1045 (28000) | Access denied for user 'u_test'@'%' (using password: YES) | The error message is because the username or password is incorrect or the permissions are insufficient. |
---
id: regular-change-operations
sidebar_position: 5.3
---
# DML Statements
This topic describes the DML statements supported by StoneDB.
In this topic, table **t_test** created by executing the following statement is used in all code examples.
```sql
CREATE TABLE t_test(
id INT NOT NULL AUTO_INCREMENT,
first_name VARCHAR(10) NOT NULL,
last_name VARCHAR(10) NOT NULL,
sex VARCHAR(5) NOT NULL,
score INT NOT NULL,
copy_id INT NOT NULL,
PRIMARY KEY (`id`)
) engine=STONEDB;
```
## INSERT
```sql
insert into t_test values(1,'jack','rose','0',58,1);
```
## UPDATE
```sql
update t_test set score=200 where id=1;
```
## INSERT INTO ... SELECT
```sql
create table t_test2 like t_test;
insert into t_test2 select * from t_test;
```
## INSERT INTO ... ON DUPLICATE KEY UPDATE
```sql
insert into t_test1 values(1,'Bond','Jason','1',47,10) on duplicate key update last_name='James';
```
:::info
The logic of this syntax is to insert a row of data. The UPDATE statement is executed only if a primary key constraint or unique constraint conflict occurs.
:::
{
"position": 5,
"label": "Developer Guide",
"collapsible": true
}
\ No newline at end of file
---
id: statements-for-queries
sidebar_position: 5.4
---
# Statements for Queries
## Statements for common queries
### UNION/UNION ALL
```sql
select first_name from t_test1
union all
select first_name from t_test2;
```
### DISTINCT
```sql
select distinct first_name from t_test1;
```
### LIKE
```sql
select * from t_test where first_name like 'zhou%';
```
### GROUP BY/ORDER BY
```sql
select first_name,count(*) from t_test1 group by first_name order by 2;
```
### HAVING
```sql
select e.id, count(e.id), round(avg(e.score), 2)
from t_test1 e
group by e.id
having avg(e.score) > (select avg(score) from t_test1);
```
## Statements used for aggregate queries
```sql
select first_name,count(*) from t_test group by first_name;
select sum(score) from t_test;
```
## Statements used for pagination queries
```sql
select * from t_test1 limit 10;
select * from t_test1 limit 10,10;
```
## Statements used for correlated queries
### INNER JOIN
```sql
select t1.id,t1.first_name,t2.last_name from t_test1 t1,t_test2 t2 where t1.id = t2.id;
```
### LEFT JOIN
```sql
select t1.id,t1.first_name,t2.last_name from t_test1 t1 left join t_test2 t2 on t1.id = t2.id and t1.id=100;
```
### RIGHT JOIN
```sql
select t1.id,t1.first_name,t2.last_name from t_test1 t1 right join t_test2 t2 on t1.id = t2.id and t1.id=100;
```
## Statements used for subqueries
### Statement for scalar subqueries
```sql
select e.id,
e.first_name,
(select d.first_name from t_test2 d where d.id = e.id) as first_name
from t_test1 e;
```
### Statement for derived subqueries
```sql
select a.first_name, b.last_name
from t_test1 a, (select id,last_name from t_test2) b
where a.id = b.id;
```
### IN/NOT IN
```sql
select * from t_test1 where id in(select id from t_test2);
```
### EXISTS/NOT EXISTS
```sql
select * from t_test1 A where exists (select 1 from t_test2 B where B.id = A.id);
```
{
"position": 6.3,
"label": "Operators",
"collapsible": true
}
\ No newline at end of file
---
id: arithmetic-operators
sidebar_position: 6.31
---
# Arithmetic Operators
This topic describes the arithmetic operators supported by StoneDB.
| **Operator** | **Description** |
| --- | --- |
| `+` | Addition operator |
| `-` | Minus operator |
| `*` | Multiplication operator |
| `/`, `div` | Division operator |
| `%`, `mod` | Modulo operator |
The following code provides an example of using each operator.
```sql
> select 10+2 from dual;
+------+
| 10+2 |
+------+
| 12 |
+------+
1 row in set (0.00 sec)
> select 10-2 from dual;
+------+
| 10-2 |
+------+
| 8 |
+------+
1 row in set (0.00 sec)
> select 10*2 from dual;
+------+
| 10*2 |
+------+
| 20 |
+------+
1 row in set (0.00 sec)
> select 10/2 from dual;
+--------+
| 10/2 |
+--------+
| 5.0000 |
+--------+
1 row in set (0.00 sec)
> select 10 div 2 from dual;
+----------+
| 10 div 2 |
+----------+
| 5 |
+----------+
1 row in set (0.00 sec)
> select 10 mod 3 from dual;
+----------+
| 10 mod 3 |
+----------+
| 1 |
+----------+
1 row in set (0.00 sec)
> select 10 % 3 from dual;
+--------+
| 10 % 3 |
+--------+
| 1 |
+--------+
1 row in set (0.00 sec)
> select 10 mod 0 from dual;
+----------+
| 10 mod 0 |
+----------+
| NULL |
+----------+
1 row in set (0.00 sec)
> select 10 / 0 from dual;
+--------+
| 10 / 0 |
+--------+
| NULL |
+--------+
1 row in set (0.00 sec)
```
:::tip
If the divisor is 0 in a division operation or a modulo operation, the operation is invalid and NULL is returned.
:::
---
id: assignment-operators
sidebar_position: 6.35
---
# Assignment Operators
In the SET clause of an UPDATE statement, the equal sign (`=`) functions as an assignment operator. In this case, the value on the right side of the operator is assigned to the variable on the right side, provided that any WHERE conditions specified in the UPDATE statement are met.
The following code provides an example.
```sql
mysql> select * from t;
+------+------+
| col1 | col2 |
+------+------+
| 100 | 100 |
+------+------+
1 row in set (0.00 sec)
mysql> update t set col1=col1+100,col2=col1+100;
Query OK, 1 row affected (0.00 sec)
Rows matched: 1 Changed: 1 Warnings: 0
mysql> select * from t;
+------+------+
| col1 | col2 |
+------+------+
| 200 | 300 |
+------+------+
1 row in set (0.00 sec)
```
\ No newline at end of file
---
id: bitwise-operators
sidebar_position: 6.32
---
# Bitwise Operators
This topic describes the bitwise operators supported by StoneDB.
| **Operator** | **Description** |
| --- | --- |
| `&` | Bitwise AND |
| &#124; | Bitwise OR |
| `^` | Bitwise XOR |
| `!` | Bitwise inversion |
| `<<` | Left shift |
| `>>` | Right shift |
Bitwise operators are used to operate on binary numbers. In a bitwise operation, the involved numbers are first converted to binary numbers to compute the result, and then the result is converted back to a decimal value.
The following code provides an example of using each operator.
```sql
> select 3&5;
+-----+
| 3&5 |
+-----+
| 1 |
+-----+
1 row in set (0.00 sec)
> select 3|5;
+-----+
| 3|5 |
+-----+
| 7 |
+-----+
1 row in set (0.00 sec)
> select 3^5;
+-----+
| 3^5 |
+-----+
| 6 |
+-----+
1 row in set (0.00 sec)
> select ~18446744073709551612;
+-----------------------+
| ~18446744073709551612 |
+-----------------------+
| 3 |
+-----------------------+
1 row in set (0.00 sec)
> select 3>>1;
+------+
| 3>>1 |
+------+
| 1 |
+------+
1 row in set (0.00 sec)
> select 3<<1;
+------+
| 3<<1 |
+------+
| 6 |
+------+
1 row in set (0.00 sec)
```
\ No newline at end of file
---
id: comparison-operators
sidebar_position: 6.33
---
# Comparison Operators
This topic describes the comparison operators supported StoneDB.
| **Operator** | **Description** |
| --- | --- |
| `=` | Equal operator |
| `>` | Greater than operator |
| `<` | Less than operator |
| `>=` | Greater than or equal operator |
| `<=` | Less than or equal operator |
| `!=`, `<>` | Not equal operator |
| `<=>` | NULL-safe equal operator |
| `BETWEEN… AND…` | Whether a value is within a value range |
| `IN` | Whether a value is within a set of values |
| `NOT IN` | Whether a value is not within a set of values |
| `LIKE` | Simple pattern matching |
| `regexp` | Regular expression |
| `IS NULL` | NULL value test |
| `IS NOT NULL` | NOT NULL value test |
The following code provides an example of using each operator.
```sql
> select 2=3;
+-----+
| 2=3 |
+-----+
| 0 |
+-----+
1 row in set (0.00 sec)
> select 2>3;
+-----+
| 2>3 |
+-----+
| 0 |
+-----+
1 row in set (0.00 sec)
> select 2<3;
+-----+
| 2<3 |
+-----+
| 1 |
+-----+
1 row in set (0.00 sec)
> select 2>=3;
+------+
| 2>=3 |
+------+
| 0 |
+------+
1 row in set (0.00 sec)
> select 2<=3;
+------+
| 2<=3 |
+------+
| 1 |
+------+
1 row in set (0.00 sec)
> select 2<>3;
+------+
| 2<>3 |
+------+
| 1 |
+------+
1 row in set (0.00 sec)
> select 2<=>3;
+-------+
| 2<=>3 |
+-------+
| 0 |
+-------+
1 row in set (0.01 sec)
> select 5 between 1 and 10;
+--------------------+
| 5 between 1 and 10 |
+--------------------+
| 1 |
+--------------------+
1 row in set (0.00 sec)
> select 5 in (1,2,3,4,5);
+------------------+
| 5 in (1,2,3,4,5) |
+------------------+
| 1 |
+------------------+
1 row in set (0.00 sec)
> select 5 not in (1,2,3,4,5);
+----------------------+
| 5 not in (1,2,3,4,5) |
+----------------------+
| 0 |
+----------------------+
1 row in set (0.00 sec)
> select '12345' like '12%';
+--------------------+
| '12345' like '12%' |
+--------------------+
| 1 |
+--------------------+
1 row in set (0.00 sec)
> select 'beijing' REGEXP 'jing';
+-------------------------+
| 'beijing' REGEXP 'jing' |
+-------------------------+
| 1 |
+-------------------------+
1 row in set (0.00 sec)
> select 'beijing' REGEXP 'xi';
+-----------------------+
| 'beijing' REGEXP 'xi' |
+-----------------------+
| 0 |
+-----------------------+
1 row in set (0.00 sec)
> select 'a' is NULL;
+-------------+
| 'a' is NULL |
+-------------+
| 0 |
+-------------+
1 row in set (0.00 sec)
> select 'a' IS NOT NULL;
+-----------------+
| 'a' IS NOT NULL |
+-----------------+
| 1 |
+-----------------+
1 row in set (0.00 sec)
```
\ No newline at end of file
---
id: logical-operators
sidebar_position: 6.34
---
# Logical Operators
This topic describes the logical operators supported by StoneDB.
| **Operator** | **Description** |
| --- | --- |
| NOT | Logical NOT |
| AND | Logical AND |
| OR | Logical OR |
| XOR | Logical XOR |
The following code provides an example of using each operator.
```sql
> select not 1;
+-------+
| not 1 |
+-------+
| 0 |
+-------+
1 row in set (0.00 sec)
> select !0;
+----+
| !0 |
+----+
| 1 |
+----+
1 row in set (0.00 sec)
> select 2 and 0;
+---------+
| 2 and 0 |
+---------+
| 0 |
+---------+
1 row in set (0.00 sec)
> select 2 and 1;
+---------+
| 2 and 1 |
+---------+
| 1 |
+---------+
1 row in set (0.00 sec)
> select 2 or 0;
+--------+
| 2 or 0 |
+--------+
| 1 |
+--------+
1 row in set (0.00 sec)
> select 2 or 1;
+--------+
| 2 or 1 |
+--------+
| 1 |
+--------+
1 row in set (0.00 sec)
> select 1 xor 1;
+---------+
| 1 xor 1 |
+---------+
| 0 |
+---------+
1 row in set (0.00 sec)
> select 0 xor 0;
+---------+
| 0 xor 0 |
+---------+
| 0 |
+---------+
1 row in set (0.00 sec)
> select 1 xor 0;
+---------+
| 1 xor 0 |
+---------+
| 1 |
+---------+
1 row in set (0.00 sec)
> select null or 1;
+-----------+
| null or 1 |
+-----------+
| 1 |
+-----------+
1 row in set (0.00 sec)
```
\ No newline at end of file
{
"position": 6.4,
"label": "Functions",
"collapsible": true
}
\ No newline at end of file
---
id: advanced-functions
sidebar_position: 6.44
---
# Advanced Functions
\ No newline at end of file
---
id: aggregate-functions
sidebar_position: 6.45
---
# Aggregate Functions
\ No newline at end of file
---
id: date-and-time-functions
sidebar_position: 6.41
---
# Date and Time Functions
\ No newline at end of file
---
id: mathematical-functions
sidebar_position: 6.43
---
# Mathematical Functions
\ No newline at end of file
---
id: string-functions
sidebar_position: 6.42
---
# String Functions
\ No newline at end of file
{
"position": 6,
"label": "SQL Reference",
"collapsible": true
}
\ No newline at end of file
---
id: character-sets
sidebar_position: 6.1
---
# Character Sets
A character set is a collection of symbols and encodings, where the encodings determine how character strings are stored. A collation is a collection of rules for comparing and sorting character strings. A character set is associated with a collation. If you change either of them, the other also changes.
## Supported character sets
You can execute the `SHOW CHARACTER SET` statement to view the character sets supported by StoneDB.
```sql
> show character set;
+----------+---------------------------------+---------------------+--------+
| Charset | Description | Default collation | Maxlen |
+----------+---------------------------------+---------------------+--------+
| big5 | Big5 Traditional Chinese | big5_chinese_ci | 2 |
| dec8 | DEC West European | dec8_swedish_ci | 1 |
| cp850 | DOS West European | cp850_general_ci | 1 |
| hp8 | HP West European | hp8_english_ci | 1 |
| koi8r | KOI8-R Relcom Russian | koi8r_general_ci | 1 |
| latin1 | cp1252 West European | latin1_swedish_ci | 1 |
| latin2 | ISO 8859-2 Central European | latin2_general_ci | 1 |
| swe7 | 7bit Swedish | swe7_swedish_ci | 1 |
| ascii | US ASCII | ascii_general_ci | 1 |
| ujis | EUC-JP Japanese | ujis_japanese_ci | 3 |
| sjis | Shift-JIS Japanese | sjis_japanese_ci | 2 |
| hebrew | ISO 8859-8 Hebrew | hebrew_general_ci | 1 |
| tis620 | TIS620 Thai | tis620_thai_ci | 1 |
| euckr | EUC-KR Korean | euckr_korean_ci | 2 |
| koi8u | KOI8-U Ukrainian | koi8u_general_ci | 1 |
| gb2312 | GB2312 Simplified Chinese | gb2312_chinese_ci | 2 |
| greek | ISO 8859-7 Greek | greek_general_ci | 1 |
| cp1250 | Windows Central European | cp1250_general_ci | 1 |
| gbk | GBK Simplified Chinese | gbk_chinese_ci | 2 |
| latin5 | ISO 8859-9 Turkish | latin5_turkish_ci | 1 |
| armscii8 | ARMSCII-8 Armenian | armscii8_general_ci | 1 |
| utf8 | UTF-8 Unicode | utf8_general_ci | 3 |
| ucs2 | UCS-2 Unicode | ucs2_general_ci | 2 |
| cp866 | DOS Russian | cp866_general_ci | 1 |
| keybcs2 | DOS Kamenicky Czech-Slovak | keybcs2_general_ci | 1 |
| macce | Mac Central European | macce_general_ci | 1 |
| macroman | Mac West European | macroman_general_ci | 1 |
| cp852 | DOS Central European | cp852_general_ci | 1 |
| latin7 | ISO 8859-13 Baltic | latin7_general_ci | 1 |
| utf8mb4 | UTF-8 Unicode | utf8mb4_general_ci | 4 |
| cp1251 | Windows Cyrillic | cp1251_general_ci | 1 |
| utf16 | UTF-16 Unicode | utf16_general_ci | 4 |
| utf16le | UTF-16LE Unicode | utf16le_general_ci | 4 |
| cp1256 | Windows Arabic | cp1256_general_ci | 1 |
| cp1257 | Windows Baltic | cp1257_general_ci | 1 |
| utf32 | UTF-32 Unicode | utf32_general_ci | 4 |
| binary | Binary pseudo charset | binary | 1 |
| geostd8 | GEOSTD8 Georgian | geostd8_general_ci | 1 |
| cp932 | SJIS for Windows Japanese | cp932_japanese_ci | 2 |
| eucjpms | UJIS for Windows Japanese | eucjpms_japanese_ci | 3 |
| gb18030 | China National Standard GB18030 | gb18030_chinese_ci | 4 |
+----------+---------------------------------+---------------------+--------+
```
## Variables relevant to character sets
You can execute the following statement to show the variables relevant to character sets.
```sql
> show variables like '%char%';
+--------------------------+----------------------------------+
| Variable_name | Value |
+--------------------------+----------------------------------+
| character_set_client | utf8 |
| character_set_connection | utf8 |
| character_set_database | utf8mb4 |
| character_set_filesystem | binary |
| character_set_results | utf8 |
| character_set_server | utf8mb4 |
| character_set_system | utf8 |
| character_sets_dir | /stonedb/install/share/charsets/ |
+--------------------------+----------------------------------+
8 rows in set (0.01 sec)
```
Variable description:
- **character_set_client** specifies the character set used by the server to receive requests from the client.
- **character_set_connection** specifies the character set converted by the server from the character set specified by **character_set_client**.
- **character_set_results** specifies the character set used by the server to respond to requests sent by the client.
The character set used by a client to send requests to a server and receive response from the server is the character set used by the client OS, which can be specified by the **LC_ALL**, **LC_CTYPE**, or **LANG **variable. Among the three variables, **LC_ALL** has the highest priority, **LC_CTYPE** the second, and then **LANG** the lowest.
If the client OS uses the UTF-8 character set and **default-character-set** is set to **gbk** during the client startup, **character_set_client**, **character_set_connection**, and **character_set_results** are automatically set to **gbk**. Now, the client requests a table that contains Chinese characters. The representation of each Chinese character in the server and that in the client are two different strings. In a UNIX OS, data received by the server will be converted based on the character set specified by **character_set_connection**. In this case, the data presented to the server is garbled text.
:::info
For StoneDB, if no character set is specified for the database when the database is being created, the database uses the character set specified by **character_set_server**, by default. If a table is not specified with a character set when the table is being created, the table uses the character set used by the database. Once a table is created, you cannot change or convert its character set.
:::
---
id: data-types
sidebar_position: 6.2
---
# Data Types
The following table lists the data types supported by StoneDB.
| **Category** | **Data type** |
| --- | --- |
| Integer | TINYINT, SMALLINT, MEDIUMINT, INT, BIGINT |
| Floating point | FLOAT, DOUBLE |
| Fixed point | DECIMAL |
| Date and time | YEAR, TIME, DATE, DATETIME, TIMESTAMP |
| String | CHAR, VARCHAR, TINYTEXT, TEXT, MEDIUMTEXT, LONGTEXT |
| Binary string | BINARY, VARBINARY, TINYBLOB, BLOB, MEDIUMBLOB, LONGBLOB |
:::caution
When you create a StoneDB table, do not include keyword **unsigned** or **zerofill**.
:::
## Integer data types
The following table provides the value range of each integer data type.
| **Data type** | **Bytes of storage** | **Min. value** | **Max. value** |
| --- | --- | --- | --- |
| TINYINT | 1 | -128 | 127 |
| SMALLINT | 2 | -32768 | 32767 |
| MEDIUMINT | 3 | -8388608 | 8388607 |
| INT | 4 | -2147483647 | 2147483647 |
| BIGINT | 8 | -9223372036854775808 | 9223372036854775807 |
On StoneDB, the precision for DECIMAL numbers cannot be higher than 18. For example, if you specify **decimal(19)** in your code, an error will be reported. **DECIMAL(6, 2)** indicates that up to 6 places are supported at the left of the decimal and up to 2 at the right, and thus the value range is [-9999.99, 9999.99].
## String data types
The storage required for a string varies according to the character set in use. The length range also differs. The following table describes the length range of each string data type when character set latin1 is in use.
| **Data type** | **Size** |
| --- | --- |
| CHAR(M) | [0,255] |
| VARCHAR(M) | [0,65535] |
| TINYTEXT | [0,255] |
| TEXT | [0,65535] |
| MEDIUMTEXT | [0,16777215] |
| LONGTEXT | [0,4294967295] |
## Date and time data types
The following table describes the value range of each date and time data type.
| **Data type** | **Format** | **Min. value** | **Max. value** |
| --- | --- | --- | --- |
| YEAR | YYYY | 1901 | 2155 |
| TIME | HH:MM:SS | -838:59:59 | 838:59:59 |
| DATE | YYYY-MM-DD | 0001-01-01 | 9999-12-31 |
| DATETIME | YYYY-MM-DD HH:MM:SS | 0001-01-01 00:00:00 | 9999-12-31 23:59:59 |
| TIMESTAMP | YYYY-MM-DD HH:MM:SS | 1970-01-01 08:00:01 | 2038-01-19 11:14:07 |
{
"position": 7.2,
"label": "Performance Monitoring Commands",
"collapsible": true
}
\ No newline at end of file
---
id: cpu-monitor
sidebar_position: 7.22
---
# Commands for CPU Monitoring
\ No newline at end of file
---
id: disk-io-monitor
sidebar_position: 7.24
---
# Commands for I/O Monitoring
This topic describes common commands used for I/O monitoring.
## iostat
The `iostat` command is used to monitor the performance of system I/O devices.
Command output example:
```shell
# iostat -x 1 1
Linux 3.10.0-957.el7.x86_64 (htap2) 06/13/2022 _x86_64_ (64 CPU)
avg-cpu: %user %nice %system %iowait %steal %idle
0.06 0.00 0.03 0.01 0.00 99.90
Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 0.00 0.00 0.00 0.04 0.00 85.75 0.00 0.25 0.25 0.00 0.15 0.00
sdb 0.06 0.11 7.61 1.10 1849.41 50.81 436.48 0.36 40.93 46.75 0.48 1.56 1.35
dm-0 0.00 0.00 0.28 0.19 8.25 12.05 87.01 0.00 4.81 7.37 0.94 1.61 0.08
```
Parameter description:
| **Parameter** | **Description** |
| --- | --- |
| rrqm/s | The number of read requests merged per second. |
| wrqm/s | The number of write requests merged per second. |
| r/s | The number (after merges) of read requests completed per second. |
| w/s | The number (after merges) of write requests completed per second. |
| rkB/s | The number of kilobytes read per second. |
| wkB/s | The number of kilobytes written per second. |
| avgrq-sz | The average size of the requests, expressed in sectors (512 bytes). |
| avgqu-sz | The average queue length of the I/O requests. |
| await | The average time for I/O requests to be served. |
| r_await | The average time for read requests to be served. |
| w_await | The average time for write requests to be served. |
| svctm | The average service time for I/O requests. |
| %util | The percentage of CPU time spent on I/O requests. |
## dstat
The `dstat` command is used to collect performance statistics about CPUs and I/O devices.
Command output example:
```shell
# dstat 1 5
--total-cpu-usage-- -dsk/total- -net/total- ---paging-- ---system--
usr sys idl wai stl| read writ| recv send| in out | int csw
0 0 99 0 0|9133B 1260k| 0 0 | 0 5B|1059 1009
2 0 98 0 0| 0 212k| 572B 2258B| 0 0 |1099 761
2 0 98 0 0| 0 0 | 320B 1456B| 0 0 | 919 674
2 0 98 0 0| 0 0 | 256B 1448B| 0 0 | 949 665
2 0 98 0 0| 0 0 |1366B 2190B| 0 0 |1031 812
```
Parameter description:
| **Parameter** | **Description** |
| --- | --- |
| usr | The percentage of CPU time spent in running user space processes. |
| sys | The percentage of CPU time spent in running system processes. |
| idl | The percentage of CPU time spent idle. |
| wai | The percentage of CPU time spent in wait. |
| stl | The percentage of CPU time spent on the hypervisor. |
| read | The number of read requests completed per second. |
| writ | The number of write requests completed per second. |
| recv | The number of packets received per second. |
| send | The number of packets sent per second. |
| in | The number of times information is copied into memory. |
| out | The number of times information is moved out of memory. |
| int | The number of interruptions occurred per second. |
| csw | The number of context switchover per second. |
\ No newline at end of file
---
id: mem-monitor
sidebar_position: 7.23
---
# Command for Memory Monitoring
The `free` command can be used to monitor memory usage.
Command output example:
```shell
# free -g
total used free shared buff/cache available
Mem: 251 44 1 0 205 205
Swap: 7 0 7
```
Parameter description:
| **Parameter** | **Description** |
| --- | --- |
| total | The amount of memory. <br />**total** = **used** + **free** + **buff/cache** |
| used | The amount of used memory. |
| free | The amount of free memory. |
| shared | The amount of shared memory. |
| buff/cache | The amount of memory used as buffers and cache. |
| available | The amount of available memory.<br />**available** = **free** + **buff/cache** |
\ No newline at end of file
---
id: network-monitor
sidebar_position: 7.25
---
# Command for Network Monitoring
The topic describes the `netstat` command that is used to monitor network status.
Command output example:
```shell
# netstat -i
Kernel Interface table
Iface MTU RX-OK RX-ERR RX-DRP RX-OVR TX-OK TX-ERR TX-DRP TX-OVR Flg
docker0 1500 70712 0 0 0 131086 0 0 0 BMRU
eno1 1500 124457829 0 101794 0 57907365 0 0 0 BMRU
eno2 1500 0 0 0 0 0 0 0 0 BMU
eno3 1500 0 0 0 0 0 0 0 0 BMU
eno4 1500 0 0 0 0 0 0 0 0 BMU
enp130s0 1500 0 0 0 0 0 0 0 0 BMU
lo 65536 1123 0 0 0 1123 0 0 0 LRU
vethed74 1500 269 0 0 0 407 0 0 0 BMRU
```
Parameter description:
| **Parameter** | **Description** |
| --- | --- |
| Iface | The name of the network adapter. |
| MTU | The maximum transmission unit. Default value: 1500. |
| OK | The number of error-free packets. |
| ERR | The number of damaged packets. |
| DRP | The number of dropped packets. |
| OVR | The number of packets that exceeds the threshold. |
| Flg | The flag set for the interface. The value can be:<br />- **B**: A broadcast address is configured.<br />- **L**: The interface is a loopback device.<br />- **M**: All packets are received.<br />- **R**: The interface is running.<br />- **U**: The interface is active.<br /> |
:::info
**RX** indicates received. **TX** indicates sent.
:::
\ No newline at end of file
---
id: top-commands
sidebar_position: 7.21
---
# The top command
\ No newline at end of file
{
"position": 7.4,
"label": "Database Tuning",
"collapsible": true
}
\ No newline at end of file
---
id: parameter-tuning
sidebar_position: 7.43
---
# Database parameter tuning
\ No newline at end of file
---
id: sql-best-practices
sidebar_position: 7.41
---
# Best Practices for SQL Coding
\ No newline at end of file
---
id: sql-tuning
sidebar_position: 7.42
---
# Optimize SQL Statements
\ No newline at end of file
{
"position": 7.5,
"label": "Architecture Tuning",
"collapsible": true
}
\ No newline at end of file
---
id: read_write-splitting
sidebar_position: 7.51
---
# Read/Write Splitting
\ No newline at end of file
{
"position": 7.61,
"label": "OLAP",
"collapsible": true
}
\ No newline at end of file
---
id: olap-performance-test-method
sidebar_position: 7.611
---
# OLAP Performance Test Method
\ No newline at end of file
---
id: tcph-test-report
sidebar_position: 7.612
---
# TCP-H Test Report
\ No newline at end of file
{
"position": 7.62,
"label": "OLTP",
"collapsible": true
}
\ No newline at end of file
---
id: oltp-performance-test-method
sidebar_position: 7.621
---
# OLTP Performance Test Method
\ No newline at end of file
{
"position": 7.6,
"label": "Performance Tests",
"collapsible": true
}
\ No newline at end of file
{
"position": 7,
"label": "Performance Tuning",
"collapsible": true
}
\ No newline at end of file
---
id: os-tuning
sidebar_position: 7.3
---
# OS Tuning
\ No newline at end of file
---
id: overview
sidebar_position: 7.1
---
# Overview
\ No newline at end of file
{
"position": 8,
"label": "Data Migration to StoneDB",
"collapsible": true
}
\ No newline at end of file
---
id: use-gravity-to-migrate
sidebar_position: 8.2
---
# Use Gravity to Migrate Data to StoneDB
## Gravity introduction
Gravity is a data migration tool developed by Mobike written in Golang. Though it is not frequently updated on GitHub, many developers are responding to issues. Gravity supports full synchronization, incremental synchronization, and publish of data updates to message queues. It can be deployed on elastic cloud servers (ECSs), Docker containers, and Kubernetes containers.
It is designed to be a customizable data migration tool that:
- Supports multiple data sources and destinations.
- Supports Kubernetes-based replication clusters.
*TODO*
For more information about Gravity on GitHub, visit [https://github.com/moiot/gravity](https://github.com/moiot/gravity).
## Use cases
- Data Bus: uses change data capture (MySQL binlog, MongoDB Oplog) and batch table scan to publish data to Kafka for downstream consumption.
- Unidirectional data synchronization: fully or incrementally synchronizes data from one MySQL cluster to another MySQL cluster.
- Bidirectional data synchronization: fully or incrementally synchronizes data between two MySQL clusters.
- Synchronization of shards to the merged table: synchronizes MySQL sharded tables to the merged table. You can specify the corresponding relationship between the source table and the destination table.
- Online data mutation: supports data changes during the replication. For example, you can rename, encrypt, and decrypt columns.
## Features
- **Input support**
| **Input** | **Status** |
| --- | --- |
| MySQL Binlog | ✅ |
| MySQL Scan | ✅ |
| Mongo Oplog | ✅ |
| TiDB Binlog | Doing |
| PostgreSQL WAL | Doing |
- **Output support**
| **Output** | **Status** |
| --- | --- |
| Kafka | ✅ |
| MySQL/TiDB | ✅ |
| MongoDB | Doing |
- **Data mutation support**
| **Mutation** | **Status** |
| --- | --- |
| Filter data | ✅ |
| Rename columns | ✅ |
| Delete columns | ✅ |
For information about the architecture, visit: [https://github.com/moiot/gravity/blob/master/docs/2.0/00-arch.md](https://github.com/moiot/gravity/blob/master/docs/2.0/00-arch.md).
### Limits
The binlog format of the data source can only be** row**.
## **Configuration file example**
```bash
# 'name' specifies the cluster name. It is mandatory.
name = "mysql2mysqlDemo"
# Name of the database that stores information about binlog positions and heartbeats. The default value is '_gravity'. This database is automatically generated on the data source.
internal-db-name = "_gravity"
#
# Define the input plugin. The following uses 'mysql' as an example.
#
[input]
# Type of the databases used for synchronization.
type = "mysql"
# Synchronization task type. Possible values are 'stream', 'batch', and 'replication'. 'stream' specifies incremental synchronization, 'batch' specifies full synchronization, and 'replication' specifies both full synchronization and incremental synchronization.
mode = "replication"
[input.config.source]
host = "192.168.30.183"
username = "zz"
password = "********"
port = 3307
#
# Define the output plugin. The following uses 'mysql' as an example.
#
[output]
type = "mysql"
[output.config.target]
host = "192.168.30.101"
username = "root"
password = "********"
port = 3306
# Define routing rules.
[[output.config.routes]]
match-schema = "zg"
match-table = "test_source_table"
target-schema = "zg"
target-table = "test_target_table
```
## Deployment schemes
### Deploy Gravity on a Docker container
```shell
docker run -d -p 8080:8080 -v ${PWD}/config.toml:/etc/gravity/config.toml --net=host --name=innodb2stone moiot/gravity:latest
```
### Deploy Gravity on a Kubernetes container
```shell
wget https://codeload.github.com/moiot/gravity-operator/tar.gz/refs/tags/v0.3.12 -C gravity-operator-0.3.12.tar.gz
tar -zxvf gravity-operator-0.3.12.tar.gz
cd gravity-operator/charts/gravity-operator
helm install --name gravity-operator ./
```
Then perform the following steps to create a synchronization task:
1. On the Kubernetes dashboard, check that Gravity is running properly and find the port corresponding to **admin web-server**.
1. Use the port to log in to Gravity.
1. Configure the template to create the synchronization task.
The parameters that you need to configure in the template are similar to those provided in the configuration file example.
### Deploy Gravity on an ECS
We do not recommend this scheme because it requires preparations of the Golang environment and the compilation is complex.
```shell
git clone https://github.com/moiot/gravity.git
cd gravity && make
bin/gravity -config mysql2mysql.toml
```
## Configure monitoring for synchronization tasks
Add Gravity to Prometheus to monitor synchronization tasks. The following code provides an example.
```bash
- job_name: "gravity_innodb2stone"
static_configs:
- targets: ["192.168.46.150:8080"]
labels:
instance: innodb2stone
```
The following are two screenshot examples of the Grafana monitoring dashboard. For details about display templates of Grafana, visit [https://github.com/moiot/gravity/tree/master/deploy/grafana](https://github.com/moiot/gravity/tree/master/deploy/grafana).
**Example 1:**
*image todo*
**Example 2:**
*image todo*
\ No newline at end of file
---
id: use-outter-to-migrate
sidebar_position: 8.1
---
# Use Outter to Migrate Data to StoneDB
{
"position": 9,
"label": "Troubleshooting",
"collapsible": true
}
\ No newline at end of file
---
id: excessive-large-directory
sidebar_position: 9.2
---
# Excessively Large Data Directory
The data directory contains data files, binlogs, and error logs. If the data directory is run out of capacity, the database will be suspended and cannot provide services. To prevent this issue, monitoring on capacity usage must be strengthened in routine maintenance. This topic describes common causes of this issue.
## Big transactions
If big transactions exist, a large amount of binlogs are generated. If the binlog cache is insufficient, excessive binlogs will be temporarily stored to temporary files on disks. Big transactions not only occupy too much disk space, but result in long primary/secondary replication latency. Therefore, we recommend that you split each big transactions into multiple small transactions in your production environment.
## CARTESIAN JOIN
When SQL statements do not strictly follow syntaxes, for example, no condition is specified during table association, Cartesian products are generated. If the tables to associate are large, the table space will be used up. Therefore, we recommend that you check the execution plan each time you finish writing an SQL statement. If "Using join buffer (Block Nested Loop)" exists in the execution plan, check the two associated tables to see whether the association condition of the driven table is not indexed or no association condition exists.
:::info
You can start StoneDB to release the temporary table space.
:::
## Subqueries and grouped orderings
Subqueries and grouped orderings use temporary tables to cache intermediate result sets. If the temporary files run out of space, the intermediate result sets will be temporarily stored to temporary files on disks.
---
id: failed-to-connect
sidebar_position: 9.7
---
# Failed to Connect to StoneDB
\ No newline at end of file
---
id: failed-to-operate-table
sidebar_position: 9.5
---
# Failed to Operate on Data in StoneDB Tables
\ No newline at end of file
---
id: failed-to-start-in-kvm
sidebar_position: 9.6
---
# Failed to Start StoneDB in a KVM
\ No newline at end of file
---
id: failed-to-start
sidebar_position: 9.1
---
# Failed to Start StoneDB
Many issues will cause start failures of StoneDB. If StoneDB cannot be started, we recommend you check whether any error information is recorded in **mysqld.log**. This topic describes common causes of a start failure of StoneDB.
## Improper parameter settings
If the failure is caused by improper parameter settings, check **mysqld.log** to see which parameters are improperly configured.
The following example indicates that parameter **datadir** is improperly configured.
```bash
[ERROR] failed to set datadir to /stonedb/install/dataxxx/
```
## Denial to access resources
If the port is occupied, the directory owner is incorrect, or the permission on the directory is insufficient, you cannot access the directory.
```bash
Error: unable to create temporary file; errno: 13
```
## Damaged data pages
If a relevant data page is damaged, StoneDB cannot be started. In this case, you must restore the data page from a backup.
---
id: mdl-wait
sidebar_position: 9.3
---
# Metadata Lock Waits
\ No newline at end of file
---
id: resource-bottleneck
sidebar_position: 9.9
---
# Diagnose System Resource Bottlenecks
\ No newline at end of file
---
id: slow-query
sidebar_position: 9.8
---
# Diagnose Slow SQL Queries
\ No newline at end of file
---
id: stonedb-crashed
sidebar_position: 9.4
---
# StoneDB Crashed
\ No newline at end of file
{
"position": 10,
"label": "FAQ",
"collapsible": true
}
\ No newline at end of file
---
id: install-faq
sidebar_position: 10.1
---
# Installation FAQ
\ No newline at end of file
---
id: stonedb-faq
sidebar_position: 10.2
---
# StoneDB FAQ
\ No newline at end of file
---
id: troubleshoot-faq
sidebar_position: 10.3
---
# Other FAQ
\ No newline at end of file
......@@ -3,8 +3,14 @@ id: download
sidebar_position: 100
---
# Please wait for release
Release time: June 30, 2022
# Downloads
## StoneDB
Click here to download the latest installation package of StoneDB.
| Version | Release Date | Installation Package | MD5 |
| --- | --- | --- | --- |
| 1.0 | 2022-6-29 | [Download](https://static.stoneatom.com/stonedb-ce-5.6-v1.0.0.el7.x86_64.tar.gz) | 831f5799034fbb388f6ff26675b7951e |
| Next Version | Coming soon~ | |
---
id: release-notes
sidebar_position: 11.0
---
# Release Notes
Release time: June 30, 2022
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册