From 40ea40e93f983663845872ba4ed21da786ab2272 Mon Sep 17 00:00:00 2001 From: He Wang Date: Fri, 27 May 2022 20:12:54 +0800 Subject: [PATCH] update outdated tutorial & remove git lfs pic (#41) * update logproxy-client doc * uninstall git lfs --- .gitattributes | 1 - README.md | 20 +++++-- docs/images/logproxy-client-workflow.png | 3 - docs/quickstart/logproxy-client-tutorial.md | 62 +++++++++++++++------ 4 files changed, 60 insertions(+), 26 deletions(-) delete mode 100644 .gitattributes delete mode 100644 docs/images/logproxy-client-workflow.png diff --git a/.gitattributes b/.gitattributes deleted file mode 100644 index 24a8e87..0000000 --- a/.gitattributes +++ /dev/null @@ -1 +0,0 @@ -*.png filter=lfs diff=lfs merge=lfs -text diff --git a/README.md b/README.md index 3beeac9..77d3c53 100644 --- a/README.md +++ b/README.md @@ -6,12 +6,24 @@ OceanBase Log Client is a library for obtaining log of [OceanBase](https://githu Getting Started --------------- -There are modules as following: +### Work with LogProxy -- `common`: some common utils -- `logproxy-client`: the client for [oblogproxy](https://github.com/oceanbase/oblogproxy) +#### Use LogProxyClient -You can try the `logproxy-client` following the [LogProxyClient Tutorial](docs/quickstart/logproxy-client-tutorial.md). +You can use `logproxy-client` with [oblogproxy](https://github.com/oceanbase/oblogproxy) to get the commit log of OceanBase cluster, see the [tutorial](docs/quickstart/logproxy-client-tutorial.md) for more details. + +#### Version Compatibility + +`oblogproxy` compresses the record data from `1.0.1`, and `logproxy-client` fixed the decompression process with [#33](https://github.com/oceanbase/oblogclient/pull/33) from `1.0.4`, so there is a version compatibility as follows: + +| `oblogproxy` | `logproxy-client` | +|:----------------:|:-----------------:| +| `1.0.0` | `1.0.0` - `1.0.3` | +| `1.0.1` or later | `1.0.4` or later | + +### Connect to OceanBase Directly + +Coming soon. Communication --------------- diff --git a/docs/images/logproxy-client-workflow.png b/docs/images/logproxy-client-workflow.png deleted file mode 100644 index 76b8ed5..0000000 --- a/docs/images/logproxy-client-workflow.png +++ /dev/null @@ -1,3 +0,0 @@ -version https://git-lfs.github.com/spec/v1 -oid sha256:ae573db88f9c517809951ff3df273fafe1ede81a213b2840338fff74dff9d072 -size 53062 diff --git a/docs/quickstart/logproxy-client-tutorial.md b/docs/quickstart/logproxy-client-tutorial.md index ce1e326..c04f667 100644 --- a/docs/quickstart/logproxy-client-tutorial.md +++ b/docs/quickstart/logproxy-client-tutorial.md @@ -47,12 +47,6 @@ If you'd rather like the latest snapshots of the upcoming major version, use our ``` -## Workflow - -![image](../images/logproxy-client-workflow.png) - -When `LogProxyClient.start()` is executed, a new thread will be created in `ClientStream`. The thread will initialize a netty channel which will receive log data from LogProxy and put the data as `TransferPacket` to a BlockingQueue. When the netty connection is established, the thread will poll the queue and pass the `LogMessage` in TransferPacket to `RecordListener.notify`. - ## Usage ### Basic Usage @@ -70,18 +64,34 @@ To connect to LogProxy, there are some parameters to set in `ObReaderConfig`: These parameters are used in `obcdc` (former `liboblog`), and the items not listed above can be passed to `obcdc` through the `ObReaderConfig` constructor with parameters. -Here is an example to set ObReaderConfig with a user of sys tenant, and the OceanBase and LogProxy server are on the same machine. +Here is an example to set `ObReaderConfig` with OceanBase Community Edition: ```java -ObReaderConfig config = new ObReaderConfig(); +// obcdc params that are not listed above can be passed through the constructor +Map extraConfigs = new HashMap<>(); +extraConfigs.put("working_mode", "storage"); + +// set 'rootserver_list' and other params +ObReaderConfig config = new ObReaderConfig(extraConfigs); config.setRsList("127.0.0.1:2882:2881"); -config.setUsername("user@sys"); -config.setPassword("pswd"); +config.setUsername("username"); +config.setPassword("password"); config.setStartTimestamp(0L); -config.setTableWhiteList("sys.db1.tb1|sys.db2.*"); +config.setTableWhiteList("tenant.*.*"); ``` -Once ObReaderConfig is set properly, you can use it to instance a LogProxyClient and listen to the log data. +If you want to work with OceanBase Enterprise Edition, you can set the `ObReaderConfig` with `cluster_url` like below: + +```java +ObReaderConfig config = new ObReaderConfig(); +config.setClusterUrl("http://127.0.0.1:8080/services?Action=ObRootServiceInfo&User_ID=alibaba&UID=ocpmaster&ObRegion=tenant"); +config.setUsername("username"); +config.setPassword("password"); +config.setStartTimestamp(0L); +config.setTableWhiteList("tenant.*.*"); +``` + +Once ObReaderConfig is set properly, you can use it to instance a LogProxyClient and monitor the log data. ```java LogProxyClient client = new LogProxyClient("127.0.0.1", 2983, config); @@ -96,10 +106,7 @@ client.addListener(new RecordListener() { @Override public void onException(LogProxyClientException e) { - if (e.needStop()) { - // add error hander here - client.stop(); - } + logger.error(e.getMessage()); } }); @@ -109,7 +116,25 @@ client.join(); The method `LogProxyClient.start()` will start a new thread which serving with a netty socket to receive data from LogProxy. -For details about `LogMessage`, see [LogMessage](../formats/logmessage.md). +There are also some configurations for the client in `ClientConf`, if you don't want to use its default values, you can customize a `ClientConf` and pass it to the corresponding constructor to create the client instance. + +```java +ClientConf clientConf = + ClientConf.builder() + .clientId("myClientId") + .transferQueueSize(1024) + .connectTimeoutMs(1000) + .readWaitTimeMs(1000) + .retryIntervalS(1) + .maxReconnectTimes(10) + .idleTimeoutS(10) + .nettyDiscardAfterReads(1) + .ignoreUnknownRecordType(true) + .build(); +LogProxyClient client = new LogProxyClient("127.0.0.1", 2983, config, clientConf); +``` + +The received log records are parsed to `LogMessage` in the client handler, you can see [LogMessage doc](../formats/logmessage.md) for more details. ### SSL Encryption @@ -123,7 +148,8 @@ SslContext sslContext = SslContextBuilder.forClient() this.getClass().getClassLoader().getResourceAsStream("client.crt"), this.getClass().getClassLoader().getResourceAsStream("client.key")) .build(); -LogProxyClient client = new LogProxyClient("127.0.0.1", 2983, config, sslContext); +ClientConf clientConf = ClientConf.builder().sslContext(sslContext).build(); +LogProxyClient client = new LogProxyClient("127.0.0.1", 2983, config, clientConf); ``` Here you need provide following files: -- GitLab