Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
lijianghuflute
canal
提交
991418ce
canal
项目概览
lijianghuflute
/
canal
与 Fork 源项目一致
从无法访问的项目Fork
通知
3
Star
0
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
canal
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
提交
991418ce
编写于
11月 16, 2018
作者:
M
mcy
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
修改rdb批量提交分批的问题
上级
d34bee33
变更
3
隐藏空白更改
内联
并排
Showing
3 changed file
with
20 addition
and
28 deletion
+20
-28
client-adapter/launcher/src/main/java/com/alibaba/otter/canal/adapter/launcher/config/RefresherConfig.java
.../otter/canal/adapter/launcher/config/RefresherConfig.java
+0
-22
client-adapter/rdb/src/main/java/com/alibaba/otter/canal/client/adapter/rdb/RdbAdapter.java
...om/alibaba/otter/canal/client/adapter/rdb/RdbAdapter.java
+19
-6
client-adapter/rdb/src/main/java/com/alibaba/otter/canal/client/adapter/rdb/service/RdbSyncService.java
...tter/canal/client/adapter/rdb/service/RdbSyncService.java
+1
-0
未找到文件。
client-adapter/launcher/src/main/java/com/alibaba/otter/canal/adapter/launcher/config/RefresherConfig.java
已删除
100644 → 0
浏览文件 @
d34bee33
package
com.alibaba.otter.canal.adapter.launcher.config
;
import
org.springframework.cloud.context.refresh.ContextRefresher
;
import
org.springframework.cloud.context.scope.refresh.RefreshScope
;
import
org.springframework.context.ConfigurableApplicationContext
;
import
org.springframework.context.annotation.Bean
;
import
org.springframework.context.annotation.Configuration
;
@Configuration
public
class
RefresherConfig
{
@Bean
public
RefreshScope
refreshScope
()
{
return
new
RefreshScope
();
}
@Bean
public
ContextRefresher
contextRefresher
(
ConfigurableApplicationContext
configurableApplicationContext
,
RefreshScope
refreshScope
)
{
return
new
ContextRefresher
(
configurableApplicationContext
,
refreshScope
);
}
}
client-adapter/rdb/src/main/java/com/alibaba/otter/canal/client/adapter/rdb/RdbAdapter.java
浏览文件 @
991418ce
...
...
@@ -5,6 +5,8 @@ import java.sql.SQLException;
import
java.util.*
;
import
java.util.concurrent.ExecutorService
;
import
java.util.concurrent.Executors
;
import
java.util.concurrent.TimeUnit
;
import
java.util.concurrent.locks.Condition
;
import
java.util.concurrent.locks.Lock
;
import
java.util.concurrent.locks.ReentrantLock
;
...
...
@@ -20,6 +22,7 @@ import com.alibaba.fastjson.serializer.SerializerFeature;
import
com.alibaba.otter.canal.client.adapter.OuterAdapter
;
import
com.alibaba.otter.canal.client.adapter.rdb.config.ConfigLoader
;
import
com.alibaba.otter.canal.client.adapter.rdb.config.MappingConfig
;
import
com.alibaba.otter.canal.client.adapter.rdb.monitor.RdbConfigMonitor
;
import
com.alibaba.otter.canal.client.adapter.rdb.service.RdbEtlService
;
import
com.alibaba.otter.canal.client.adapter.rdb.service.RdbSyncService
;
import
com.alibaba.otter.canal.client.adapter.rdb.support.SimpleDml
;
...
...
@@ -44,8 +47,11 @@ public class RdbAdapter implements OuterAdapter {
private
List
<
SimpleDml
>
dmlList
=
Collections
.
synchronizedList
(
new
ArrayList
<>());
private
Lock
syncLock
=
new
ReentrantLock
();
private
Condition
condition
=
syncLock
.
newCondition
();
private
ExecutorService
executor
=
Executors
.
newFixedThreadPool
(
1
);
private
RdbConfigMonitor
rdbConfigMonitor
;
public
Map
<
String
,
MappingConfig
>
getRdbMapping
()
{
return
rdbMapping
;
}
...
...
@@ -107,18 +113,21 @@ public class RdbAdapter implements OuterAdapter {
executor
.
submit
(()
->
{
while
(
running
)
{
try
{
int
size1
=
dmlList
.
size
();
Thread
.
sleep
(
3000
);
int
size2
=
dmlList
.
size
();
if
(
size1
==
size2
)
{
syncLock
.
lock
();
if
(!
condition
.
await
(
3
,
TimeUnit
.
SECONDS
))
{
// 超时提交
sync
();
}
}
catch
(
Exception
e
)
{
logger
.
error
(
e
.
getMessage
(),
e
);
}
finally
{
syncLock
.
unlock
();
}
}
});
rdbConfigMonitor
=
new
RdbConfigMonitor
();
rdbConfigMonitor
.
init
(
configuration
.
getKey
(),
this
);
}
@Override
...
...
@@ -131,10 +140,9 @@ public class RdbAdapter implements OuterAdapter {
if
(
configMap
!=
null
)
{
configMap
.
values
().
forEach
(
config
->
{
List
<
SimpleDml
>
simpleDmlList
=
SimpleDml
.
dml2SimpleDml
(
dml
,
config
);
dmlList
.
addAll
(
simpleDmlList
);
if
(
dmlList
.
size
()
>
commitSize
)
{
if
(
dmlList
.
size
()
>
=
commitSize
)
{
sync
();
}
});
...
...
@@ -148,6 +156,7 @@ public class RdbAdapter implements OuterAdapter {
try
{
syncLock
.
lock
();
if
(!
dmlList
.
isEmpty
())
{
condition
.
signal
();
rdbSyncService
.
sync
(
dmlList
);
dmlList
.
clear
();
}
...
...
@@ -251,6 +260,10 @@ public class RdbAdapter implements OuterAdapter {
@Override
public
void
destroy
()
{
running
=
false
;
if
(
rdbConfigMonitor
!=
null
)
{
rdbConfigMonitor
.
destroy
();
}
executor
.
shutdown
();
if
(
rdbSyncService
!=
null
)
{
...
...
client-adapter/rdb/src/main/java/com/alibaba/otter/canal/client/adapter/rdb/service/RdbSyncService.java
浏览文件 @
991418ce
...
...
@@ -10,6 +10,7 @@ import java.util.LinkedHashMap;
import
java.util.List
;
import
java.util.Map
;
import
java.util.concurrent.*
;
import
java.util.concurrent.atomic.AtomicInteger
;
import
javax.sql.DataSource
;
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录