Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
李少辉-开发者
gitlab-foss
提交
cef3fd28
G
gitlab-foss
项目概览
李少辉-开发者
/
gitlab-foss
通知
15
Star
0
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
G
gitlab-foss
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
提交
cef3fd28
编写于
6月 08, 2016
作者:
D
Douglas Barbosa Alexandre
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
Wrap all rate limiting logic inside GitHub API client
上级
3803b380
变更
2
隐藏空白更改
内联
并排
Showing
2 changed file
with
40 addition
and
57 deletion
+40
-57
lib/gitlab/github_import/client.rb
lib/gitlab/github_import/client.rb
+33
-2
lib/gitlab/github_import/importer.rb
lib/gitlab/github_import/importer.rb
+7
-55
未找到文件。
lib/gitlab/github_import/client.rb
浏览文件 @
cef3fd28
module
Gitlab
module
GithubImport
class
Client
GITHUB_SAFE_REMAINING_REQUESTS
=
100
GITHUB_SAFE_SLEEP_TIME
=
500
attr_reader
:client
,
:api
def
initialize
(
access_token
)
...
...
@@ -11,7 +14,7 @@ module Gitlab
)
if
access_token
::
Octokit
.
auto_paginate
=
tru
e
::
Octokit
.
auto_paginate
=
fals
e
@api
=
::
Octokit
::
Client
.
new
(
access_token:
access_token
,
...
...
@@ -36,7 +39,7 @@ module Gitlab
def
method_missing
(
method
,
*
args
,
&
block
)
if
api
.
respond_to?
(
method
)
api
.
send
(
method
,
*
args
,
&
block
)
request
{
api
.
send
(
method
,
*
args
,
&
block
)
}
else
super
(
method
,
*
args
,
&
block
)
end
...
...
@@ -55,6 +58,34 @@ module Gitlab
def
github_options
config
[
"args"
][
"client_options"
].
deep_symbolize_keys
end
def
rate_limit
api
.
rate_limit!
end
def
rate_limit_exceed?
rate_limit
.
remaining
<=
GITHUB_SAFE_REMAINING_REQUESTS
end
def
rate_limit_sleep_time
rate_limit
.
resets_in
+
GITHUB_SAFE_SLEEP_TIME
end
def
request
sleep
rate_limit_sleep_time
if
rate_limit_exceed?
data
=
yield
last_response
=
api
.
last_response
while
last_response
.
rels
[
:next
]
sleep
rate_limit_sleep_time
if
rate_limit_exceed?
last_response
=
last_response
.
rels
[
:next
].
get
data
.
concat
(
last_response
.
data
)
if
last_response
.
data
.
is_a?
(
Array
)
end
data
end
end
end
end
lib/gitlab/github_import/importer.rb
浏览文件 @
cef3fd28
...
...
@@ -3,9 +3,6 @@ module Gitlab
class
Importer
include
Gitlab
::
ShellAdapter
GITHUB_SAFE_REMAINING_REQUESTS
=
100
GITHUB_SAFE_SLEEP_TIME
=
500
attr_reader
:client
,
:project
,
:repo
,
:repo_url
def
initialize
(
project
)
...
...
@@ -28,52 +25,12 @@ module Gitlab
private
def
turn_auto_pagination_off!
client
.
auto_paginate
=
false
end
def
turn_auto_pagination_on!
client
.
auto_paginate
=
true
end
def
rate_limit
client
.
rate_limit!
end
def
rate_limit_exceed?
rate_limit
.
remaining
<=
GITHUB_SAFE_REMAINING_REQUESTS
end
def
rate_limit_sleep_time
rate_limit
.
resets_in
+
GITHUB_SAFE_SLEEP_TIME
end
def
paginate
turn_auto_pagination_off!
sleep
rate_limit_sleep_time
if
rate_limit_exceed?
data
=
yield
last_response
=
client
.
last_response
while
last_response
.
rels
[
:next
]
sleep
rate_limit_sleep_time
if
rate_limit_exceed?
last_response
=
last_response
.
rels
[
:next
].
get
data
.
concat
(
last_response
.
data
)
if
last_response
.
data
.
is_a?
(
Array
)
end
turn_auto_pagination_on!
data
end
def
credentials
@credentials
||=
project
.
import_data
.
credentials
if
project
.
import_data
end
def
import_labels
labels
=
paginate
{
client
.
labels
(
repo
,
per_page:
100
)
}
labels
=
client
.
labels
(
repo
,
per_page:
100
)
labels
.
each
{
|
raw
|
LabelFormatter
.
new
(
project
,
raw
).
create!
}
true
...
...
@@ -82,7 +39,7 @@ module Gitlab
end
def
import_milestones
milestones
=
paginate
{
client
.
milestones
(
repo
,
state: :all
,
per_page:
100
)
}
milestones
=
client
.
milestones
(
repo
,
state: :all
,
per_page:
100
)
milestones
.
each
{
|
raw
|
MilestoneFormatter
.
new
(
project
,
raw
).
create!
}
true
...
...
@@ -91,9 +48,9 @@ module Gitlab
end
def
import_issues
data
=
paginate
{
client
.
issues
(
repo
,
state: :all
,
sort: :created
,
direction: :asc
,
per_page:
100
)
}
issues
=
client
.
issues
(
repo
,
state: :all
,
sort: :created
,
direction: :asc
,
per_page:
100
)
data
.
each
do
|
raw
|
issues
.
each
do
|
raw
|
gh_issue
=
IssueFormatter
.
new
(
project
,
raw
)
if
gh_issue
.
valid?
...
...
@@ -112,7 +69,7 @@ module Gitlab
hooks
=
client
.
hooks
(
repo
).
map
{
|
raw
|
HookFormatter
.
new
(
raw
)
}.
select
(
&
:valid?
)
disable_webhooks
(
hooks
)
pull_requests
=
paginate
{
client
.
pull_requests
(
repo
,
state: :all
,
sort: :created
,
direction: :asc
,
per_page:
100
)
}
pull_requests
=
client
.
pull_requests
(
repo
,
state: :all
,
sort: :created
,
direction: :asc
,
per_page:
100
)
pull_requests
=
pull_requests
.
map
{
|
raw
|
PullRequestFormatter
.
new
(
project
,
raw
)
}.
select
(
&
:valid?
)
source_branches_removed
=
pull_requests
.
reject
(
&
:source_branch_exists?
).
map
{
|
pr
|
[
pr
.
source_branch_name
,
pr
.
source_branch_sha
]
}
...
...
@@ -146,14 +103,12 @@ module Gitlab
def
update_webhooks
(
hooks
,
options
)
hooks
.
each
do
|
hook
|
sleep
rate_limit_sleep_time
if
rate_limit_exceed?
client
.
edit_hook
(
repo
,
hook
.
id
,
hook
.
name
,
hook
.
config
,
options
)
end
end
def
restore_branches
(
branches
)
branches
.
each
do
|
name
,
sha
|
sleep
rate_limit_sleep_time
if
rate_limit_exceed?
client
.
create_ref
(
repo
,
"refs/heads/
#{
name
}
"
,
sha
)
end
...
...
@@ -162,15 +117,12 @@ module Gitlab
def
clean_up_restored_branches
(
branches
)
branches
.
each
do
|
name
,
_
|
sleep
rate_limit_sleep_time
if
rate_limit_exceed?
client
.
delete_ref
(
repo
,
"heads/
#{
name
}
"
)
project
.
repository
.
rm_branch
(
project
.
creator
,
name
)
end
end
def
apply_labels
(
issuable
)
sleep
rate_limit_sleep_time
if
rate_limit_exceed?
issue
=
client
.
issue
(
repo
,
issuable
.
iid
)
if
issue
.
labels
.
count
>
0
...
...
@@ -183,12 +135,12 @@ module Gitlab
end
def
import_comments
(
issuable
)
comments
=
paginate
{
client
.
issue_comments
(
repo
,
issuable
.
iid
,
per_page:
100
)
}
comments
=
client
.
issue_comments
(
repo
,
issuable
.
iid
,
per_page:
100
)
create_comments
(
issuable
,
comments
)
end
def
import_comments_on_diff
(
merge_request
)
comments
=
paginate
{
client
.
pull_request_comments
(
repo
,
merge_request
.
iid
,
per_page:
100
)
}
comments
=
client
.
pull_request_comments
(
repo
,
merge_request
.
iid
,
per_page:
100
)
create_comments
(
merge_request
,
comments
)
end
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录