Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
looyolo
scrapy
提交
19ca986a
S
scrapy
项目概览
looyolo
/
scrapy
与 Fork 源项目一致
从无法访问的项目Fork
通知
2
Star
0
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
S
scrapy
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
提交
19ca986a
编写于
9月 02, 2016
作者:
K
Konstantin Lopuhin
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
Move garbage_collect to scrapy.utils.python
上级
7c67047e
变更
2
隐藏空白更改
内联
并排
Showing
2 changed file
with
22 addition
and
2 deletion
+22
-2
scrapy/cmdline.py
scrapy/cmdline.py
+2
-2
scrapy/utils/python.py
scrapy/utils/python.py
+20
-0
未找到文件。
scrapy/cmdline.py
浏览文件 @
19ca986a
...
...
@@ -4,7 +4,6 @@ import optparse
import
cProfile
import
inspect
import
pkg_resources
import
gc
import
scrapy
from
scrapy.crawler
import
CrawlerProcess
...
...
@@ -12,6 +11,7 @@ from scrapy.commands import ScrapyCommand
from
scrapy.exceptions
import
UsageError
from
scrapy.utils.misc
import
walk_modules
from
scrapy.utils.project
import
inside_project
,
get_project_settings
from
scrapy.utils.python
import
garbage_collect
from
scrapy.settings.deprecated
import
check_deprecated_settings
def
_iter_command_classes
(
module_name
):
...
...
@@ -171,4 +171,4 @@ if __name__ == '__main__':
finally
:
# Twisted prints errors in DebugInfo.__del__, but PyPy does not run gc.collect()
# on exit: http://doc.pypy.org/en/latest/cpython_differences.html?highlight=gc.collect#differences-related-to-garbage-collection-strategies
g
c
.
collect
()
g
arbage_
collect
()
scrapy/utils/python.py
浏览文件 @
19ca986a
"""
This module contains essential stuff that should've come with Python itself ;)
"""
import
gc
import
os
import
re
import
inspect
...
...
@@ -8,6 +9,8 @@ import weakref
import
errno
import
six
from
functools
import
partial
,
wraps
import
sys
import
time
from
scrapy.utils.decorators
import
deprecated
...
...
@@ -355,3 +358,20 @@ def global_object_name(obj):
'scrapy.http.request.Request'
"""
return
"%s.%s"
%
(
obj
.
__module__
,
obj
.
__name__
)
if
sys
.
platform
.
startswith
(
'java'
):
def
garbage_collect
():
# Some JVM GCs will execute finalizers in a different thread, meaning
# we need to wait for that to complete before we go on looking for the
# effects of that.
gc
.
collect
()
time
.
sleep
(
0.1
)
elif
hasattr
(
sys
,
"pypy_version_info"
):
def
garbage_collect
():
# Collecting weakreferences can take two collections on PyPy.
gc
.
collect
()
gc
.
collect
()
else
:
def
garbage_collect
():
gc
.
collect
()
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录