Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
looyolo
scrapy
提交
9e3b868d
S
scrapy
项目概览
looyolo
/
scrapy
与 Fork 源项目一致
从无法访问的项目Fork
通知
2
Star
0
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
S
scrapy
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
提交
9e3b868d
编写于
4月 03, 2021
作者:
A
Andrey Rakhmatullin
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
Use __qualname__ in middleware handling.
上级
8c5a3a51
变更
2
显示空白变更内容
内联
并排
Showing
2 changed file
with
6 addition
and
13 deletion
+6
-13
scrapy/core/downloader/middleware.py
scrapy/core/downloader/middleware.py
+3
-6
scrapy/core/spidermw.py
scrapy/core/spidermw.py
+3
-7
未找到文件。
scrapy/core/downloader/middleware.py
浏览文件 @
9e3b868d
...
...
@@ -36,8 +36,7 @@ class DownloaderMiddlewareManager(MiddlewareManager):
response
=
yield
deferred_from_coro
(
method
(
request
=
request
,
spider
=
spider
))
if
response
is
not
None
and
not
isinstance
(
response
,
(
Response
,
Request
)):
raise
_InvalidOutput
(
f
"Middleware
{
method
.
__self__
.
__class__
.
__name__
}
"
".process_request must return None, Response or "
f
"Middleware
{
method
.
__qualname__
}
must return None, Response or "
f
"Request, got
{
response
.
__class__
.
__name__
}
"
)
if
response
:
...
...
@@ -55,8 +54,7 @@ class DownloaderMiddlewareManager(MiddlewareManager):
response
=
yield
deferred_from_coro
(
method
(
request
=
request
,
response
=
response
,
spider
=
spider
))
if
not
isinstance
(
response
,
(
Response
,
Request
)):
raise
_InvalidOutput
(
f
"Middleware
{
method
.
__self__
.
__class__
.
__name__
}
"
".process_response must return Response or Request, "
f
"Middleware
{
method
.
__qualname__
}
must return Response or Request, "
f
"got
{
type
(
response
)
}
"
)
if
isinstance
(
response
,
Request
):
...
...
@@ -70,8 +68,7 @@ class DownloaderMiddlewareManager(MiddlewareManager):
response
=
yield
deferred_from_coro
(
method
(
request
=
request
,
exception
=
exception
,
spider
=
spider
))
if
response
is
not
None
and
not
isinstance
(
response
,
(
Response
,
Request
)):
raise
_InvalidOutput
(
f
"Middleware
{
method
.
__self__
.
__class__
.
__name__
}
"
".process_exception must return None, Response or "
f
"Middleware
{
method
.
__qualname__
}
must return None, Response or "
f
"Request, got
{
type
(
response
)
}
"
)
if
response
:
...
...
scrapy/core/spidermw.py
浏览文件 @
9e3b868d
...
...
@@ -18,10 +18,6 @@ def _isiterable(possible_iterator):
return
hasattr
(
possible_iterator
,
'__iter__'
)
def
_fname
(
f
):
return
f
"
{
f
.
__self__
.
__class__
.
__name__
}
.
{
f
.
__func__
.
__name__
}
"
class
SpiderMiddlewareManager
(
MiddlewareManager
):
component_name
=
'spider middleware'
...
...
@@ -46,7 +42,7 @@ class SpiderMiddlewareManager(MiddlewareManager):
try
:
result
=
method
(
response
=
response
,
spider
=
spider
)
if
result
is
not
None
:
msg
=
(
f
"Middleware
{
_fname
(
method
)
}
must return None "
msg
=
(
f
"Middleware
{
method
.
__qualname__
}
must return None "
f
"or raise an exception, got
{
type
(
result
)
}
"
)
raise
_InvalidOutput
(
msg
)
except
_InvalidOutput
:
...
...
@@ -83,7 +79,7 @@ class SpiderMiddlewareManager(MiddlewareManager):
elif
result
is
None
:
continue
else
:
msg
=
(
f
"Middleware
{
_fname
(
method
)
}
must return None "
msg
=
(
f
"Middleware
{
method
.
__qualname__
}
must return None "
f
"or an iterable, got
{
type
(
result
)
}
"
)
raise
_InvalidOutput
(
msg
)
return
_failure
...
...
@@ -108,7 +104,7 @@ class SpiderMiddlewareManager(MiddlewareManager):
if
_isiterable
(
result
):
result
=
self
.
_evaluate_iterable
(
response
,
spider
,
result
,
method_index
+
1
,
recovered
)
else
:
msg
=
(
f
"Middleware
{
_fname
(
method
)
}
must return an "
msg
=
(
f
"Middleware
{
method
.
__qualname__
}
must return an "
f
"iterable, got
{
type
(
result
)
}
"
)
raise
_InvalidOutput
(
msg
)
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录