Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
looyolo
scrapy
提交
d2aa51c0
S
scrapy
项目概览
looyolo
/
scrapy
与 Fork 源项目一致
从无法访问的项目Fork
通知
2
Star
0
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
S
scrapy
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
提交
d2aa51c0
编写于
2月 27, 2017
作者:
P
Paul Tremberth
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
Update tests
上级
04e4d086
变更
1
隐藏空白更改
内联
并排
Showing
1 changed file
with
40 addition
and
3 deletion
+40
-3
tests/test_spidermiddleware_referer.py
tests/test_spidermiddleware_referer.py
+40
-3
未找到文件。
tests/test_spidermiddleware_referer.py
浏览文件 @
d2aa51c0
from
six.moves.urllib.parse
import
urlparse
from
unittest
import
TestCase
from
scrapy.exceptions
import
NotConfigured
...
...
@@ -326,9 +327,7 @@ class CustomPythonOrgPolicy(ReferrerPolicy):
depending on the scheme of the target URL.
"""
def
referrer
(
self
,
response
,
request
):
from
scrapy.utils.httpobj
import
urlparse_cached
scheme
=
urlparse_cached
(
request
).
scheme
scheme
=
urlparse
(
request
).
scheme
if
scheme
==
'https'
:
return
b
'https://python.org/'
elif
scheme
==
'http'
:
...
...
@@ -446,3 +445,41 @@ class TestRefererMiddlewarePolicyHeaderPredecence002(MixinNoReferrer, TestRefere
class
TestRefererMiddlewarePolicyHeaderPredecence003
(
MixinNoReferrerWhenDowngrade
,
TestRefererMiddleware
):
settings
=
{
'REFERRER_POLICY'
:
'scrapy.spidermiddlewares.referer.OriginWhenCrossOriginPolicy'
}
resp_headers
=
{
'Referrer-Policy'
:
POLICY_NO_REFERRER_WHEN_DOWNGRADE
.
title
()}
class
TestReferrerPolicyOnRedirect
(
TestRefererMiddleware
):
req_meta
=
{}
resp_headers
=
{}
#settings = {}
settings
=
{
'REFERRER_POLICY'
:
'scrapy.spidermiddlewares.referer.UnsafeUrlPolicy'
}
scenarii
=
[
(
# origin
'http://scrapytest.org/1'
,
# target + redirection
[
'http://scrapytest.org/2'
,
'http://scrapytest.org/3'
,],
b
'http://scrapytest.org/1'
,
# expected initial referer
b
'http://scrapytest.org/1'
,
# expected referer for the redirection request
),
]
def
setUp
(
self
):
self
.
spider
=
Spider
(
'foo'
)
settings
=
Settings
(
self
.
settings
)
self
.
mw
=
RefererMiddleware
(
settings
)
def
test_
(
self
):
for
origin
,
target_chain
,
init_referrer
,
final_referrer
in
self
.
scenarii
:
response
=
self
.
get_response
(
origin
)
request
=
self
.
get_request
(
target_chain
.
pop
())
out
=
list
(
self
.
mw
.
process_spider_output
(
response
,
[
request
],
self
.
spider
))
self
.
assertEquals
(
out
[
0
].
headers
.
get
(
'Referer'
),
init_referrer
)
request
.
meta
[
'redirected_urls'
]
=
target_chain
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录