Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
looyolo
scrapy
提交
241ae9f2
S
scrapy
项目概览
looyolo
/
scrapy
与 Fork 源项目一致
从无法访问的项目Fork
通知
2
Star
0
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
S
scrapy
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
提交
241ae9f2
编写于
2月 26, 2016
作者:
P
Paul Tremberth
浏览文件
操作
浏览文件
下载
差异文件
Merge pull request #1820 from redapple/http-tls-settings
[MRG+1] Document DOWNLOADER_* settings for HTTP/1.0 and TLS
上级
84dea194
709b4fa8
变更
2
隐藏空白更改
内联
并排
Showing
2 changed file
with
79 addition
and
0 deletion
+79
-0
docs/news.rst
docs/news.rst
+7
-0
docs/topics/settings.rst
docs/topics/settings.rst
+72
-0
未找到文件。
docs/news.rst
浏览文件 @
241ae9f2
...
...
@@ -26,6 +26,9 @@ This 1.1 release brings a lot of interesting features and bug fixes:
- Selectors were extracted to the parsel_ library (:issue:`1409`). This means
you can use Scrapy Selectors without Scrapy and also upgrade the
selectors engine without needing to upgrade Scrapy.
- HTTPS downloader now does TLS protocol negotiation by default,
instead of forcing TLS 1.0. You can also set the SSL/TLS method
using the new :setting:`DOWNLOADER_CLIENT_TLS_METHOD`.
- These bug fixes may require your attention:
...
...
@@ -85,6 +88,10 @@ Additional New Features and Enhancements
interval (:issue:`1282`).
- Download handlers are now lazy-loaded on first request using their
scheme (:issue:`1390`, :issue:`1421`).
- HTTPS download handlers do not force TLS 1.0 anymore; instead,
OpenSSL's ``SSLv23_method()/TLS_method()`` is used allowing to try
negotiating with the remote hosts the highest TLS protocol version
it can (:issue:`1794`, :issue:`1629`).
- ``RedirectMiddleware`` now skips the status codes from
``handle_httpstatus_list`` on spider attribute
or in ``Request``'s ``meta`` key (:issue:`1334`, :issue:`1364`,
...
...
docs/topics/settings.rst
浏览文件 @
241ae9f2
...
...
@@ -366,6 +366,78 @@ Default: ``'scrapy.core.downloader.Downloader'``
The downloader to use for crawling.
.. setting:: DOWNLOADER_HTTPCLIENTFACTORY
DOWNLOADER_HTTPCLIENTFACTORY
----------------------------
Default: ``'scrapy.core.downloader.webclient.ScrapyHTTPClientFactory'``
Defines a Twisted ``protocol.ClientFactory`` class to use for HTTP/1.0
connections (for ``HTTP10DownloadHandler``).
.. note::
HTTP/1.0 is rarely used nowadays so you can safely ignore this setting,
unless you use Twisted<11.1, or if you really want to use HTTP/1.0
and override :setting:`DOWNLOAD_HANDLERS_BASE` for ``http(s)`` scheme
accordingly, i.e. to
``'scrapy.core.downloader.handlers.http.HTTP10DownloadHandler'``.
.. setting:: DOWNLOADER_CLIENTCONTEXTFACTORY
DOWNLOADER_CLIENTCONTEXTFACTORY
-------------------------------
Default: ``'scrapy.core.downloader.contextfactory.ScrapyClientContextFactory'``
Represents the classpath to the ContextFactory to use.
Here, "ContextFactory" is a Twisted term for SSL/TLS contexts, defining
the TLS/SSL protocol version to use, whether to do certificate verification,
or even enable client-side authentication (and various other things).
.. note::
Scrapy default context factory **does NOT perform remote server
certificate verification**. This is usually fine for web scraping.
If you do need remote server certificate verification enabled,
Scrapy also has another context factory class that you can set,
``'scrapy.core.downloader.contextfactory.BrowserLikeContextFactory'``,
which uses the platform's certificates to validate remote endpoints.
**This is only available if you use Twisted>=14.0.**
If you do use a custom ContextFactory, make sure it accepts a ``method``
parameter at init (this is the ``OpenSSL.SSL`` method mapping
:setting:`DOWNLOADER_CLIENT_TLS_METHOD`).
.. setting:: DOWNLOADER_CLIENT_TLS_METHOD
DOWNLOADER_CLIENT_TLS_METHOD
----------------------------
Default: ``'TLS'``
Use this setting to customize the TLS/SSL method used by the default
HTTP/1.1 downloader.
This setting must be one of these string values:
- ``'TLS'``: maps to OpenSSL's ``TLS_method()`` (a.k.a ``SSLv23_method()``),
which allows protocol negotiation, starting from the highest supported
by the platform; **default, recommended**
- ``'TLSv1.0'``: this value forces HTTPS connections to use TLS version 1.0 ;
set this if you want the behavior of Scrapy<1.1
- ``'TLSv1.1'``: forces TLS version 1.1
- ``'TLSv1.2'``: forces TLS version 1.2
- ``'SSLv3'``: forces SSL version 3 (**not recommended**)
.. note::
We recommend that you use PyOpenSSL>=0.13 and Twisted>=0.13
or above (Twisted>=14.0 if you can).
.. setting:: DOWNLOADER_MIDDLEWARES
DOWNLOADER_MIDDLEWARES
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录