From 6ed08d23329bc33142804ea78320993d0c680175 Mon Sep 17 00:00:00 2001 From: Paul Tremberth Date: Wed, 27 Jan 2016 11:53:29 +0100 Subject: [PATCH] Add note for DEPTH_PRIORITY --- docs/topics/settings.rst | 21 +++++++++++++++++++-- 1 file changed, 19 insertions(+), 2 deletions(-) diff --git a/docs/topics/settings.rst b/docs/topics/settings.rst index 116a10f83..052be4429 100644 --- a/docs/topics/settings.rst +++ b/docs/topics/settings.rst @@ -276,6 +276,8 @@ DEPTH_LIMIT Default: ``0`` +Scope: ``scrapy.spidermiddlewares.depth.DepthMiddleware`` + The maximum depth that will be allowed to crawl for any site. If zero, no limit will be imposed. @@ -286,9 +288,20 @@ DEPTH_PRIORITY Default: ``0`` -An integer that is used to adjust the request priority based on its depth. +Scope: ``scrapy.spidermiddlewares.depth.DepthMiddleware`` + +An integer that is used to adjust the request priority based on its depth: + +- **a positive value will decrease the priority** +- a negative value will increase priority + +If zero (default), no priority adjustment is made from depth. -If zero, no priority adjustment is made from depth. +.. note:: + + This setting adjusts priority **in the opposite way** compared to + other priority settings :setting:`REDIRECT_PRIORITY_ADJUST` + and :setting:`RETRY_PRIORITY_ADJUST`. .. setting:: DEPTH_STATS @@ -297,6 +310,8 @@ DEPTH_STATS Default: ``True`` +Scope: ``scrapy.spidermiddlewares.depth.DepthMiddleware`` + Whether to collect maximum depth stats. .. setting:: DEPTH_STATS_VERBOSE @@ -306,6 +321,8 @@ DEPTH_STATS_VERBOSE Default: ``False`` +Scope: ``scrapy.spidermiddlewares.depth.DepthMiddleware`` + Whether to collect verbose depth stats. If this is enabled, the number of requests for each depth is collected in the stats. -- GitLab