- 22 2月, 2012 1 次提交
-
-
由 Pablo Hoffman 提交于
-
- 19 2月, 2012 2 次提交
-
-
由 Pablo Hoffman 提交于
-
由 Pablo Hoffman 提交于
-
- 18 2月, 2012 2 次提交
-
-
由 Daniel Graña 提交于
identify autothrottle debug mode stats with slot key, in order to allow to track concurrency/delay issues with spiders which crawls more than one site.
-
由 Martin Olveyra 提交于
track concurrency/delay issues with spiders which crawls more than one site.
-
- 17 2月, 2012 2 次提交
-
-
由 Pablo Hoffman 提交于
-
由 Pablo Hoffman 提交于
-
- 12 2月, 2012 1 次提交
-
-
由 Pablo Hoffman 提交于
-
- 08 2月, 2012 1 次提交
-
-
由 Daniel Graña 提交于
-
- 28 1月, 2012 1 次提交
-
-
由 Daniel Graña 提交于
-
- 26 1月, 2012 3 次提交
-
-
由 Daniel Graña 提交于
-
由 Daniel Graña 提交于
-
由 Daniel Graña 提交于
-
- 17 1月, 2012 3 次提交
-
-
由 Pablo Hoffman 提交于
autothrottle code improvements (download delay + style)
-
由 Martin Olveyra 提交于
-
由 Martin Olveyra 提交于
limit download delay to a minimal of spider.download_delay if given
-
- 16 1月, 2012 1 次提交
-
-
由 Pablo Hoffman 提交于
~10x speed-up for libxml2 XPathSelector
-
- 15 1月, 2012 1 次提交
-
-
由 Сергей Прохоров 提交于
-
- 13 1月, 2012 3 次提交
-
-
由 Pablo Hoffman 提交于
-
由 Pablo Hoffman 提交于
-
由 Pablo Hoffman 提交于
-
- 05 1月, 2012 3 次提交
-
-
由 Pablo Hoffman 提交于
-
由 Pablo Hoffman 提交于
-
由 Pablo Hoffman 提交于
-
- 04 1月, 2012 5 次提交
-
-
由 Pablo Hoffman 提交于
-
由 Pablo Hoffman 提交于
Items are stored the same way as logs, in jsonlines format. Also renamed logs_to_keep setting to jobs_to_keep.
-
由 Pablo Hoffman 提交于
-
由 Pablo Hoffman 提交于
-
由 Pablo Hoffman 提交于
-
- 03 1月, 2012 1 次提交
-
-
由 Pablo Hoffman 提交于
SitemapSpider: added support for sitemap urls ending in .xml and .xml.gz, even if they have a wrong content type
-
- 02 1月, 2012 4 次提交
-
-
由 Pablo Hoffman 提交于
-
由 Pablo Hoffman 提交于
bumped version to 0.15.1, mainly to avoid package upgrade issues with new versioning based on git describe
-
由 Pablo Hoffman 提交于
-
由 Pablo Hoffman 提交于
tests: fixed httpcache testcase.
-
- 31 12月, 2011 4 次提交
-
-
-
由 Pablo Hoffman 提交于
httpcache: keep stats of cache hit/miss/store and don't store already cached response
-
-
-
- 29 12月, 2011 2 次提交
-
-
由 Pablo Hoffman 提交于
-
由 Pablo Hoffman 提交于
* the api is now a GET instead of POST (for consistency) * the api also returns pending and finished jobs, in addition to running ones * only the last 100 finished jobs are kept (can be changed through the finished_to_keep setting)
-