identify autothrottle debug mode stats with slot key, in order to allow to track concurrency/delay issues with spiders which crawls more than one site.

track concurrency/delay issues with spiders which crawls more than one
site.

autothrottle code improvements (download delay + style)

limit download delay to a minimal of spider.download_delay if given

~10x speed-up for libxml2 XPathSelector

Items are stored the same way as logs, in jsonlines format.

Also renamed logs_to_keep setting to jobs_to_keep.

SitemapSpider: added support for sitemap urls ending in .xml and .xml.gz, even if they have a wrong content type

bumped version to 0.15.1, mainly to avoid package upgrade issues with new versioning based on git describe

tests: fixed httpcache testcase.

httpcache: keep stats of cache hit/miss/store and don't store already cached response

* the api is now a GET instead of POST (for consistency)
* the api also returns pending and finished jobs, in addition to running
  ones
* only the last 100 finished jobs are kept (can be changed through the
  finished_to_keep setting)