* changed for new name as "macOS"  (issue #4308)

* updated macOS name

* update macOS name

* updated macOS name

* update for new macOS name

* docs/intro/install.rst: fix macOS header symbols
Co-Authored-By: Nelacuesta <elacuesta@users.noreply.github.com>
Co-authored-by: NAdrián Chaves <adrian@chaves.io>
Co-authored-by: Nelacuesta <elacuesta@users.noreply.github.com>

Stop deprecation warnings on arbitrary SCRAPY-prefixed env vars

Fix the ReadTheDocs build

* fix issue 4175 - Scrapy does not use a non-zero exit code when pipeline's open_spider throws the exception

* remove extra blank lines

* remove redundant code

* remove blank line at end of file

* more suitable naming for response and make if-condition shorter

* avoid error - AttributeError: 'Deferred' object has no attribute 'result'

* use getattr to make code concisely

* add test

* remove useless file

* modify test class name

* remove unneccessary files

* Fix Flake8-reported issue

* fix these items which are suggested by Gallaecio
・Sort those imports at tests/test_cmdline_crawl_with_pipeline/__init__.py
・Remove the unused setUp method.
・Remove comments generated by Scrapy’s project generation tool.
・Remove the [deploy] section from the scrapy.cfg file (I don’t think it’s needed here)
・Remove BOT_NAME and NEWSPIDER_MODULE from settings.py (I think there are not needed either, although I’m less sure about NEWSPIDER_MODULE)

* have to reserve BOT_NAME, SPIDER_MODULES in settings.py

* Remove unneeded empty lines

* Empty __init__.py file with unneeded comments

* Remove an unneeded empty line at the end

* Remove unneeed empty line from __init__.py file

* Update __init__.py

* Update __init__.py

* Update exception.py

* Update normal.py

* Update __init__.py

* Update __init__.py

* fix W391 blank line at end of file
Co-authored-by: NAdrián Chaves <adrian@chaves.io>

Documentation linkcheck run, fixing some links.

Closes #4359

Make BaseItemExporter’s dont_fail parameter keyword-only

Fix flake8 E131, E211, E251, E271, E701

* Expose certificate for HTTPS responses

* Fix test (missing inlineCallbacks decorator)

* Note about Response.certificate

* Explicitly cover None as the default value of Response.certificate
Co-authored-by: NAdrián Chaves <adrian@chaves.io>

* move serialization/deserialization logic to downstream queues

* make memory queues conform to common interface

* make ScrapyPriorityQueue conform common interface

* ScrapyPriorityQueue works with disk

* make key as string

* return list instead of dict as earlier

* downloader aware pq works with new interface

* we don`t need these methods anymore

* create directories for files

* remove dummy priority

* remove priority as parameter, let every queue decide for itself

* rename obj to request

* DownloaderAwarePriorityQueue is too thin wrapper around _SlotPriorityQueues, just remove second one

* remove priority as parameter, let every queue decide for itself

* rename argument

* more granular class separation

* python2 compatible

* one more argument for common interface

* more simple downstream queue interface

* single place for easier customization

* rename function

* shorter

* shorter

* use named arguments

* fix typo

* add docstring

* Update scrapy/pqueues.py
Co-Authored-By: NMikhail Korobov <kmike84@gmail.com>

* Update scrapy/pqueues.py
Co-Authored-By: NMikhail Korobov <kmike84@gmail.com>

* 4 spaces indentation

* we ok with existing directories

* remove unused import

* rename method

* remove unused imports

* it has no sense now

* relining

* note about queues

* add value

* Revert "it has no sense now"

This reverts commit b61604275ba090ebd8e30a6d3a6fbe281c74c189.

* pep8 E261

* pep8 E303

* pep8 E501

* pep8 E123

* pep8 E123

* use create instance

* remove excessive import
Co-authored-by: NMikhail Korobov <kmike84@gmail.com>

Support yield in async def callbacks.

fix E22X flake8

delete old deprecated functions from scrapy.utils.python

Remove deprecated `sel` shortcut in scrapy shell

Implement Response.cb_kwargs

LogFormatter error formatting

Use safe_url_string in link extraction

Update installation instructions regarding Python 3 and virtual environments