locale.getpreferredencoding(False) is not always utf8

In order to include pip/setuptools/wheel in the freeze output
Closes #1610

closes #3201

closes #703

* Add --require-hashes option. This is handy in deployment scripts to force application authors to hash their requirements. It is also a convenient way to get pip to show computed hashes for a virgin, unhashed requirements file. Eventually, additions to `pip freeze` should fill a superset of this use case.
  * In --require-hashes mode, at least one hash is required to match for each requirement.
  * Option-based requirements (--sha256=...) turn on --require-hashes mode implicitly.
  * Internet-derived URL-based hashes are "necessary but not sufficient": they do not satisfy --require-hashes mode when they match, but they are still used to guard against transmission errors.
  * Other URL-based requirements (#md5=...) are treated just like flag-based ones, except they don't turn on --require-hashes.
* Complain informatively, with the most devastating errors first so you don't chase your tail all day only to run up against a brick wall at the end. This also means we don't complain that a hash is missing, only for the user to find, after fixing it, that we have no idea how to even compute a hash for that type of requirement.
  * Complain about unpinned requirements when hash-checking mode is on, lest they cause the user surprise later.
  * Complain about missing hashes.
  * Complain about requirement types we don't know how to hash (like VCS ones and local dirs).
* Have InstallRequirement keep its original Link around (original_link) so we can differentiate between URL hashes from requirements files and ones downloaded from the (untrustworthy) internet.
* Remove test_download_hashes, which is obsolete. Similar coverage is provided in test_utils.TestHashes and the various hash cases in test_req.py.

Still not very happy

And call abspath() when not following symlinks

Make pip.utils.rmtree retry in case antivirus etc holds a directory
(cherry picked from commit 065b76c3)

By extracting the logic into pip.operations.freeze, the hope is that
folks could do a freeze programmatically more easily.

flake8 ignores some errors by default, these ignored by defaults
change sometimes. This will make things stabler by selecting
everything.

With the vendoring of pkg_resources we no longer need to worry
about the version of the installed setuptools.

* Move virtualenv creation out of TestPipEnvironment
* Remove global state and force explicit use of TestPipEnvironment
  instances
* Remove "backup" virtualenv copying and instead create new
  virtual environments each time.
* Remove the monkeypatched "PyPICache" functionality
* Remove things that were not being used anymore and were dead
  weight
* Remove sitecustomize support which was primarily used to
  monkeypatch the "PyPICache" but was used in one or two other
  tests.

* Better output with the bare asserts we use throughout the tests
* Function fixtures are pretty nice, especially as a way to
  start a background server or create an isolated virtualenv