[Lockfile](https://pypi.org/project/lockfile/) has not been maintained
since around 2015. Further, the functionality it provided seems poor - a
review of the code indicated that it used the presence of the PID file
itself as the lock - in Unix, using a file's existence followed by a
creation is not atomic, so a lock could be prone to race conditions.

The lockfile package also did not clean up after itself - a process
which was destroyed unexpectedly would not clear the created locks, so
some faulty logic was added to mainUtils.py, which checked to see if a
process with the same PID as the lockfile's creator was running. This
is obviously failure prone, as a new process might be assigned the same
PID as the old lockfile's owner, without actually being the same process.

(Of note, the SIG_DFL argument to os.kill() is not a signal at all, but
rather of type signal.handler. It appears that the python cast this
handler to the int 0, which, according to man 2 kill, leads to no signal
being sent, but existance and permission checks are still performed. So
it is a happy accident that this code worked at all)

This commit removes lockfile from the codebase entirely.

It also adds a "PIDLockFile" class which provides an atomic-guarenteed
lock via the mkdir and rmdir commands on Unix - thus, it is not safely
portable to Windows, but this should not be an issue as only Unix-based
utilities use the "simple_main()" function.

PIDLockFile provides API compatible classes to replace most of the
functionality from lockfile.PidLockFile, but does remove any timeout
logic as it was not used in any meaningful sense - a hard-coded timeout
of 1 second was used, but an immediate result of if the lock is held is
sufficient.

PIDLockFile also includes appropriate __enter__, __exit__, and __del__
attributes, so that, should we extend this class in the future, with
syntax is functional, and __del__ calls release, so a process reaped
unexpectedly should still clean its own locks as part of the garbage
collection process.
Authored-by: NTyler Ramer <tramer@pivotal.io>

For the Python testing artifacts used by the CLI tools, utilize the
Concourse cached directories feature to create and use a pip cache dir
shared between task runs.

Be aware, the cache is scoped to the worker the task is run on. We do
not get a cache hit when subsequent builds run on different workers.

* The environment variable PIP_CACHE_DIR is used to store the cache
directory.

* Add "--retries 10" to Behave test dependency pip install commands.

Python 2 has reached the end-of-life on Jan 1st, 2020, many upstream
python modules are in the progress of dropping python 2 support, their
newer versions may not work for us.  We have encountered several such
kind of issues on the pipeline this year, to make our lifes easier we
now stick the modules to the python2 compatible versions.  For now we
only stick the modules used by the concourse scripts, later we might
want to stick more which are used by our utility scripts.

The current version of more-itertools is 8.1.0 and will cause below
error:

    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-UK2VKZ/more-itertools/setup.py", line 5, in <module>
        from more_itertools import __version__
      File "more_itertools/__init__.py", line 1, in <module>
        from .more import *  # noqa
      File "more_itertools/more.py", line 480
        yield from iterable
                 ^
    SyntaxError: invalid syntax

It used to work well with more-itertools 8.0.2, so we need to specify
that we need a version < 8.1

Also make sure that we call pip install with the proper dependence
requirements.

The combine_cli_coverage job was using pip to pull the latest version of
"coverage", which is 5.x.  However, the actual tests that generate the
coverage files use the gpMgmt/requirements-dev.txt, which pins the
version at 4.5.

This mismatch caused combine_cli_coverage job to fail.  Now we pin
its "coverage" version to 4.5 as well.

Co-authored-by Mark Sliva <msliva@pivotal.io>

Create a concourse job that runs after all CLI behave coverage jobs pass. It
downloads the collected coverage, and combines it into a single file.
Then it generates an html coverage report, which is accessible at the
URL that is printed at the end of the combine coverage job. Python
dependencies are omitted from the coverage output. The html report is
uploaded to the same bucket path as the raw coverage files.
Co-authored-by: NNikolaos Kalampalikis <nkalampalikis@pivotal.io>
Co-authored-by: NJacob Champion <pchampion@pivotal.io>
Co-authored-by: NMark Sliva <msliva@pivotal.io>
Co-authored-by: NJamie McAtamney <jmcatamney@pivotal.io>