fixes #3583

* add awesome_bot to check urls

* allow dupes

* make url checking dependent on commit message

* add web.archive.org to whitelist

cuz awesome_bot is having trouble with those links

* fix travis.yaml

* check urls

* need ruby

* another site to avoid

* check urls

* make regex more generic

* sudo deprecated

* make travis linter happy

check url

* try skip results

check url

* timeout was what was killing us

check url

* testing fixing urls

check url

* dont allow timeout

check url

* testing timeout

check url

* put fpb back

check url

* enable to check_urls on arbitrary files

check_urls:free-programming-books.md

* try again

check_urls:free-programming-books.md

* test again

check_urls:free-programming-books.md

* travis doesnt like :

check_urls=free-programming-books.md

* allow arbitrary number of files to be checked

check_urls=free-programming-books.md

* woops forgot to fix the trigger regex

check_urls=free-programming-books.md free-programming-books-en.md

* use for loop instead of xargs

check_urls=free-programming-books.md free-programming-books-en.md

* add documentation