Set tokens on AST node in `collect_tokens`

A new `HasTokens` trait is introduced, which is used to move logic from
the callers of `collect_tokens` into the body of `collect_tokens`.

In addition to reducing duplication, this paves the way for PR #80689,
which needs to perform additional logic during token collection.

Update RLS and Rustfmt

Fixes #80576

Updates Rustfmt to use `rustfmt-v1.4.31` branch. Both are updated (along with `racer`) in tandem to pull in the exact same version of rustc-ap-* libraries.

r? `@calebcartwright`

Rollup of 17 pull requests

Successful merges:

 - #79982 (Add missing methods to unix ExitStatusExt)
 - #80017 (Suggest `_` and `..` if a pattern has too few fields)
 - #80169 (Recommend panic::resume_unwind instead of panicking.)
 - #80217 (Add a `std::io::read_to_string` function)
 - #80444 (Add as_ref and as_mut methods for Bound)
 - #80567 (Add Iterator::intersperse_with)
 - #80829 (Get rid of `DepConstructor`)
 - #80895 (Fix handling of malicious Readers in read_to_end)
 - #80966 (Deprecate atomic::spin_loop_hint in favour of hint::spin_loop)
 - #80969 (Use better ICE message when no MIR is available)
 - #80972 (Remove unstable deprecated Vec::remove_item)
 - #80973 (Update books)
 - #80980 (Fixed incorrect doc comment)
 - #80981 (Fix -Cpasses=list and llvm version print with -vV)
 - #80985 (Fix stabilisation version of slice_strip)
 - #80990 (llvm: Remove the unused context from CreateDebugLocation)
 - #80991 (Fix formatting specifiers doc links)

Failed merges:

 - #80944 (Use Option::map_or instead of `.map(..).unwrap_or(..)`)

r? `@ghost`
`@rustbot` modify labels: rollup

Fix formatting specifiers doc links

d36e3e23 seems to have inadvertently changed many of these links to point to `core::fmt` instead of `std::fmt`. The information about formatting specifiers is only documented in [`std::fmt`](https://doc.rust-lang.org/std/fmt/); [`core::fmt`](https://doc.rust-lang.org/core/fmt/) is empty. 3baf6a4a seems to have already fixed a couple of these links to point back to `std::fmt`.

llvm: Remove the unused context from CreateDebugLocation

This went unused in commit 88d874de, part of #68965.

Fix stabilisation version of slice_strip

See https://github.com/rust-lang/rust/pull/77853#pullrequestreview-564921079Signed-off-by: NIan Jackson <ijackson@chiark.greenend.org.uk>

Fix -Cpasses=list and llvm version print with -vV

cc https://github.com/rust-lang/rust/pull/77975#issuecomment-759362933

Fixed incorrect doc comment

">" is right alignment, not left

Update books

## nomicon

2 commits in a5a48441d411f61556b57d762b03d6874afe575d..a8584998eacdea7106a1dfafcbf6c1c06fcdf925
2020-12-06 10:39:41 +0900 to 2021-01-06 12:49:49 -0500
- Update vector code examples
- Remove outdated information about `jemalloc`

## reference

13 commits in b278478b766178491a8b6f67afa4bcd6b64d977a..50af691f838937c300b47812d0507c6d88c14f97
2020-12-21 18:18:03 -0800 to 2021-01-12 21:19:20 -0800
- Update grammar for parser unification. (rust-lang/reference#927)
- Define constraining an implementation (rust-lang/reference#928)
- Document extra behavior of #[no_mangle] (rust-lang/reference#930)
- Add a float examle without a `.`. (rust-lang/reference#929)
- Add more details about const generics. (rust-lang/reference#921)
- Fix footnotes. (rust-lang/reference#926)
- Add "Logic errors" as behavior not considered unsafe (rust-lang/reference#919)
- Update grammar for order of parameters/arguments. (rust-lang/reference#920)
- Fix formatting in the tuple section (rust-lang/reference#923)
- document const generics (rust-lang/reference#901)
- Update mdbook (rust-lang/reference#918)
- linkage.md: update link to FFI section of the Book. (rust-lang/reference#917)
- Document array expression with a const. (rust-lang/reference#914)

## book

8 commits in 5bb44f8b5b0aa105c8b22602e9b18800484afa21..ac57a0ddd23d173b26731ccf939f3ba729753275
2020-12-18 20:07:31 -0500 to 2021-01-09 14:18:45 -0500
- Update version of mdbook we're testing with to 0.4.5 (rust-lang/book#2561)
- Fix grammar in ch13-01-closures.md (rust-lang/book#2534)
- Merge remote-tracking branch 'origin/pr/2527'
- Clarify code example ch6.3 (rust-lang/book#2485)
- Fix link added in rust-lang/book#2495 to be relative and at the bottom
- Merge remote-tracking branch 'origin/pr/2495'
- Update output to match the updated poem punctuation
- Fix rust-lang/book#2539 - Remove fancy apostrophes from poem for Windows

## rust-by-example

3 commits in 1cce0737d6a7d3ceafb139b4a206861fb1dcb2ab..03e23af01f0b4f83a3a513da280e1ca92587f2ec
2020-12-21 17:36:29 -0300 to 2021-01-09 10:20:28 -0300
- Replace for loop with iteration (rust-lang/rust-by-example#1404)
- Update mdbook (rust-lang/rust-by-example#1402)
- Add note for match guards to include catch-all (rust-lang/rust-by-example#1401)

## embedded-book

1 commits in ba34b8a968f9531d38c4dc4411d5568b7c076bfe..ceec19e873be87c6ee5666b030c6bb612f889a96
2020-11-17 00:20:43 +0000 to 2021-01-03 13:13:10 +0000
- book.toml: add link to GitHub repo  (rust-embedded/book#276)

Remove unstable deprecated Vec::remove_item

Closes #40062

The `Vec::remove_item` method was deprecated in `1.46.0` (in August of 2020). This PR now removes that unstable method entirely.

Use better ICE message when no MIR is available

The ICE message is somewhat confusing and overly specific - the issue is
that there's no MIR available.

This should make debugging these ICEs easier since the error tells you
what's actually wrong, not what it was trying to do when it failed.

cc https://github.com/rust-lang/rust/pull/80952#issuecomment-759198841
cc `````@jyn514`````

Deprecate atomic::spin_loop_hint in favour of hint::spin_loop

For https://github.com/rust-lang/rust/issues/55002

We wanted to leave `atomic::spin_loop_hint` alone when stabilizing `hint::spin_loop` so folks had some time to migrate. This now deprecates `atomic_spin_loop_hint`.

Fix handling of malicious Readers in read_to_end

A malicious `Read` impl could return overly large values from `read`, which would result in the guard's drop impl setting the buffer's length to greater than its capacity! ~~To fix this, the drop impl now uses the safe `truncate` function instead of `set_len` which ensures that this will not happen. The result of calling the function will be nonsensical, but that's fine given the contract violation of the `Read` impl.~~

~~The `Guard` type is also used by `append_to_string` which does not pass untrusted values into the length field, so I've copied the guard type into each function and only modified the one used by `read_to_end`. We could just keep a single one and modify it, but it seems a bit cleaner to keep the guard code close to the functions and related specifically to them.~~

To fix this, we now assert that the returned length is not larger than the buffer passed to the method.

For reference, this bug has been present for ~2.5 years since 1.20: https://github.com/rust-lang/rust/commit/ecbb896b9eb2acadefde57be493e4298c1aa04a3.

Closes #80894.

Get rid of `DepConstructor`

This removes fully 235 unused functions.

Follow-up to https://github.com/rust-lang/rust/pull/80325#discussion_r548491999.

r? ``@michaelwoerister``
cc ``@cjgillot``

Add Iterator::intersperse_with

This is a follow-up to #79479, tracking in #79524, as discussed https://github.com/rust-lang/rust/pull/79479#issuecomment-752671731.

~~Note that I had to manually implement `Clone` and `Debug` because `derive` insists on placing a `Clone`-bound on the struct-definition, which is too narrow. There is a long-standing issue # for this somewhere around here :-)~~

Also, note that I refactored the guts of `Intersperse` into private functions and re-used them in `IntersperseWith`, so I also went light on duplicating all the tests.

If this is suitable to be merged, the tracking issue should be updated, since it only mentions `intersperse`.

Happy New Year!

r? ``@m-ou-se``

Add as_ref and as_mut methods for Bound

Add as_ref and as_mut method for std::ops::range::Bound, patterned off
of the methods of the same name on Option.

I'm not quite sure what the process is for introducing new feature gates (this is my first contribution) so I've left these ungated, but happy to do whatever is necessary to gate them.

Add a `std::io::read_to_string` function

I recognize that you're usually supposed to open an issue first, but the
implementation is very small so it's okay if this is closed and it was 'wasted
work' :)

-----

The equivalent of `std::fs::read_to_string`, but generalized to all
`Read` impls.

As the documentation on `std::io::read_to_string` says, the advantage of
this function is that it means you don't have to create a variable first
and it provides more type safety since you can only get the buffer out
if there were no errors. If you use `Read::read_to_string`, you have to
remember to check whether the read succeeded because otherwise your
buffer will be empty.

It's friendlier to newcomers and better in most cases to use an explicit
return value instead of an out parameter.

Recommend panic::resume_unwind instead of panicking.

Fixes https://github.com/rust-lang/rust/issues/79950.

Suggest `_` and `..` if a pattern has too few fields

Fixes #80010.

Add missing methods to unix ExitStatusExt

These are the methods corresponding to the remaining exit status examination macros from `wait.h`.  `WCOREDUMP` isn't in SuS but is it is very standard.  I have not done portability testing to see if this builds everywhere, so I may need to Do Something if it doesn't.

There is also a bugfix and doc improvement to `.signal()`, and an `.into_raw()` accessor.

This would fix #73128 and fix #73129.  Please let me know if you like this direction, and if so I will open the tracking issue and so on.

If this MR goes well, I may tackle #73125 next - I have an idea for how to do it.

Update tests of "unused_lifetimes" lint for async functions and corresponding source code

Before this PR the following code would cause an error:
```
#![deny(unused_lifetimes)]
async fn f<'a>(_: &'a i32) {}
fn main() {}
```
It was happening because of the desugaring of return type in async functions. As a result of the desugaring, the return type contains all lifetimes involved in the function signature. And these lifetimes were interpreted separately from the same in the function scope => so they are unused.

Now, all lifetimes from the return type are interpreted as used. It is also not perfect, but at least this lint doesn't cause wrong errors now.

This PR connected to issues #78522, #77217

Reintroduce hir::ExprKind::If

Basically copied and paste #59288/https://github.com/rust-lang/rust-clippy/pull/4080 with some modifications.

The vast majority of tests were fixed and now there are only a few remaining. Since I am still unable to figure out the missing pieces, any help with the following list is welcome.

- [ ] **Unnecessary `typeck` exception**: [Cheated on this one to make CI green.](https://github.com/rust-lang/rust/pull/79328/files#diff-3faee9ba23fc54a12b7c43364ba81f8c5660045c7e1d7989a02a0cee1c5b2051)
- [x] **Incorrect span**: [Span should reference `then` and `else` separately.](https://github.com/rust-lang/rust/pull/79328/files#diff-cf2c46e82222ee4b1037a68fff8a1af3c4f1de7a6b3fd798aacbf3c0475abe3d)
- [x] **New note regarding `assert!`**: [Modified but not "wrong". Maybe can be a good thing?](https://github.com/rust-lang/rust/pull/79328/files#diff-9e0d7c89ed0224e2b62060c957177c27db43c30dfe3c2974cb6b5091cda9cfb5)
- [x] **Inverted report location**: [Modified but not "wrong". Locations were inverted.](https://github.com/rust-lang/rust/pull/79328/files#diff-f637ce7c1f68d523a165aa9651765df05e36c4d7d279194b1a6b28b48a323691)
- [x] **`src/test/ui/point-to-type-err-cause-on-impl-trait-return.rs` has weird errors**: [Not sure why this is happening.](https://github.com/rust-lang/rust/pull/79328/files#diff-c823c09660f5b112f95e97e8ff71f1797b6c7f37dbb3d16f8e98bbaea8072e95)
- [x] **Missing diagnostic**: [???](https://github.com/rust-lang/rust/pull/79328/files#diff-6b8ab09360d725ba4513933827f9796b42ff9522b0690f80b76de067143af2fc)

Encode optimized MIR of generators when emitting metadata

Fix #49660 - Adds checks to ensure existence of arithmetic trait implementations

The first 2 commits fix an issue with the existing `wrapping.rs` tests. It wasn't referred to from the module, so the file was being ignored. This is fixed in https://github.com/rust-lang/rust/pull/78259/commits/872dc60ed203d16d43140c1d1623474cf8a9aedc This surfaced a bug in its macro which is fixed in https://github.com/rust-lang/rust/pull/78259/commits/8ddad18283e304753e09ef651209b4a6b54148b0

Lastly, commit https://github.com/rust-lang/rust/pull/78259/commits/64d695b753bfe09797b5f95c7b9b72948da01710 is the actual tests for fixing #49660

The following checks are done:

* `Add`, `Sub`, `Mul`, `Div`, `Rem`
  * `T op T`, `T op &T`, `&T op T` and `&T op &T`
  * for all integer and floating point types
* `AddAssign`, `SubAssign`, `MulAssign`, `DivAssign`, `RemAssign`
  * `&mut T op T` and `&mut T op &T`
  * for all integer and floating point types
* `Neg`
  * `op T` and `op &T`
  * for all signed integer and floating point types
* `Not`
  * `op T` and `op &T`
  * for `bool`
* `BitAnd`, `BitOr`, `BitXor`
  * `T op T`, `T op &T`, `&T op T` and `&T op &T`
  * for all integer types and bool
* `BitAndAssign`, `BitOrAssign`, `BitXorAssign`
  * `&mut T op T` and `&mut T op &T`
  * for all integer types and bool
* `Shl`, `Shr`
  * `L op R`, `L op &R`, `&L op R` and `&L op &R`
  * for all pairs of integer types
* `ShlAssign`, `ShrAssign`
  * `&mut L op R`, `&mut L op &R`
  * for all pairs of integer types

NOTE: I'd like some feedback on improving the macros. I'm not familiar with the idioms and patterns there and composing them has been a challenge for me.

[EDIT]: updated links to commits after rebase.

These tests invoke the various op traits using all accepted types they
are implemented for as well as for references to those types.

This fixes #49660 and ensures the following implementations exist:

* `Add`, `Sub`, `Mul`, `Div`, `Rem`
  * `T op T`, `T op &T`, `&T op T` and `&T op &T`
  * for all integer and floating point types
* `AddAssign`, `SubAssign`, `MulAssign`, `DivAssign`, `RemAssign`
  * `&mut T op T` and `&mut T op &T`
  * for all integer and floating point types
* `Neg`
  * `op T` and `op &T`
  * for all signed integer and floating point types
* `Not`
  * `op T` and `op &T`
  * for `bool`
* `BitAnd`, `BitOr`, `BitXor`
  * `T op T`, `T op &T`, `&T op T` and `&T op &T`
  * for all integer types and bool
* `BitAndAssign`, `BitOrAssign`, `BitXorAssign`
  * `&mut T op T` and `&mut T op &T`
  * for all integer types and bool
* `Shl`, `Shr`
  * `L op R`, `L op &R`, `&L op R` and `&L op &R`
  * for all pairs of integer types
* `ShlAssign`, `ShrAssign`
  * `&mut L op R`, `&mut L op &R`
  * for all pairs of integer types

AFAIK it isn't currently possible to do this. It is also more in line with other tests in the surrounding modules.

A new `HasTokens` trait is introduced, which is used to move logic from
the callers of `collect_tokens` into the body of `collect_tokens`.

In addition to reducing duplication, this paves the way for PR #80689,
which needs to perform additional logic during token collection.

Box Item::Attributes

This reduces the size of Item from 128 to 40 bytes. I think this is as small as it needs to get 🎉

Builds on https://github.com/rust-lang/rust/pull/80339 and should not be merged before.

r? `@GuillaumeGomez`

Add as_ref and as_mut method for std::ops::range::Bound, patterned off
of the methods of the same name on Option.

Properly handle `SyntaxContext` of dummy spans in incr comp

Fixes #80336

Due to macro expansion, we may end up with spans with an invalid
location and non-root `SyntaxContext`. This commits preserves the
`SyntaxContext` of such spans in the incremental cache, and ensures
that we always hash the `SyntaxContext` when computing the `Fingerprint`
of a `Span`

Previously, we would discard the `SyntaxContext` during serialization to
the incremental cache, causing the span's `Fingerprint` to change across
compilation sessions.

Rework diagnostics for wrong number of generic args (fixes #66228 and #71924)

This PR reworks the `wrong number of {} arguments` message, so that it provides more details and contextual hints.

Fixes #80336

Due to macro expansion, we may end up with spans with an invalid
location and non-root `SyntaxContext`. This commits preserves the
`SyntaxContext` of such spans in the incremental cache, and ensures
that we always hash the `SyntaxContext` when computing the `Fingerprint`
of a `Span`

Previously, we would discard the `SyntaxContext` during serialization to
the incremental cache, causing the span's `Fingerprint` to change across
compilation sessions.

Was incorrectly linked to `core::fmt`, which is empty, in d36e3e23
Some of the links were fixed already in 3baf6a4a