ignore test if rust-lld not found

create ld -> rust-lld symlink at build time instead of run time

for testing in ci

copy instead of symlinking

remove linux check

test for linker, suggestions from bjorn3

fix overly restrictive lld matcher

use -Zgcc-ld flag instead of -Clinker-flavor

refactor code adding lld to gcc path

revert ci changes

suggestions from petrochenkov

rename gcc_ld to gcc-ld in dirs

The session-derive-errors test ensures the internal SessionDiagnostic
derive macro outputs the right error messages when misused.

The macro relies on the proc_macro2 crate though, which changes its span
behavior depending on whether the channel is nightly or not. This caused
test failures when bumping the channel from nightly to beta/stable.

Since SessionDiagnostic is internal-only we don't care about its
diagnostics quality outside of nightly, as the compiler itself is
developed on nightly. Thus the easiest solution is to ignore that test
on the beta and stable channels.

This also implements `// only-{channel}` and `// ignore-{channel}` in
compiletest to properly support the change.

- Pass it explicitly where appropriate
- Update stderr files and warnings; it turns that unstable-options has
  far-reaching effects on diagnostics.

CFG_PREFIX is incorrect for rustup installed rustc versions. It also
causes unnecessary recompilation when changing the install prefix.

Cargo ignores RUSTFLAGS when building proc macro crates. However,
sometimes rustc_macro needs to have conditional compilation when there
are breaking changes to the `libproc_macro` API (see for example
tell the difference between stage 0 and stage 1.

Another alternative is to unconditionally build rustc_macros with the
master libstd instead of the beta one (i.e. use `--sysroot
stage0-sysroot`), but that led to strange and maddening errors:

```
error[E0460]: found possibly newer version of crate `std` which `proc_macro2` depends on
  --> /home/joshua/.local/lib/cargo/registry/src/github.com-1ecc6299db9ec823/tracing-attributes-0.1.13/src/lib.rs:90:5
   |
90 | use proc_macro2::TokenStream;
   |     ^^^^^^^^^^^
   |
   = note: perhaps that crate needs to be recompiled?
   = note: the following crate versions were found:
           crate `std`: /home/joshua/rustc2/build/x86_64-unknown-linux-gnu/stage0-sysroot/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-b3602c301b71cc3d.rmeta
           crate `proc_macro2`: /home/joshua/rustc2/build/x86_64-unknown-linux-gnu/stage0-rustc/release/deps/libproc_macro2-a83c1f01610c129e.rlib
```

 ## User-facing changes

- Intra-doc links to primitives that currently go to rust-lang.org/nightly/std/primitive.x.html will start going to channel that rustdoc was built with. Nightly will continue going to /nightly; Beta will link to /beta; stable compilers will link to /1.52.1 (or whatever version they were built as).
- Cross-crate links from std to core currently go to /nightly unconditionally. They will start going to /1.52.0 on stable channels (but remain the same on nightly channels).
- Intra-crate links from std to std (or core to core) currently go to the same URL they are hosted at; they will continue to do so. Notably, this is different from everything else because it can preserve the distinction between /stable and /1.52.0 by using relative links.

Note that "links" includes both intra-doc links and rustdoc's own
automatically generated hyperlinks.

 ## Implementation changes

- Update the testsuite to allow linking to /beta and /1.52.1 in docs
- Use an html_root_url for the standard library that's dependent on the channel

  This avoids linking to nightly docs on stable.

- Update rustdoc to use channel-dependent links for primitives from an
  unknown crate

- Set DOC_RUST_LANG_ORG_CHANNEL from bootstrap to ensure it's in sync
- Include doc.rust-lang.org in the channel

Previously, changing the standard library with `download-rustc =
"if-unchanged"` would incorrectly reuse the cached compiler and standard
library from CI, which was confusing and led to incorrect test failures
or successes.

Co-authored-by: NJosh Triplett <josh@joshtriplett.org>

These requirements change as soon as the command finishes running, and
`setup` doesn't build anything, so the check doesn't make sense.

Previously, `x.py setup` would give hard errors if `ninja` and `cmake`
were not installed, even if the new profile didn't require them.

We do not install Zlib on the CI but recent builds somehow started picking it's shared version.
To avoid relying on CI binaries so let's explicitly disable it.

Recent commits to cc have helped to address #83043 and #43468

This change adds the bpfel-unknown-none and bpfeb-unknown-none targets
which can be used to generate little endian and big endian BPF

This enables better caching, since LLVM is only updated when needed, not
whenever x.py is run. Before, bootstrap.py had to use heuristics to
guess if LLVM would be needed, and updated the module more often than
necessary as a result.

This syncs the LLVM submodule only just before building the compiler, so
people working on the standard library never have to worry about it.
Example output:

```
Copying stage0 std from stage0 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / x86_64-unknown-linux-gnu)
Updating submodule src/llvm-project
Submodule 'src/llvm-project' (https://github.com/rust-lang/llvm-project.git) registered for path 'src/llvm-project'
Submodule path 'src/llvm-project': checked out 'f9a8d70b6e0365ac2172ca6b7f1de0341297458d'
```

- Don't try to update the LLVM submodule when using system LLVM

  Previously, this would try to update LLVM unconditionally. Now the
  submodule is only initialized if `llvm-config` is not set.

- Don't update LLVM submodule in dry runs

  This prevents the following test failures:

  ```
  running 17 tests
  fatal: invalid gitfile format: /checkout/src/llvm-project/.git
  test builder::tests::defaults::build_cross_compile ... FAILED

  ---- builder::tests::defaults::build_default stdout ----
  thread 'main' panicked at 'command did not execute successfully: "git" "rev-parse" "HEAD"
  expected success, got: exit code: 128', src/build_helper/lib.rs:139:9
  ```

- Try running git without --progress if it fails the first time

  This avoids having to do version detection to see if --progress is
  supported or not.

- Don't try to update submodules when the source repository isn't managed by git

- Update LLVM submodules that have already been checked out

- Only check for whether the submodule should be updated in lib.rs; update
it unconditionally in native.rs

This bug was only visible on mac. Also, print_step_rusage is a relatively new
internal feature, that is not heavily used, and has no tests. All of these
factors contributed to how this went uncaught this long. Thanks to Josh Triplett
for pointing it out!

Previously, setting `download-ci-llvm = true` when cmake wasn't
installed would give the following error:

```
failed to execute command: "cmake" "--help"
error: The system cannot find the file specified. (os error 2)
```

This fixes the following error:

```
error: found crates (`serde_derive` and `serde_derive`) with colliding StableCrateId values.
   --> /home/joshua/.local/lib/cargo/registry/src/github.com-1ecc6299db9ec823/cargo_metadata-0.8.2/src/lib.rs:162:1
    |
162 | extern crate serde_derive;
    | ^^^^^^^^^^^^^^^^^^^^^^^^^^
```

This is a bug in resolve (https://github.com/rust-lang/rust/issues/56935)
but it will be difficult to fix in the near future. This works around it
in the meantime by not copying serde_derive and other dependencies to
the sysroot when they're built for other tools. This rebuilds the
dependencies slightly more often than necessary, but avoids the crate
conflicts.

This can be reverted once #56935 is fixed.

- Add rustfmt to `x.py check`
- Update Cargo.lock
- Remove rustfmt from the toolstate list
- Make rustfmt an in-tree tool
- Give an error on `x.py test rustfmt` if rustfmt fails to build or if tests fail
- Don't call `save_toolstate` when testing rustfmt

To pull in this fix: https://github.com/alexcrichton/cc-rs/commit/801a87bf2f31ad1ad8bd7e8fa4f5a52b0e2b4c00

Now that stdarch has been updated, we can do this without breaking the
build.

MinGW driver for COFF LLD doesn't currently translate GNU-style `--threads=N` to native `/threads:N`, so we have to pass the option in its native form to avoid an error.

Also pass the `threads` flag to lld-link as well