Co-authored-by: NMarco Colombo <mar.colombo13@gmail.com>

This ensure that it will run the Windows executable if other files in the directory (such as Linux executables) have the same file name minus the extension.

Signed-off-by: NInfRandomness <infrandomness@gmail.com>

This adds miri to https://doc.rust-lang.org/nightly/nightly-rustc/Signed-off-by: NInfRandomness <infrandomness@gmail.com>

- Add `Interned<Vec<String>>` and use it for tail args
- Refactor `cache.rs` not to need a separate impl for each internable type

This makes it much simpler to add new interned types, rather than having
to add 4+ impl blocks for each type.

```
optimize-tests = false, master
25.98s

optimize-tests = true, master
34.69s

optimize-tests = true, patched
28.79s
```

Effects:

- faster UI tests
- llvm asserts get exercised less on build-pass tests
- the difference between opt and nopt builds shrinks a bit
- aux libs don't get optimized since they don't have a pass mode and almost never have explicit compile flags

Bootstrap currently provides `-Zunstable-options` for platforms
when using split debuginfo - this commit adds it for the illumos
target too.

Previously, it required you to manually run `x build` first, because it
assumed the LLVM binaries were already present.

This reverts the temporary fix implemented by https://github.com/rust-lang/rust/pull/95993 since a permanent fix has been implemented by https://github.com/rust-lang/cargo/pull/10594

Ensure that `static_crt` is set in the bootstrapper whenever using `cc` to get a compiler command line.

When attempting to build rustc with LLVM on Windows, I noticed that the CRT flag provided to the C and C++ Compilers was inconsistent:

```
"-DCMAKE_C_FLAGS=-nologo -MT -Brepro" "-DCMAKE_CXX_FLAGS=-nologo -MD -Brepro"
```

Since the bootstrapper also sets the various `LLVM_USE_CRT` variables, this resulted in cl.exe reporting a bunch of warnings:

```
cl : Command line warning D9025 : overriding '/MD' with '/MT'
```

The root cause for this is that `cc_detect::find` was creating a `cc::Build` twice, but didn't set `static_crt` the second time.

It's possible that this what is also causing #81381

Running steps multiple times defeats the whole point of #96501,
since lint messages will be duplicated.

Create symlinks to workaround file missing error in llvm-config

This is difficult to support without submodule handling in bootstrap.py, because cargo will refuse
to vendor sources unless it knows the Cargo.toml files of all tools in tree. Moving vendor support
to rustbuild means that rustbuild will be built without vendoring.

Rather than trying to solve this, just remove support altogether and require
people to use `rustc-src` if they want vendoring (or run `cargo vendor` manually).

These submodules were previously updated in python because Cargo gives a hard error if toml files
are missing from the workspace:

```
error: failed to load manifest for workspace member `/home/jnelson/rust-lang/rust/src/tools/rls`

Caused by:
  failed to read `/home/jnelson/rust-lang/rust/src/tools/rls/Cargo.toml`

Caused by:
  No such file or directory (os error 2)
failed to run: /home/jnelson/rust-lang/rust/build/x86_64-unknown-linux-gnu/stage0/bin/cargo build --manifest-path /home/jnelson/rust-lang/rust/src/bootstrap/Cargo.toml
```

However, bootstrap doesn't actually need to be part of the workspace.
Remove it so we can move submodule handling fully to Rust, avoiding duplicate code between Rust and Python.

Note that this does break `cargo run`; it has to be `cd src/bootstrap && cargo run` now.
Given that we're planning to make the main entrypoint a shell script (or rust binary),
I think this is a good tradeoff for reduced complexity in bootstrap.py.

This used to take upwards of 5 seconds for me locally. I found that the
culprit was copying the downloaded LLVM shared object:
```
[22:28:03] Install "/home/jnelson/rust-lang/rust/build/x86_64-unknown-linux-gnu/ci-llvm/lib/libLLVM-14-rust-1.62.0-nightly.so" to "/home/jnelson/rust-lang/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libLLVM-14-rust-1.62.0-nightly.so"
[22:28:09]   c Sysroot { compiler: Compiler { stage: 1, host: x86_64-unknown-linux-gnu(x86_64-unknown-linux-gnu) } }
```

It turned out that `install()` used full copies unconditionally. Change
it to use `copy()` internally, which uses hard links instead when
available.

Note that this has a change in behavior: Installing a file will also
change permissions on the source, not just the destination, if hard
links are used.

To avoid changing the behavior on symlinks for existing code, I
introduce a new function `copy_internal` which only dereferences
symlinks when told to do so.

This avoids regressions in rustup-toolchain-install-master

This was surprisingly complicated. The main changes are:
1. Invert the order of iteration in `StepDescription::run`.

    Previously, it did something like:
    ```python
    for path in paths:
    for (step, should_run) in should_runs:
        if let Some(set) = should_run.pathset_for_path(path):
        step.run(builder, set)
    ```

    That worked ok for individual paths, but didn't allow passing more than one path at a time to `Step::run`
    (since `pathset_for_paths` only had one path available to it).
    Change it to instead look at the intersection of `paths` and `should_run.paths`:

    ```python
    for (step, should_run) in should_runs:
    if let Some(set) = should_run.pathset_for_paths(paths):
        step.run(builder, set)
    ```

2. Change `pathset_for_path` to take multiple pathsets.

    The goal is to avoid `x test library/alloc` testing *all* library crates, instead of just alloc.
    The changes here are similarly subtle, to use the intersection between the paths rather than all
    paths in `should_run.paths`. I added a test for the behavior to try and make it more clear.

    Note that we use pathsets instead of just paths to allow for sets with multiple aliases (*cough* `all_krates` *cough*).
    See the documentation added in the next commit for more detail.

3. Change `StepDescription::run` to explicitly handle 0 paths.

   Before this was implicitly handled by the `for` loop, which just didn't excute when there were no paths.
   Now it needs a check, to avoid trying to run all steps (this is a problem for steps that use `default_condition`).

4. Change `RunDescription` to have a list of pathsets, rather than a single path.

5. Remove paths as they're matched

   This allows checking at the end that no invalid paths are left over.
   Note that if two steps matched the same path, this will no longer run both;
   but that's a bug anyway.

6. Handle suite paths separately from regular sets.

   Running multiple suite paths at once instead of in separate `make_run` invocations is both tricky and not particularly useful.
   The respective test Steps already handle this by introspecting the original paths.

   Avoid having to deal with it by moving suite handling into a seperate loop than `PathSet::Set` checks.

As was discovered in https://github.com/rust-lang/rust/pull/93628#issuecomment-1154697627 ,
adding #[cfg(bootstrap)] to a rust-internal proc macro crate
would yield an unexpected cfg name error, at least on later
stages wher the bootstrap cfg arg wasn't set.

rustc already passes arguments to mark bootstrap as expected,
however the means of delivery through the RUSTFLAGS env var
is unable to reach proc macro crates, as described
in the issue linked in the code this commit touches.

This wouldn't be an issue for cfg args that get passed through
RUSTFLAGS, as they would never become *active* either, so
any usage of one of these flags in a proc macro's code would
legitimately yield a lint warning. But since dc302587,
rust takes extra measures to pass --cfg=bootstrap even in
proc macros, by passing it via the wrapper. Thus, we need
to send the flags to mark bootstrap as expected also from the
wrapper, so that #[cfg(bootstrap)] also works from proc macros.

I want to thank Urgau and jplatte for helping me find the cause of this. ❤️

Fixes: Error on bootstrapping : Empty search path given via '-L' (solaris) #97260

This commit allows users to change the contents of the "config" key in
src/stage0.json without having it overridden the next time the
bump-stage0 tool is executed.

This reverts commit 2d854f9c.