Fixes #9536

This is enough for dynamic libraries, but not executables because MinGW
does not output a .reloc section even with `--dynamicbase`. It could
either be worked around by exporting a DLL symbol from the executable or
fixed in MinGW itself.

This is already enabled by default for x86_64 executables on Windows,
but it needs to be manually enabled on x86.

Closes #16533

For historical reasons, "Win32" has been used in Rust codebase to mean "Windows OS in general".
This is confusing, especially now, that Rust supports Win64 builds.

[breaking-change]

Rust already builds all code as position independent by default, so the
linker can be told to build a position independent executable if it's
not disabled with `-C relocation-model=dynamic-no-pic`. Position
independent code does have a significant cost on i686 (not on x86_64 or
ARM) but there's no significant cost to linking code that's already
position independent as a position independent executable.

Address space layout randomization makes exploiting vulnerabilities much
more difficult by providing a statistical defence against an attempt to
find or modify existing code / data. Without ASLR, it's trivial to use a
vulnerability to take over control of the process via return-oriented
programming.

Rust code can be used for return-oriented programming whether it is safe
or unsafe, so even a fully safe application needs to be built as a
position independent executable to defend against vulnerabilities in
unsafe blocks or C libraries.

Sample program:

    extern crate libc;

    use std::mem;

    static mut global: u32 = 5;
    static constant: u32 = 5;
    fn foo() {}

    fn main() {
        let local = 5;
        println!("stack: {}, global: {}, constant: {}, fn: {}, lib fn: {}",
                 &local as *const u32,
                 unsafe { &global as *const u32 },
                 &constant as *const u32,
                 unsafe { mem::transmute::<_, *const ()>(foo) },
                 unsafe { mem::transmute::<_, *const ()>(libc::mprotect) });
    }

Before:

    stack: 0x3ff15eb9f94, global: 0x6ab488, constant: 0x47db40, fn: 0x4030e0, lib fn: 0x32749547530
    stack: 0x3b5d47d80e4, global: 0x6ab488, constant: 0x47db40, fn: 0x4030e0, lib fn: 0x394469a7530
    stack: 0x3fe2c4e5564, global: 0x6ab488, constant: 0x47db40, fn: 0x4030e0, lib fn: 0x399734a2530
    stack: 0x3e525e0fb24, global: 0x6ab488, constant: 0x47db40, fn: 0x4030e0, lib fn: 0x2f62a810530
    stack: 0x3b50fb3eae4, global: 0x6ab488, constant: 0x47db40, fn: 0x4030e0, lib fn: 0x2e590e86530

After:

    stack: 0x38cf12c90a4, global: 0x3e2d46b488, constant: 0x3e2d23cf80, fn: 0x3e2d1c2510, lib fn: 0x2617d3b4530
    stack: 0x3d733faf474, global: 0x7eb1839488, constant: 0x7eb160af80, fn: 0x7eb1590510, lib fn: 0x32d30c1f530
    stack: 0x3bb42212ec4, global: 0x5bbb365488, constant: 0x5bbb136f80, fn: 0x5bbb0bc510, lib fn: 0x3595e6c1530
    stack: 0x39f678c1ab4, global: 0x22c4e3c488, constant: 0x22c4c0df80, fn: 0x22c4b93510, lib fn: 0x3835b727530
    stack: 0x3afb25bd394, global: 0x493eab2488, constant: 0x493e883f80, fn: 0x493e809510, lib fn: 0x3478d6a7530

This may also be necessary on other platforms, but I can only test on
Linux right now. Note that GDB gained support for debugging position
independent executables in version 7.1 (March 2010).

As discovered in #15460, a particular #[link(kind = "static", ...)] line is not
actually guaranteed to link the library at all. The reason for this is that if
the external library doesn't have any referenced symbols in the object generated
by rustc, the entire library is dropped by the linker.

For dynamic native libraries, this is solved by passing -lfoo for all downstream
compilations unconditionally. For static libraries in rlibs this is solved
because the entire archive is bundled in the rlib. The only situation in which
this was a problem was when a static native library was linked to a rust dynamic
library.

This commit brings the behavior of dylibs in line with rlibs by passing the
--whole-archive flag to the linker when linking native libraries. On OSX, this
uses the -force_load flag. This flag ensures that the entire archive is
considered candidate for being linked into the final dynamic library.

This is a breaking change because if any static library is included twice in the
same compilation unit then the linker will start emitting errors about duplicate
definitions now. The fix for this would involve only statically linking to a
library once.

Closes #15460
[breaking-change]

Before this commit, the LLVM IR of exported items was simply zip-compressed and stored as an object file inside rlib archives. This commit adds a header to this "object" containing a file identifier and a format version number so the compiler can deal with changes in the way bytecode objects are stored within rlibs.

While updating the format of bytecode objects, this commit also worksaround a problem in LLDB which could not handle odd-sized objects within archives before mid-2014.

Sometimes dsymutil writes to stdout/stderr which rust isn't reading, which may
cause a deadlock.

Closes #16060

Not included are two required patches:

* LLVM: segmented stack support for DragonFly [1]

* jemalloc: simple configure patches

[1]: http://reviews.llvm.org/D4705

Closes #15828

When invoking the compiler in parallel, the intermediate output of the object
files and bytecode can stomp over one another if two crates with the same name
are being compiled.

The output file is already being disambiguated with `-C extra-filename`, so this
commit alters the naming of the temporary files to also mix in the extra
filename to ensure that file names don't clash.

Part of the original discussions around the `--crate-name` flag brought up that
mass confusion can arise when the flag specifies a different name than is
contained in the crate.

The current primary use case of the `--crate-name` flag is through cargo and
not requiring a `#[crate_name]` attribute, but if the `#[crate_name]` attribute
is specified it will likely go awry when the two names deviate from one another.
This commit requires that if both are provided they both match to prevent this
confusion.

* Deprecated `to_owned` in favor of `to_vec`
* Deprecated `into_owned` in favor of `into_vec`

[breaking-change]

The default code model is usually unsuitable for kernels,
so we add an option to specify which model we want.

Use `String::from_utf8_lossy` instead

[breaking-change]

Use `String::from_utf8` instead

[breaking-change]

This makes it much saner for clients to use the library since
they don't have to worry about shadowing one llvm with another.

It's just an annoying error if you use --crate-name on the command line and you
also have a #[crate_name] specified

[breaking-change]

In a cargo-driven world the primary location for the name of a crate will be in
its manifest, not in the source file itself. The purpose of this flag is to
reduce required duplication for new cargo projects.

This is a breaking change because the existing --crate-name flag actually
printed the crate name. This flag was renamed to --print-crate-name, and to
maintain consistence, the --crate-file-name flag was renamed to
--print-file-name.

To maintain backwards compatibility, the --crate-file-name flag is still
recognized, but it is deprecated.

[breaking-change]

This comit implements a new flag, --extern, which is used to specify where a
crate is located. The purpose of this flag is to bypass the normal crate
loading/matching of the compiler to point it directly at the right file.

This flag takes the form `--extern foo=bar` where `foo` is the name of a crate
and `bar` is the location at which to find the crate. Multiple `--extern`
directives are allowed with the same crate name to specify the rlib/dylib pair
for a crate. It is invalid to specify more than one rlib or more than one dylib,
and it's required that the crates are valid rust crates.

I have also added some extensive documentation to metadata::loader about how
crate loading should work.

RFC: 0035-remove-crate-id

The compiler will no longer insert a hash or version into a filename by default.
Instead, all output is simply based off the crate name being compiled. For
example, a crate name of `foo` would produce the following outputs:

* bin => foo
* rlib => libfoo.rlib
* dylib => libfoo.{so,dylib} or foo.dll
* staticlib => libfoo.a

The old behavior has been moved behind a new codegen flag,
`-C extra-filename=<hash>`. For example, with the "extra filename" of `bar` and
a crate name of `foo`, the following outputs would be generated:

* bin => foo (same old behavior)
* rlib => libfoobar.rlib
* dylib => libfoobar.{so,dylib} or foobar.dll
* staticlib => libfoobar.a

The makefiles have been altered to pass a hash by default to invocations of
`rustc` so all installed rust libraries will have a hash in their filename. This
is done because the standard libraries are intended to be installed into
privileged directories such as /usr/local. Additionally, it involves very few
build system changes!

RFC: 0035-remove-crate-id
[breaking-change]

This metadata is used to drive the hash of all symbols in a crate. The flag can
be specified a multiple number of times

RFC: 0035-remove-crate-id

This commit removes all support in the compiler for the #[crate_id] attribute
and all of its derivative infrastructure. A list of the functionality removed is:

* The #[crate_id] attribute no longer exists
* There is no longer the concept of a version of a crate
* Version numbers are no longer appended to symbol names
* The --crate-id command line option has been removed

To migrate forward, rename #[crate_id] to #[crate_name] and only the name of the
crate itself should be mentioned. The version/path of the old crate id should be
removed.

For a transitionary state, the #[crate_id] attribute is still accepted if
the #[crate_name] is not present, but it is warned about if it is the only
identifier present.

RFC: 0035-remove-crate-id
[breaking-change]

Being able to index into the bytes of a string encourages
poor UTF-8 hygiene. To get a view of `&[u8]` from either
a `String` or `&str` slice, use the `as_bytes()` method.

Closes #12710.

[breaking-change]

This commit removes superfluous to_string calls from various places

It was previously assumed that the object file generated by LLVM would always
require the __morestack function, but that assumption appears to be incorrect,
as outlined in #15108. This commit forcibly tells the linker to include the
entire archive, regardless of whether it's currently necessary or not.

Closes #15108

This commit disables rustc's emission of rpath attributes into dynamic libraries
and executables by default. The functionality is still preserved, but it must
now be manually enabled via a `-C rpath` flag.

This involved a few changes to the local build system:

* --disable-rpath is now the default configure option
* Makefiles now prefer our own LD_LIBRARY_PATH over the user's LD_LIBRARY_PATH
  in order to support building rust with rust already installed.
* The compiletest program was taught to correctly pass through the aux dir as a
  component of LD_LIBRARY_PATH in more situations.

The major impact of this change is that neither rustdoc nor rustc will work
out-of-the-box in all situations because they are dynamically linked. It must be
arranged to ensure that the libraries of a rust installation are part of the
LD_LIBRARY_PATH. The default installation paths for all platforms ensure this,
but if an installation is in a nonstandard location, then configuration may be
necessary.

Additionally, for all developers of rustc, it will no longer be possible to run
$target/stageN/bin/rustc out-of-the-box. The old behavior can be regained
through the `--enable-rpath` option to the configure script.

This change brings linux/mac installations in line with windows installations
where rpath is not possible.

Closes #11747
[breaking-change]

This commit is the final step in the libstd facade, #13851. The purpose of this
commit is to move libsync underneath the standard library, behind the facade.
This will allow core primitives like channels, queues, and atomics to all live
in the same location.

There were a few notable changes and a few breaking changes as part of this
movement:

* The `Vec` and `String` types are reexported at the top level of libcollections
* The `unreachable!()` macro was copied to libcore
* The `std::rt::thread` module was moved to librustrt, but it is still
  reexported at the same location.
* The `std::comm` module was moved to libsync
* The `sync::comm` module was moved under `sync::comm`, and renamed to `duplex`.
  It is now a private module with types/functions being reexported under
  `sync::comm`. This is a breaking change for any existing users of duplex
  streams.
* All concurrent queues/deques were moved directly under libsync. They are also
  all marked with #![experimental] for now if they are public.
* The `task_pool` and `future` modules no longer live in libsync, but rather
  live under `std::sync`. They will forever live at this location, but they may
  move to libsync if the `std::task` module moves as well.

[breaking-change]

In addition to avoiding 16-byte filenames with bytecode files, this commit also
avoids 16-byte filenames with object files pulled in from native libraries.